Rating the learner is an important part of language teaching. Tests can be used to place students in courses according to their knowledge, to diagnose their strengths and weaknesses, to identify their proficiency etc. There are many theories about what tests should analyse and how they should be taken. Depending on the relevance of the results, the ratings need to be more or less accurate. Furthermore, they should have a beneficial backwash on teaching. It is thus at least as important to examine rating itself as it is to look at teaching methods.
As there are many different ways of rating language students, this work will concentrate on two specific approaches: rating scales and Rapid Profile. Whereas the first have been in use for a while, Rapid Profile is a newly developed computer programme which has not been discussed very much so far. It is thus particularly interesting to compare the two rating methods in order to investigate their advantages and disadvantages.
Table of Contents
1 Introduction
2 Language Testing: Requirements, Types and Properties
2.1. Test requirements
2.2. Types of tests
2.3. Test properties
3 Rating Scales
4 Rapid Profile
4.1. What is Rapid Profile?
4.2. The theory behind it
5 Rating Scales vs. Rapid Profile
6 Conclusion
Objectives and Topics
This paper aims to compare two distinct methods for assessing English language learners—traditional rating scales and the computer-based Rapid Profile tool—to determine their effectiveness, reliability, and pedagogical impact.
- Foundational principles of language testing (reliability and validity).
- Methodological differences between rating scales and software-assisted profiling.
- The theoretical basis of Processability Theory in language acquisition.
- Comparative analysis of objectivity and inter-rater reliability.
- Evaluation of "backwash" effects on classroom teaching practices.
Excerpt from the Book
4.2. The theory behind it
The programme is based on Processability Theory as conceptualised by Manfred Pienemann in 1998. Based on “the architecture of the human language processor”, this theory focuses on the developmental problem which deals with the steps in which language acquisition occurs and their repercussion on it. Processability Theory states that a language is learned in stages by acquiring the necessary procedural skills. These stages are marked by specific key structures which are ordered in a hierarchy. Features belonging to a higher level cannot be processed unless the previous ones have been acquired. Thus, stages cannot be left out as they depend on each other, the lower ones constituting the basis for the top ones.
In this way, the way in which the language acquisition takes place can be predicted. An early learner of English will only be capable of speaking in one-word structures, whereas the competent learner will be able to form more complex sentences. Processability Theory states that once a structure has been learned it does not have to be repeated again, which permits the learner to focus on the next structure. If a learner makes mistakes because he is in a lower stage than the structure he is trying to express, this feature should not be taught until he is developmentally ready.
Summary of Chapters
1 Introduction: This chapter outlines the importance of learner assessment and establishes the research goal of comparing traditional rating scales with the computer-based Rapid Profile.
2 Language Testing: Requirements, Types and Properties: This section details the fundamental requirements for effective testing, including reliability, validity, and the various categories of language tests.
3 Rating Scales: This chapter examines the structure, usage, and development of rating scales, highlighting the role of level descriptors and the common challenges regarding rater subjectivity.
4 Rapid Profile: This chapter introduces the Rapid Profile programme, explaining its computer-based assessment method, its user interface, and the training requirements for analysts.
5 Rating Scales vs. Rapid Profile: This chapter provides a critical comparison of both methods, evaluating them based on their objectivity, flexibility, and pedagogical utility.
6 Conclusion: This chapter synthesizes the findings, concluding that Rapid Profile offers superior reliability and objective assessment for early-stage language learners compared to traditional rating scales.
Keywords
Language Testing, Rating Scales, Rapid Profile, Processability Theory, Second Language Acquisition, Reliability, Validity, Proficiency, Diagnostic Testing, Computer-based Assessment, Pedagogical Backwash, Learner Assessment, Linguistic Profiling, Inter-rater Reliability, Language Stages.
Frequently Asked Questions
What is the primary focus of this research paper?
The paper focuses on evaluating and comparing two different approaches to assessing language learners: traditional manual rating scales and the newly developed computer-based programme known as Rapid Profile.
What are the central themes discussed in the work?
The central themes include the principles of language testing (reliability/validity), the practical application of rating scales, the technical function of the Rapid Profile software, and the theoretical underpinnings of Processability Theory.
What is the main objective of the study?
The study aims to investigate the advantages and disadvantages of both assessment methods to determine which one provides more reliable and valid insights into a learner's communicative competence.
Which scientific methodology is used for the evaluation?
The work employs a comparative, analytical methodology, juxtaposing the practical application of rating scales against the computer-automated profiling system while referencing established theories of second language acquisition.
What does the main body of the text cover?
The main body covers the theoretical foundations of language testing, the mechanisms of rating scales, the operational steps of Rapid Profile, and a comparative critique of how these methods influence classroom teaching.
Which keywords characterize this work?
Key terms include Language Testing, Rapid Profile, Processability Theory, Reliability, Validity, and Pedagogical Backwash.
How does Processability Theory influence the Rapid Profile programme?
Processability Theory provides the hierarchy for the software; it posits that language is acquired in fixed, sequential stages, allowing the software to determine exactly which structures a learner is developmentally prepared to master.
Why is subjectivity considered a major issue in traditional rating scales?
Subjectivity is problematic because human raters often have different interpretations of level descriptors and standards of severity, which can lead to inconsistent results and low inter-rater reliability.
What is the "emergence criterion" mentioned in the study?
The emergence criterion is a principle used by Rapid Profile stating that a linguistic structure only needs to appear in the learner's output—even inconsistently—to demonstrate that the learner has attained a specific developmental stage.
- Quote paper
- Ana Colton-Sonnenberg (Author), 2006, Rating learners of English as a foreign language: Rating Scales vs. Rapid Profile , Munich, GRIN Verlag, https://www.grin.com/document/71209
