Homeworks academic service


Assessment and classroom based assessment english language essay

English as a Second Language Writing and Automated Essay Evaluation Introduction Much of the published work on automated scoring of writing has focused on writing instruction and assessment in K—16 education in the United States, with the implicit assumption that the writers being assessed have native or native-like command of English.

In this context, English language learners ELLs appear to be of somewhat peripheral concern. However, readers may be surprised to find out that there are more people learning and communicating in English in the world than there are native speakers Kachru, 1997 ; Assessment and classroom based assessment english language essay, 2002 ; some are immigrants or children of immigrants in the United States U.

Given the importance of written communication for global education and business and thus the need to teach and assess writing throughout the world, the interest in reliable and efficient automated scoring systems for assessing the writing of ELLs is increasing, and the applicability of automated essay scoring AES systems to non-native speakers NNSs 1 of English is an ever-more important concern.

One might even argue that the largest market for automated scoring of English writing is not in assessing the writing ability of native speakers, but rather that of NNSs of English. The goal of this chapter is to lay out some groundwork for understanding the potential place for AES systems for ELLs, whether they are in school with their native speaker peers in the U.

The chapter is organized as follows. Then, I discuss the construct of writing for different populations of ELLs, specifically with regard to the relative importance of second language proficiency and writing ability in different contexts, and current trends in teaching writing to ELLs. Next, I describe briefly the two main functions of computer-assisted writing evaluation: Finally, I discuss considerations for implementing automated scoring and automated feedback in different contexts.

Contexts for Teaching and Assessing ELL Writing Second language writing instruction and assessment varies widely depending on the context and population of concern. Many NNSs in schools and universities in the U. Even if these students frequently do not have complete control over English vocabulary and grammar, their language proficiency is presumed to be strong enough so that the focus of assessment can be writing per se, not language proficiency as demonstrated through writing.

The main question for these students is whether assessments designed for native speakers, whether scored by human raters or computers, are valid and fair for NNSs. The other main writing assessment purpose for ELLs in English-speaking countries is to evaluate English language proficiency through writing. In these settings, there is a greater focus on control over syntax and vocabulary, along with rhetorical concerns such as development and organization.

In K—12 and university settings, NNSs are often tested to determine whether they need additional English language services before or concurrent with their regular program of study.

For this population, the focus of the assessment tends to be language proficiency as demonstrated through writing, rather than strictly writing skills per se, especially at lower levels of proficiency. In EFL contexts, there are three main purposes for assessing English writing ability. A large testing industry has grown around the need to assess English language proficiency for students coming to the U. Secondly, English is a requirement for university students in many countries.

Third, there is a great deal of international interest in developing English language tests for workplace certification, particularly tests aligned with the Common European Frame of Reference CEFR standards Europe, 2001 that have been promulgated over the past ten years.

English examinations have fairly high stakes for students, as their future may depend on their test scores.

Chapter 1. What is Performance-Based Learning and Assessment, and Why is it Important?

Thus, test preparation is a large industry in many places, including general English courses or courses specifically tailored towards preparation for a specific exam.

To the extent that writing is a central component of the examination, writing will be part of the curriculum. In all of these situations, the primary purpose of these assessments is to evaluate language ability in a specific context—academic or vocational, for example—through writing. Second Language Writing Ability The discussion of contexts above highlights the fact that ELLs differ greatly in terms of a number of variables e. EFL, proficiency level that influence how the construct of writing is defined for a given purpose.

  1. However, readers may be surprised to find out that there are more people learning and communicating in English in the world than there are native speakers Kachru, 1997 ; McKay, 2002 ; some are immigrants or children of immigrants in the United States U. It is not clear how automated scoring systems would be used to evaluate integrated responses.
  2. At a single university with a test for only a few hundred students, scoring essays can require several raters working over multiple days, and many tests are much larger. Pedagogical practices and perceived learning effectiveness in EFL writing classes.
  3. Again, even though raters may not specifically focus on such features, the relationship between these features, which can be counted automatically, and rater scores reflects the fact that writers who are able to express complex ideas generally have mastered the syntactic means to do so in a sophisticated way.
  4. Certain elements of grammar, for example, appear to be resistant to explicit instruction and acquired late, such as the use of relative clauses and the English article system Ellis, 2005. Thus, AES systems are not currently well suited to evaluate features of writing such as authorial voice or strength of argument, which depend on shared background knowledge and assumptions between reader and writer.

Crucially, these variables are related to the degree to which the focus of instruction and assessment is on linguistic or rhetorical concerns; that is, concerns about mastering sentence structure, morphology, and vocabulary vs.

This focus, in turn, has major implications for the use of AES and to its acceptance by stakeholders, including students, teachers, administrators, and users of test scores. There is a general consensus in the field that second language writing ability is dependent upon both writing ability and second language ability e. Native English speakers learning to write essays assessment and classroom based assessment english language essay school generally have automatic control over basic text production processes and extensive experience with English texts; thus, although their academic vocabulary and control over advanced structures such as relative clauses and non-finite subordination strategies may still be growing, they do not need to devote cognitive effort to subtler aspects of English grammar such as article and preposition use or the formation of verb tenses.

NNSs of English, on the other hand, vary tremendously in their control over English syntax, morphology, and vocabulary, their familiarity with English written genres, and their experience in writing, either in English or in their home language. Those who are strong writers in their first language can often transfer writing skills to their second language given a certain level of proficiency. However, limited English proficiency can hamper student writing because of the need to focus attention on language rather than content Weigle, 2005.

  1. For example, Roca de Larios, Murphy and Marin 2002 suggest that writing strategies such as problem-solving strategies, goal setting and organization, and having a sense of audience can be effectively taught within the course of a single semester. With article errors, these figures are slightly better.
  2. Considerations for the use of automated scoring and feedback differ for these different situations.
  3. Demonstrate an awareness of English language and literature in different national, historical, social, political, and cultural contexts.
  4. If automated feedback on errors is to be useful, it should be able to correctly identify the kinds of errors that ESL students are likely to make, and recent advances in AES systems have improved this ability, particularly in the areas of articles and preposition usage. The teacher determines the relative importance of each activity in determining an overall grade point average, just as teachers do with traditional assessments.
  5. This process is further simplified as teachers and schools begin to collect and maintain lists of generic tasks and assessments that teachers can adapt for individual lessons.

At lower levels of language proficiency, then, the focus of assessment is generally on linguistic issues; that is, the degree to which writers have control over basic vocabulary, syntax, and paragraph structure. As writers gain more control over these skills, the focus can shift to higher order concerns such as development, strength of argument, and precision in language use. From this discussion it is clear that what is meant by writing in assessment can be very different in different contexts, and that therefore when we talk about automated scoring of writing we need to be clear about what kind of writing is meant.

While automated scoring of writing has been quite controversial in the composition community e. Deane in press argues that when the use case emphasizes the identification of students who need to improve the fluency, control, and sophistication of their text production processes, or affords such students the opportunity to practice and increase their fluency while also learning strategies that will decrease their cognitive load, the case for AES is relatively strong; but if the focus of assessment is to use quality of argumentation, sensitivity to audience, and other such elements to differentiate among students who have already achieved fundamental control of text production processes, the case for AES is relatively weak.

The former is precisely the assessment and classroom based assessment english language essay for many ELLs both in ESL and EFL settings, and thus a relatively strong argument can be made for automated scoring and feedback systems if they can be implemented wisely. As discussed above, writing instruction for L2 learners varies by age, proficiency level, and context.

However, some pedagogical principles apply across instructional settings. Depending on how proficient they are, therefore, students learning to write in their second language need more of everything: Writing teachers, especially in second language academic contexts such as first-year composition courses, need to find ways to balance the need to provide opportunities to learn and practice new language structures with opportunities to improve written fluency without getting bogged down in grammatical concerns.

It is well known that language proficiency, as it relates to writing, develops slowly over a number of years and depends on extensive exposure to different texts in different genres. Certain elements of grammar, for example, appear to be resistant to explicit instruction and acquired late, such as the use of relative clauses and the English article system Ellis, 2005.

However, other aspects of writing seem to be amenable to instruction regardless of English proficiency level. For example, Roca de Larios, Murphy and Marin 2002 suggest that writing strategies such as problem-solving strategies, goal setting and organization, and having a sense of audience can be effectively taught within the course of a single semester.

The major paradigm in writing instruction in the U. That is, the process of idea generation, drafting, giving and receiving feedback, and revising, which is used by expert writers, is modeled and supported by the teacher. In such an approach, students submit multiple drafts of essays and only the second or third draft of the essay is graded.

In other countries, assessment and classroom based assessment english language essay course, writing instruction varies across educational settings. In many places writing is viewed as a support skill for reinforcing and practicing the grammar and vocabulary learned in class, rather than as a skill to be developed in its own right for communication.

As many EFL teachers are NNSs themselves, they often do not feel equipped to write or comment on student writing in English, and much of the writing that is done in class is in preparation for examinations. As a result there are many settings where a process approach is not implemented; rather, the focus may be explicitly on practicing strategies for writing timed essays of the kind that are typical on large-scale assessments see, for example, You, 2004.

Recommend to a friend

The Role of Error Correction in Second Language Writing Instruction One of the features of AES systems emphasized by proponents is the ability to provide instantaneous feedback on writing, and particularly on sentence-level errors in grammar and usage.

Thus, it is important to explore the role of error correction in L2 writing instruction. After all, if error correction is not useful and does not lead to improvements in writing, as claimed by Truscott 199619992007 and others, there is little point in automating it.

However, the pendulum has swung back the other direction with the understanding that second language writers need and want to improve their knowledge of academic English grammar and vocabulary. One implication of this is that students want feedback on their writing, particularly about the errors that they make.

The issue of error correction is one that has caused a great deal of controversy and anxiety for second language writing teachers, particularly in light of some scholarship that suggests that error correction is unhelpful or even harmful e. Truscott, 199619992007. It is clear from the research that L2 writers want comprehensive error correction; it is also clear that teachers frequently do not always know how to provide useful feedback on errors or feel that the time spent in providing comprehensive error feedback reaps useful benefits.

Ferris 2011 provides a thorough review of the research on corrective feedback in L2 writing.

Ohio University

This research suggests that students welcome feedback on errors, and that focused feedback which can be used in revising has both short-term and longer-term effects on their control over specific structures targeted in feedback. The degree to which L2 teachers actually provide accurate and comprehensive feedback is unclear. Although an often-cited paper by Zamel 1985 reports that teachers are inconsistent, arbitrary, and contradictory in their comments, there are no statistics in the paper to substantiate this assertion, and it is quite likely that 25 years of improved teacher education has improved the situation substantially.

In a study designed to gather data on the nature and effect of teacher feedback, Ferris 2006 collected writing samples from six sections of an ESL composition course containing teacher feedback on 15 error types e.

While these results cannot be generalized to other settings, they at least provide a baseline on which to judge automated scoring systems: To summarize, in some contexts writing instruction assessment and classroom based assessment english language essay ELLs is very much focused on language acquisition, while in others such instruction is focused on writing strategies.

In reality, most ELLs need both types of instruction; one complication is that teachers dealing with ELLs are often trained either in second language acquisition or in composition pedagogy, but frequently not both. In order to understand an appropriate role for AES in L2 writing assessment, it is important to take what we know from both perspectives.

Specifically, for automated scoring, it is important that scoring systems be able to identify the features of language that characterize learners at different proficiency levels and not just errorsand, at the same time, allow learners who are still developing in their language to demonstrate their writing competence overall content development, quality of ideas, organization, etc.

This is particularly important when ELLs are assessed along with their native speaker peers. In terms of automated feedback, ELLs want and need specific feedback on language, but such feedback must be digestible and written in a way that is useful for learning.

ELLs also want and need feedback on content and organization, but features associated with these aspects of writing are more challenging from a computational perspective to provide automatically. As Chapelle and Chung 2010 have suggested, the ideal hybrid may be one in which sentence-level feedback is provided by an automated mechanism, leaving the teacher free to comment on higher order concerns.

Nevertheless, research in natural language processing is focusing on these higher-order concerns, as discussed in later chapters: Scoring and Feedback The difference between automated scoring and automated feedback is important when considering their use for ELLs. Automated scoring is primarily intended for large-scale tests that is, beyond the level of the classroom and automated feedback is primarily intended for instructional use, although these two functions are sometimes blurred.

For instance, it is possible, and may be very welcome, to provide diagnostic feedback on essays in large-scale tests. It is also possible, though perhaps less desirable, to incorporate automated scoring along with automatic feedback in a classroom setting.

For the moment, however, we will make a distinction between scoring—that is, using automated tools to produce a score that is intended to be equivalent to a human score on the same essay for the purpose of some decision, such as admission or placement—and feedback: As noted above, automated scoring for high-stakes decisions is a highly controversial issue within mainstream composition studies. Large-scale tests are often produced by private companies employing testing and measurement experts and are used to make relatively high-stakes decisions, such as admission to higher education, graduation, and placement.

Some tests, such as the Graduate Record Examination, are taken by L1 and L2 students without distinction; others are designed specifically as English proficiency tests for L2 speakers; and yet others, such as the University of California test, incorporate mechanisms for determining whether students who need writing courses would benefit from classes specifically designed for L2 speakers.

In other cases, AES is not currently used but may be under consideration or development for the future. The most obvious potential advantage of AES for large-scale assessment is the savings in terms of time and cost, given the labor-intensive nature of human scoring of writing, as well as the reliability of AES in producing the same score for a given essay. Most large-scale tests require essays to be double-rated for reliability, with a third rater used if the first two raters disagree.

At a single university with a test for only a few hundred students, scoring essays can require several raters working over multiple days, and many tests are much larger. An automated scoring system that can accomplish this scoring in a matter of minutes represents a major savings of time, if not necessarily money.

Furthermore, the decisions made on the basis of test scores are too important to be made by machines. One of the goals of this volume was to address the issue of system transparency. Several systems are described in the chapters that follow. In contrast to large-scale testing, much writing assessment is done at the classroom level by a teacher who knows the students, is in control of the curriculum, and understands the context for the assessment. Within classroom assessment we can distinguish assessment and classroom based assessment english language essay and formative assessment.

Summative assessment is used to evaluate how much students have learned in a course and whether specific aims have been met.

Formative assessment, on the other hand, is used to help teachers and students diagnose and address specific writing problems during a course, before a final grade is recorded.