Articles

Where Does Performance-Based Assessment Fit in with the High Stakes Test Environment,
Especially for English Language Learners?

by Kismet Oz, M.Ed. TESOL (Teaching English as a Second Language), an English Horizons teacher

 The Controversy Over Assessments and How It Relates to ELLS

       A controversy over testing has been brewing in America for over a decade now. The intensity has increased exponentially since the No Child Left Behind Act (NCLB) in 2001 has put more pressure on schools to measure up to standards imposed from the federal level.  This controversy has separated parents, educators and legislators into two camps: those in favor of standardized testing as the ultimate measure of achievement, and those in favor of incorporating alternative assessment into testing programs to more accurately measure achievement that takes contemporary learning theory and learner circumstances into consideration.  The argument against performance assessment asserts that it is "non academic, subjective, invasive of privacy, and part of social engineering by elites." (1).  Those who are concerned with the narrow scope of high stakes standardized tests say that such tests do not provide a true measure of learner competencies or learning outcomes which fall into four general categories:

• Knowledge - the facts, concepts, principles, and theories of a subject matter.

• Skills - the behaviors required to carry out an activity at a demonstrated level of proficiency.

• Attitudes - the feelings, opinions, or dispositions related to the subject matter.

• Learning Abilities - the general skills or attitudes deemed important for learning, such as reading, thinking, writing, problem solving and self direction. (Feasley & Olgren, 2002) (2)

        The added problem which greatly concerns teachers of English Language Learners (ELLs) is that "standardized test results are likely to reflect limited proficiency in English and a lack of opportunity to learn the subject matter of tests" (Pierce-Valdez, 2002). (3) Menken (2000) concludes that because of this, standardized tests "may not accurately assess learner's content knowledge or skills." (4) The academic proficiency required for ELLs to perform well independently on tests can take 5-7 years to develop. Most language programs only accommodate students for 2-3 years before they are placed in mainstream classes. This puts ELLs at a disadvantage for acquiring cognitive academic language proficiency (CALP).  In English Language Learners and High Stakes Tests: An Overview of the Issues, Coltrane (2003) emphasizes that "with issues such as school funding, grade-level promotion, and graduation at stake, using standardized test scores as a basis for major decisions could potentially be detrimental to ELLs and to the schools that serve them." (5)  The reality of  NCLB is that "the new law mandates that students historically exempted from the testing pool, such as special education students and English language learners, must now take tests and have their scores counted (Karp, 2003)." (6)  At first glance, this would seem to be a positive move for ELLs because it forces schools to be more accountable for achievement of learners who have, in many cases, been left behind. The reality of including ELLs into testing calculations, according to Karp, is that it "make it harder for schools to reach the unreasonable adequate yearly progress targets as mandated," and it forces "students to take inappropriate and unhelpful mandated assessments, like tests in languages they don't understand." (7) Widely used standardized tests often contain cultural and semantic information which is difficult for the ELL student to infer or understand. This culture bias also compromises the validity of the tests for ELLs. Some are concerned that the NCLB regulations will reduce bilingual education programs which will only serve to exacerbate the problem.  It is clear that there are major concerns about including ELLS in standardized testing.

        During a recent tour of classroom observation, I was present during the Illinois Standard Achievement Testing. The companion test for ELLs in the state is called the IMAGE Test which stands for the Illinois Measure of Annual Growth in English, but it is a well known fact among teachers that this test is actually being used to measure content knowledge. I found this baffling and completely antithetical to the concept of validity. During the process of my research, I located a document from the minutes of a Illinois State Board of Education meeting that asked the Assessment and Accountability committee studying and reporting on the IMAGE test, to address three important questions:

1. Can one test be used to measure language proficiency and content knowledge?

2. What does the latest research say about assessing second language learners in achievement using an English assessment?

3. What are the possibilities for native language assessment for Illinois ELLs on ISAT/PSAE? (8)

        It is encouraging that the current use of the IMAGE test is being reevaluated.  If the latest research is used as a guide for decision making, then it seems likely that positive changes can result.  Although the final recommendations by the committee are not published, I do know that as of March, 2005, as mentioned above, the same test was still being used to test content knowledge and that it strictly prohibited any native language support.  I also know that over a period of several weeks, preparation for the IMAGE test was all that was taking place for the ELLs in third grade and fifth grades who were going to be taking the test. I observed in the classroom during testing and watched some of the students sit in frozen state. Some did not even get through one page of the test. Of those that did complete the tests, a glance at the answers clearly indicated a lack of understanding of the language used on the test.  The interesting thing is that during class time while doing practice exercises for the test and having things explained to them in terms they could understand, the students were performing very well.  This may not be empirical, but it seems to me that this is the exact type of situation where performance assessment would give a true measure of the ELL's competencies.

        Teachers of English to Speakers of Other Languages (TESOL), issued a position statement in 2002 that delineates the Issues for Assessment and Accountability of English for Speakers of Other Languages.  Among the recommendations made in the statement, there are three which I find particularly significant to this discussion:

• In assessing linguistically and culturally diverse students, there should be a broad range of accommodations that match best instructional practices and the students' language proficiency. In this way, ESOL students can participate more meaningfully in standardized state and local assessments, when they are ready to do so.

• The entire school faculty needs to be knowledgeable about the challenges ESOL students face on standardized tests. Using this knowledge as a foundation, the staff should prepare ESOL students for success by helping them with language, content, vocabulary, formats, test-taking skills, and cultural orientation of the tests.

• Exit criteria for ESL/bilingual programs and services should be aligned to the academic skills required for success in mainstream classes. (9)

Performance Assessment: What it is and how it is done

        Performance assessment is a subset of authentic assessment (Brown, 2004,). (10) The characteristics listed by Brown of performance assessment are:

1.          Students make a constructed response.

2.          They engage in high order thinking, with open ended tasks.

3.          Tasks are meaningful, engaging and authentic.

4.          Tasks call for integration of language skills.

5.          Both process and product are assessed.

6.          Depth of a student's mastery is emphasized over breadth (Brown, 2004, pg. 255). (11)

       In language programs, performance based assessment involves, "oral production, open ended responses, integrated performance, group performance and other interactive tasks"  (Brown, 2004, pg. 11). (12)  Brown concedes that such testing is more time consuming and expensive but the payoff is that with such direct testing, a higher level of validity is achieved because "learners are measured in the process of performing targeted linguistic acts" (Brown, 2004, pg. 11). (13)  In the debate over testing, it is very important that test validity is emphasized. It is validity, (particularly construct and consequential validity) that will end up making the case for a particular kind of testing in the long run.  Brown cautions that it is important for teachers to formulate performance assessment with the same attention to standards as traditional tests. It is therefore important that: 1. Clear goals and objectives be set for the performance; 2. Performances should be scaffolding on previous learning; 3. A reliable rubric should be used for scoring; 4. Assessments should be used to provide systematic feedback (Brown, 2004, pg. 255). (14) These standards also ensure that a great deal of positive washback will be achieved in the process.  It is also necessary to take into account individual learner levels of proficiency to determine the level of accommodation which means the "(adjustment to be made to the testing situation to allow the test taker to display  more adequately his/her competency" during the performance assessment (Aguirre-Munoz & Baker, 1998).(15)

        Performance based tests are considered direct in that they test actual abilities which are a good predictor of how students will perform in the classroom. (Richard-Amato, 2003).(16)  They are also integrative because they measure the learner's ability to integrate various skills to perform a task such as answering questions in a conversation, discuss a reading, tell a story, write a letter, etc. (Richard-Amato, 2003, pg. 133). (17)  A further distinction that can be applied to performance based assessment is that they are pragmatic which is defined by Oller (1979) in Richard-Amato, (2003)(18) as an integrative test meeting two naturalness criteria: The context must rely on a natural sequencing of events, and a knowledge of the world and how it works must be relevant to its content.  Pragmatic tests can be dictation, cloze procedures, oral interviews, essay writing, and other similar tasks, which are considered to be highly reliable, valid and practical for measuring language ability. (Richard-Amato, 2003, page 134).(19)  Performance tasks that are well designed also have the benefit of providing comprehensible input to students, generating descriptive information that can guide instruction, and providing information for teaching and learning that results in improved student performance which makes them diagnostic in nature as well (Pierce-Valdez, 2002).(20)  Consequently, another positive benefit to using performance assessments is that when teachers "use these forms of alternative assessments they will naturally teach in ways that emphasize reflection, critical thinking, and personal investment in one's own learning" (Liskin-Gasparro,1997). (21)

        There are three broad categories of performance based tests that can be used:  products, performances and process oriented assessments.  Products are student created materials that provides concrete examples of their application of knowledge. Performances have students demonstrate their application of knowledge to the teacher and process oriented assessments "provide insight into student thinking, reasoning and motivation" (Pierce-Valdez, 2002). (22)  Various forms of performance based tests being used fall into these categories are: 

• Station Activities that use open ended questions to elicit student's thinking strategies while doing a discrete task within a set period of time.

• Domain Projects that ask students to set up criteria and make judgments to evaluate an idea or concept.

• Portfolios which contain a collection of student's projects which are collected in sequence to measure progress over time.

• Videotaping which can be used for interviews to assess student's understanding of content (Ascher, 1990). (23)

        This table, adapted from Dietel, et al. (1991) provides a useful foundation for designing performance based assessments. (24)

       There are three recommended approaches teachers can use to provide feedback on student performance: The Checklist Approach that indicates the presence or absence of certain performance elements; Narrative/Anecdotal Approach that involve writing a report based on the teachers observations; The Rating Scale Approach that takes the form of a numerical grading scale rubric (Brualdi, 1998). (25)  There are excellent resources for teachers and schools to aid in the development of performance based assessments. Performance Assessment.org provides a comprehensive explanation of the six components necessary to implement a PBA program including: Active Learning, Formative and Summative Documentation, Strategies for Corrective Action, Multiple Ways for Students to Express and Exhibit Learning, Graduation level performance-based tasks aligned with Learning Standards, and a Focus on Professional Development. (26)

        In conclusion, I believe that there is plenty of room to meet in the middle between standardized and alternative assessment. In particular, performance assessment can be used in constructive ways to help students prepare for standardized tests and it is an excellent way to diversify how learner competencies are measured. The unifying factor is that "these two types of assessment do not need to be mutually exclusive; both can provide important information to teachers" (Loadman & Thomas, 2005)  if used in the appropriate contexts. (27)  Robert L. Linn (2001) of the Center for Research on Evaluation, Standards, and Student Testing recommends that "all of the weight should not be placed on one test", and  that policy makers should, "put in place a system for evaluating both the intended positive effects and the more likely unintended negative effects of the system." (28) These two points are paramount.  Unquestionably the issue of assessment in schools is being reevaluated from many fronts and with great expenditure of resources. I maintain that if the true intention among all parties concerned is for the attainment of academic success for all students, then the controversy will eventually lead to a paradigm shift in education and assessments that reach a balance between accountability and fairness.

REFERENCES

[1]      Right Wing Attacks Performance Assessment. Fair Test Examiner. (Summer, 1994). Retrieved May 9, 2005 from Fair Test Examiner 

[2]      Feasley & Olgren, (2002). Evaluation in Distance Education. Madison, WI: University of Wisconsin-Madison

[3]      Pierce-Valdez, L., (2002). Performance-based assessment: Promoting achievement for English language learners. Center for Applied Linguistics. 26:1. Retrieved May 9, 2005  from CAL.org
   
[4]      Menken, K. (2000). What are the critical issues in wide-scale assessment of English language learners? Issue Brief No. 6, Washington, DC: National Clearninghouse for Bilingual Education. Retrieved May 9, 2005 from http://www.ncela.gwu.edu/pubs/issuebriefs/ib6.pdf

[5]      Coltrane, Bronwyn, (2003). English Language Learners and High-Stakes Tests: An Overview of the Issues. Eric Digest. Retrieved May 9, 2005 from http://www.ericdigests.org/2003-4/high-stakes.html

[6]      Karp, S., (2003). Equity claims for NCLB don't pass the test.  Retrieved May 9, 2005 from http://resultsforamerica.org/education/toolkit_critique.php

[7]      ibid.

[8]      The Subcommittee on IMAGE for the Task Force on Assessment and Accountability for the Illinois State Board of Education Meeting Minutes from February 19, 2003.  Retrieved May 9, 2005 from  http://www.isbe.state.il.us/aatf/powerpoint/image2006.ppt

[9]       Teaching English to Speakers of Other Languages. (TESOL) Position Statements (June, 2002). Retrieved May 9, 2005 from http://www.tesol.org/s_tesol/sec_document.asp?TrackID=&SID=1&DID=369&CID=32&VID=2&RTID=0&CIDQS=&Taxonomy=False&specialSearch=False

[10]     Brown, H. D., (2004). Language assessment: Principles and classroom practices. White Plains, NY: Longman.

[11]     ibid.

[12]     ibid.

[13]     ibid.

[14]     ibid.

[15]    Aguirre-Munoz & Baker, (1998).  Improving the equity and validity of assessment based information systems. CSE Technical Report 462. National Center for Research on Evaluation, Standards and Student Testing (CRESST). Retrieved May 9, 2005 from http://www.cse.ucla.edu/CRESST/Reports/TECH462.PDF

[16]    Richard-Amato, P. (2003). Making it happen: From interactive to participatory language teaching. White Plains, NY: Longman

[17]    ibid.

[18]    ibid.

[19]    ibid.

[20]    Pierce-Valdez, L., (2002). Performance-based assessment: Promoting achievement for English language learners. Center for Applied Linguistics. 26:1. Retrieved  May 9, 2005  from CAL.org

[21]]    Liskin-Gasparro, J., (1997, October).  Comparing Traditional and Performance Based Assessment. Symposium  on Spanish Second Language Acquisition. University of Texas, Austin. Retrieved May 9, 2005 from http://sedl.org/loteced/comparing_assessment.html  
   
[22]     Pierce-Valdez, L., (2002). Performance-based assessment: Promoting achievement for English language learners. Center for Applied Linguistics. 26:1. Retrieved May 9, 2005 from CAL.org

[23]     Ascher, C., (1990). Can performance-based assessments improve urban schooling? ERIC Digest Number 56. Retrieved May 9, 2005 from http://www.ericdigests.org/pre-9218/urban.htm

[24]     Dietel, R., Herman, J., & Knuth, R., (1991). What does research say about assessment? North Central Regional Education Laboratory (NCREL). Retrieved May 9, 2005 from http://www.ncrel.org/sdrs/areas/stw_esys/4assess.htm

[25]     Brualdi, A., (1998). Implementing performance assessment in the classroom. Practical Assessment, Research & Evaluation, 6(2).   Retrieved May 9, 2005 from http://www.ascd.org/ed_topics/cl200002_brualdi.html

[26]     Components of performance assessment. Published by Performance.org. Retrieved May 9, 2005 from  http://www.performanceassessment.org/performance/pcomponents.html

[27]     Loadman, W., & Thomas. A., (2005) Standardized test scores and alternative assessments: Different pieces of the same puzzle.  Eisenhower National Clearninghouse Retrieved May 9, 2005 from ENC.org

[28]    Linn, Robert L. (2001). Assessments and accountability (condensed version). Practical assessment, research & evaluation, 7(11). Retrieved May 9, 2005 from http://PAREonline.net/getvn.asp?v=7&n=11

© Copyright, Kismet Oz 2005.  All rights reserved.  This material is not to be republished or reproduced in any form without the express consent of the author.