line

home
Bureau of Educational and Cultural Affairs
homepage

Report on Academic Specialist Visit to Brazil: November 29-December 19, 1998

By John M. Norris, University
of Hawaii at Manoa


About the Specialist

John M. Norris is a researcher for the National Foreign Language Resource Center at the University of Hawaii at Manoa, where he also teachers courses in language testing, philosophy of second/foreign language education, and ESL in the Department of English as a Second Language. He holds a bachelor's degree (1989) in Modern Languages from Texas A&M University and a master's degree (1996) in ESL from the University of Hawaii, and he is currently completing a doctorate in Second Language Acquisition at the University of Hawaii.

John's early interest in second/foreign language learning led to the study of several L2s (German, Spanish, French, Portuguese, Mandarin Chinese) and inspired research as a Fulbright Scholar at the Ruhr University in Bochum, Germany, and as a Kent J. Brown Fellow at the University of North Carolina.

More recently, John has taught ESL in Brazil and Hawaii and has focused his research efforts on validity in second/foreign language measurement and assessment, task-based language learning/teaching/testing, and research methods in second language acquisition. His recent publications focus primarily on the development of useful instruments and systems for assessing the acquisition of L2s and for making valid decisions and interpretations in L2 educational contexts (see Norris, 1996, 1997, forthcoming; Norris, Brown, Hudson, & Yoshioka, 1998; Norris, Brown, & Hudson, forthcoming). John has lectured and conducted workshops on L2 assessment at universities and language schools in the United States, Japan, Brazil, and Spain, and he recently organized and chaired the 18th Second Language Research Forum (1998), an international conference on research in second language acquisition.

 

Areas of Specialization

  • Language testing
  • Educational assessment
  • Research methods in second language acquisition
  • Philosophy of second/foreign language education

 

Trip Report

For the ELSpecialist program reported here, I spent three weeks at two different US binational centers in Brazil: three days at the Centro Cultural Brasil - Estados Unidos in Goiania, and 15 days at the Associacao Cultural Brasil - Estados Unidos in Salvador.

Essentially, my charge as a specialist at these two binational centers was to advise the administration and staff regarding the development, analysis, and improvement of a variety of English language tests to meet their program needs. Much of my recent work has revolved around the complexities of developing valid means for measuring language acquisition as well as for making decisions in language education contexts, and I am especially concerned with the extent to which interpretations about examinees' language abilities can be validly based on the use of particular types of language assessments. Interestingly, I found that this very concern was shared by the administration and academic coordination teams at each of the binational centers; that is, they were worried that their language tests were not telling them (and teachers, students, employers, parents, etc.) what they really wanted to know about students' English language abilities. The ELSpecialist program thus provided a unique opportunity for the exchange of ideas between myself and the Brazilian binational centers: they contributed the very real classroom- and program-level decision making needs (with associated constraints) and language testing demands, and I contributed a theoretical perspective and methodological framework for addressing these needs and meeting these demands.

Through intensive workshop-style sessions at each of the binational centers, we worked to synthesize an appropriate and systematic approach to language test development and use. We initiated this process by attempting to answer a fundamental question for educational assessments:

"Who is making what kinds of decisions about whom (or what) and for what purposes?"

By addressing the various parts of this question, I believe a crucial change was initiated in the way the binational centers view the use of language tests. In Brazilian educational contexts in general, as well as in the many English language schools across the country, tests tend to function almost solely in a gatekeeping role. However, as academic coordinators at the binational centers well realized, such an approach to assessment often does not provide useful information for making the kinds of decisions and interpretations that they want and need to make (e.g., decisions about: student achievement of particular language learning objectives, student language proficiency at particular program levels, pedagogical and program effectiveness, certification of student language abilities, etc.). In order to better address the actual decisions that need to be made with the assistance of language assessments, we also agreed that other program-wide issues required explicit statements from which test development could proceed, including: (a) program, teacher, and learner needs; (b) curricular goals and objectives; and (c) pedagogical philosophy and available means.

By the conclusion of our program, we had established a research and development framework and agenda for addressing the following general areas: (a) needs analysis, (b) specification of curricular objectives, (c) definition of an assessment use system, (d) test and item specification, (e) follow-up research and analysis, and (f) the application of technology in assessment. As the development teams at the binational centers continue to apply their impressive energy level and enthusiasm to this project, and via ongoing interaction between the binational centers and myself, we hope not only to provide valid English language assessments, but we also hope to fundamentally enhance the binational center learning experience.

 

Suggested Bibliography

Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice. Oxford: Oxford University Press.

Brown, JD (1988). Understanding Research in Second Language Learning: A teacher's guide to statistics and research design. London: Cambridge University Press.

Brown, JD (1995). The Elements of Language Curriculum: A systematic approach to program development. New York: Heinle & Heinle Publishers.

Brown, J. D. (1996). Testing in language programs. Upper Saddle River, NJ: Prentice Hall Regents.

Brown, J. D. (1998). New ways of classroom assessment. Alexandria, VA: TESOL.

Brown, J. D., & Hudson, T. D. (1998). Criterion-referenced language testing and assessments: A teacher's guide. Unpublished manuscript, University of Hawaii at Manoa.

Messick, S. (1989). "Meaning and values in test validation: The science and ethics of assessment." Educational Researcher, 18(2), 5-11.

Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13-23.

Norris, J. M. (1996). Performance and portfolio assessment (1985-1995): An extended annotated bibliography of sources useful for language teachers (Net Work #4) [HTML document]. Honolulu: University of Hawaii, Second Language Teaching & Curriculum Center. [http://www.lll.hawaii.edu/nflrc/NetWorks/NW4]

Norris, J. M. (1997). The German Speaking Test: Utility and caveats. Die Unterrichtspraxis, 30(2), 148-58.

Norris, J. M. (1997). Native speaker judgments as indicators of L2 oral proficiency: Redefining the role of the native speaker in proficiency guidelines. University of Hawaii Working Papers in ESL, 16(1), 47-95.

Norris, J. M. (1998). The audio-mirror: Reflecting on student speaking ability. In J. D. Brown (Ed.), New ways of classroom assessment. Alexandria, VA: TESOL.

Norris, J. M. (1998). Interviews and presentations for clarifying authentic public speaking needs. In J. D. Brown (Ed.), New ways of classroom assessment. Alexandria, VA: TESOL.

Norris, J. M. (1998). The reading beat: Investigative questioning and reading comprehension. In J. D. Brown (Ed.), New ways of classroom assessment. Alexandria, VA: TESOL.

Norris, J. M. (forthcoming). A validation study of the ACTFL Guidelines and the German Speaking Test. Bochum, Germany: Manuskripte zur Sprachlehrforschung.

Norris, J. M. (forthcoming). Identifying rating criteria for task-based EAP assessment. In T. Hudson, & J. D. Brown (Eds.), Processes in developing second language assessments: Diverse applications (Technical Report). Honolulu: University of Hawaii, Second Language Teaching & Curriculum Center.

Norris, J. M., Brown, J. D., Hudson, T. D., & Yoshioka, J. K. (1998). Designing second language performance assessments (Technical Report #18). Honolulu: University of Hawaii, Second Language Teaching & Curriculum Center.

Norris, J. M., Brown, J. D., & Hudson, T. D. (forthcoming). Constructing and validating second language performance assessments: Tasks, tests, and rating criteria (Technical Report). Honolulu: University of Hawaii, Second Language Teaching & Curriculum Center.

 

Recommended Links

National Foreign Language Resource Center, University of Hawaii at Manoa
http://www.lll.hawaii.edu/nflrc/

Department of English as a Second Language, University of Hawaii
http://www.lll.hawaii.edu/esl/

Associacao Cultural Brasil - Estados Unidos
in Salvador, Brazil
http://www.acbeubahia.org.br/

Norris, J. M. (1996). Performance and portfolio assessment (1985-1995): An extended annotated bibliography of sources useful for language teachers (Net Work #4) [HTML document]. Honolulu: University of Hawaii, Second Language Teaching & Curriculum Center. [http://www.lll.hawaii.edu/nflrc/NetWorks/NW4]


 

Return to top of page

Return to English Language Specialist Program main page

 


On October 1, 1999, the Bureau of Educational and Cultural Affairs will become part of the
U.S. Department of State. Bureau webpages are being updated accordingly. Thank you for your patience.