Report on Academic
Specialist Visit to Brazil: November 29-December 19, 1998
By John
M. Norris, University
of Hawaii at Manoa
About the Specialist
John M. Norris is a researcher for the
National Foreign Language Resource Center at the University of Hawaii
at Manoa, where he also teachers courses in language testing, philosophy
of second/foreign language education, and ESL in the Department of
English as a Second Language. He holds a bachelor's degree (1989)
in Modern Languages from Texas A&M University and a master's degree
(1996) in ESL from the University of Hawaii, and he is currently completing
a doctorate in Second Language Acquisition at the University of Hawaii.
John's early interest in second/foreign
language learning led to the study of several L2s (German, Spanish,
French, Portuguese, Mandarin Chinese) and inspired research as a Fulbright
Scholar at the Ruhr University in Bochum, Germany, and as a Kent J.
Brown Fellow at the University of North Carolina.
More recently, John has taught ESL in
Brazil and Hawaii and has focused his research efforts on validity
in second/foreign language measurement and assessment, task-based
language learning/teaching/testing, and research methods in second
language acquisition. His recent publications focus primarily on the
development of useful instruments and systems for assessing the acquisition
of L2s and for making valid decisions and interpretations in L2 educational
contexts (see Norris, 1996, 1997, forthcoming; Norris, Brown, Hudson,
& Yoshioka, 1998; Norris, Brown, & Hudson, forthcoming). John has
lectured and conducted workshops on L2 assessment at universities
and language schools in the United States, Japan, Brazil, and Spain,
and he recently organized and chaired the 18th Second Language Research
Forum (1998), an international conference on research in second language
acquisition.
Areas of Specialization
Trip Report
For the ELSpecialist program reported
here, I spent three weeks at two different US binational centers in
Brazil: three days at the Centro Cultural Brasil - Estados Unidos
in Goiania, and 15 days at the Associacao Cultural Brasil - Estados
Unidos in Salvador.
Essentially, my charge as a specialist
at these two binational centers was to advise the administration and
staff regarding the development, analysis, and improvement of a variety
of English language tests to meet their program needs. Much of my
recent work has revolved around the complexities of developing valid
means for measuring language acquisition as well as for making decisions
in language education contexts, and I am especially concerned with
the extent to which interpretations about examinees' language abilities
can be validly based on the use of particular types of language assessments.
Interestingly, I found that this very concern was shared by the administration
and academic coordination teams at each of the binational centers;
that is, they were worried that their language tests were not telling
them (and teachers, students, employers, parents, etc.) what they
really wanted to know about students' English language abilities.
The ELSpecialist program thus provided a unique opportunity for the
exchange of ideas between myself and the Brazilian binational centers:
they contributed the very real classroom- and program-level decision
making needs (with associated constraints) and language testing demands,
and I contributed a theoretical perspective and methodological framework
for addressing these needs and meeting these demands.
Through intensive workshop-style sessions
at each of the binational centers, we worked to synthesize an appropriate
and systematic approach to language test development and use. We initiated
this process by attempting to answer a fundamental question for educational
assessments:
"Who is making what kinds of decisions
about whom (or what) and for what purposes?"
By addressing the various parts of this
question, I believe a crucial change was initiated in the way the
binational centers view the use of language tests. In Brazilian educational
contexts in general, as well as in the many English language schools
across the country, tests tend to function almost solely in a gatekeeping
role. However, as academic coordinators at the binational centers
well realized, such an approach to assessment often does not provide
useful information for making the kinds of decisions and interpretations
that they want and need to make (e.g., decisions about: student achievement
of particular language learning objectives, student language proficiency
at particular program levels, pedagogical and program effectiveness,
certification of student language abilities, etc.). In order to better
address the actual decisions that need to be made with the assistance
of language assessments, we also agreed that other program-wide issues
required explicit statements from which test development could proceed,
including: (a) program, teacher, and learner needs; (b) curricular
goals and objectives; and (c) pedagogical philosophy and available
means.
By the conclusion of our program, we
had established a research and development framework and agenda for
addressing the following general areas: (a) needs analysis, (b) specification
of curricular objectives, (c) definition of an assessment use system,
(d) test and item specification, (e) follow-up research and analysis,
and (f) the application of technology in assessment. As the development
teams at the binational centers continue to apply their impressive
energy level and enthusiasm to this project, and via ongoing interaction
between the binational centers and myself, we hope not only to provide
valid English language assessments, but we also hope to fundamentally
enhance the binational center learning experience.
Suggested Bibliography
Bachman, L. F., & Palmer, A. S. (1996).
Language testing in practice. Oxford: Oxford University Press.
Brown, JD (1988). Understanding Research
in Second Language Learning: A teacher's guide to statistics and research
design. London: Cambridge University Press.
Brown, JD (1995). The Elements of
Language Curriculum: A systematic approach to program development.
New York: Heinle & Heinle Publishers.
Brown, J. D. (1996). Testing in language
programs. Upper Saddle River, NJ: Prentice Hall Regents.
Brown, J. D. (1998). New ways of classroom
assessment. Alexandria, VA: TESOL.
Brown, J. D., & Hudson, T. D. (1998).
Criterion-referenced language testing and assessments: A teacher's
guide. Unpublished manuscript, University of Hawaii at Manoa.
Messick, S. (1989). "Meaning and
values in test validation: The science and ethics of assessment."
Educational Researcher, 18(2), 5-11.
Messick, S. (1994). The interplay of
evidence and consequences in the validation of performance assessments.
Educational Researcher, 23(2), 13-23.
Norris, J. M. (1996). Performance
and portfolio assessment (1985-1995): An extended annotated bibliography
of sources useful for language teachers (Net Work #4) [HTML document].
Honolulu: University of Hawaii, Second Language Teaching & Curriculum
Center. [http://www.lll.hawaii.edu/nflrc/NetWorks/NW4]
Norris, J. M. (1997). The German Speaking
Test: Utility and caveats. Die Unterrichtspraxis, 30(2), 148-58.
Norris, J. M. (1997). Native speaker
judgments as indicators of L2 oral proficiency: Redefining the role
of the native speaker in proficiency guidelines. University of
Hawaii Working Papers in ESL, 16(1), 47-95.
Norris, J. M. (1998). The audio-mirror:
Reflecting on student speaking ability. In J. D. Brown (Ed.), New
ways of classroom assessment. Alexandria, VA: TESOL.
Norris, J. M. (1998). Interviews and
presentations for clarifying authentic public speaking needs. In J.
D. Brown (Ed.), New ways of classroom assessment. Alexandria,
VA: TESOL.
Norris, J. M. (1998). The reading beat:
Investigative questioning and reading comprehension. In J. D. Brown
(Ed.), New ways of classroom assessment. Alexandria, VA: TESOL.
Norris, J. M. (forthcoming). A validation
study of the ACTFL Guidelines and the German Speaking Test. Bochum,
Germany: Manuskripte zur Sprachlehrforschung.
Norris, J. M. (forthcoming). Identifying
rating criteria for task-based EAP assessment. In T. Hudson, & J.
D. Brown (Eds.), Processes in developing second language assessments:
Diverse applications (Technical Report). Honolulu: University
of Hawaii, Second Language Teaching & Curriculum Center.
Norris, J. M., Brown, J. D., Hudson,
T. D., & Yoshioka, J. K. (1998). Designing second language performance
assessments (Technical Report #18). Honolulu: University of Hawaii,
Second Language Teaching & Curriculum Center.
Norris, J. M., Brown, J. D., & Hudson,
T. D. (forthcoming). Constructing and validating second language
performance assessments: Tasks, tests, and rating criteria (Technical
Report). Honolulu: University of Hawaii, Second Language Teaching
& Curriculum Center.
Recommended Links
National Foreign Language Resource Center,
University of Hawaii at Manoa
http://www.lll.hawaii.edu/nflrc/
Department of English as a Second Language,
University of Hawaii
http://www.lll.hawaii.edu/esl/
Associacao Cultural Brasil - Estados
Unidos
in Salvador, Brazil
http://www.acbeubahia.org.br/
Norris, J. M. (1996). Performance
and portfolio assessment (1985-1995): An extended annotated bibliography
of sources useful for language teachers (Net Work #4) [HTML document].
Honolulu: University of Hawaii, Second Language Teaching & Curriculum
Center. [http://www.lll.hawaii.edu/nflrc/NetWorks/NW4]
Return to top of page
Return
to English Language Specialist Program main page
