. . .
Vol 35 No 2, April - June 1997
Page 16
PREVIOUS ... CONTENTS ... SEARCH ... NEXT

Forum

Current Proficiency Testing
A Reflection of Teaching
BY CHRISTINE IRVINE-NIAKARIS


EFL proficiency tests did not use to be based on any particular syllabus but were designed to measure different levels of language ability in general. This made it difficult for teachers to prepare students for a proficiency test since there were no identified learning objectives that could serve as guidelines for selecting appropriate teaching materials and techniques. In recent years, however, research in the field of testing has focused more on the relationship between testing and the teaching which precedes the test—referred to in the literature as the “washback” effect. As a result, some proficiency test specifications—what language is tested and how it is tested—now more closely resemble syllabus specifications of ELT coursebooks that reflect current methodological principles. This development brings proficiency tests more in line with what we expect of achievement tests, thus, reflecting teaching.

However, in order for teachers to exploit the positive link between teaching and testing effectively, they need to analyze the objectives of the test and then be able to select appropriate materials and procedures that will serve these objectives. This is not an easy task for teachers who have not had recent training in the principles of testing. It may be helpful, then, to see in practice how test objectives can be inferred from a close analysis of an actual language examination, and, from there, to see how these objectives relate to teaching.

 



This article will look at the washback effect, using as an example the new University of Michigan Examination for the Certificate of Competency (ECCE), designed for intermediate- to upper-intermediate-level learners. The ECCE can be considered an example of the movement towards achievement-oriented proficiency tests.


ECCE: Teaching and testing relationship


Current language teaching methodology views language use as a communicative, interactive process taking as its starting point the notion of communicative competence. Since the development of communicative competence is the main objective for teaching, determining what we teach and how we teach, it would appear that a model of communicative competence would help us identify testing objectives and serve as an appropriate framework for evaluating the washback effect, provided, of course, that the test under review has been designed on the basis of the principles of communicative competence.

What is tested in the ECCE: Components of communicative competence. The term “communicative competence” originated in sociolinguistics with Dell Hymes (1972). Communicative teaching is guided by the model put forward first by Canale and Swain in 1980 and revised by Canale in 1983. According to this model, communicative competence de-scribes the learner’s ability to use the language in listening, speaking, reading and writing in terms of the following competencies:

  • linguistic competence (knowledge of form)
  • sociolinguistic competence (ability to use language appropriately in different contexts)
  • discourse competence (cohesion and coherence)
  • strategic competence (knowledge of verbal and non-verbal communication strategies).


Each of these components is tested in the ECCE in a variety of ways, both separately and in relation to one another.

Linguistic competence is tested with a multiple-choice format in the grammar section. This section tests the recognition of a wide range of basic grammatical structures including tenses, word form, prepositions, etc. Linguistic competence is also tested indirectly in the listening and reading sections through comprehension of gist and specific information which may require the understanding of a specific grammatical item. It is tested as well in the productive skills sections of the test (speaking and writing ) where candidates are required to produce grammatically correct basic sentences in their compositions and oral interviews. Most integrated skills course books aim to develop linguistic competence through skills development as well as controlled grammatical exercises which focus on recognition of form. However, teachers tend to over-emphasize the latter in their teaching mainly because it is a more concrete way of teaching grammar and because it also easily lends itself to self study.

Sociolinguistic competence is tested to some extent in the vocabulary section, where candidates are required to choose the word that best completes the sentence. Very often collocations are tested, i.e., words which are firmly associated with each other; e.g., make a mistake, apply for a job. Words are tested in a sentence context, which means that candidates can apply the strategies they have developed from classroom instruction for guessing words in context using clues such as synonyms, antonyms, punctuation, and discourse markers.

Discourse competence and linguistic competence are tested in the reading and listening sections. Comprehending gist involves making connections between different parts of the text (discourse competence) while comprehending specific detail may involve linguistic contributory skills referred to earlier. Although an integration of these comprehension sub-skills is essential to the learner’s ability to function in real communication, it is an accepted pedagogical practice to break down the comprehension process into separate skills for both teaching and testing.

All four competencies are tested integratively in the writing and speaking sections, where the candidates need to produce language typically used in everyday contexts. The language must not only be accurate (evidence of linguistic competence) but also appropriate (evidence of sociolinguistic competence) and coherent (evidence of discourse competence). A candidate also has the opportunity to demonstrate his or her strategic competence in the oral interview; here strategic competence is essential for maintaining the flow of communication when the candidate lacks a particular word or phrase, or needs to ask for clarification.

How is communicative competence tested in the ECCE? Certain sections of the test make use of real-life tasks as an appropriate vehicle for assessing communicative language ability. Most of these tasks are representative of those included in EFL coursebooks which aim at developing communicative competence.

In the first task of the listening section, the candidates listen to short dialogues or announcements related to everyday situations. Candidates need to listen for gist or specific information and respond to a set of pictures. There is no reading involved in the task, and the skills tested are appropriate for this type of text. This design is intended to enhance its authenticity as a communicative activity. In the second section candidates listen to a ten-minute interview which is presented in short segments followed by two to four questions after each section. Candidates are allowed to take notes, which is an appropriate task for this type of text. Note-taking is also an authentic sub-skill which most candidates need to develop.

The reading tasks assimilate real tasks in order to test different approaches to reading. As can be seen below, the different tasks are matched with appropriate text-types (See Figure 1 ).

The speaking section of the test is an oral interview in which candidates talk about personal background information and ask questions about a set of two or three photographs. The questions have a purpose: to help candidates come to a decision, which they must then explain. This role-play facilitates the use of certain features of spoken discourse that are prevalent in real communication such as asking for clarification and forming questions. Moreover, the fact that there is an outcome in such an activity increases its communicative potential.

The writing task represents a real-life task, in that a purpose is created for writing: Candidates write a composition or a letter in response to a reading prompt. The topics relate to the life experience of young adults so that they are not stuck for ideas and can concentrate on expressing themselves well.

Many of the features that contribute to the creation of real-life tasks have the following characteristics:

  • They are contextualised and relate to every day situations.
  • They test authentic skills.
  • They provide a purpose for communication.
  • They are appropriately matched to text type.
  • They have an outcome


On the whole, ECCE reflects current methodology in teaching by focusing on testing the components of communicative testing discretely and integratively using all four skills. It also makes an attempt to test students’ ability to use language through real-life tasks. There is a plethora of coursebooks to choose from which aim to develop communicative competence in all four skills. The teacher’s manual of such books usually includes a description of its objectives, which should make explicit reference to the teaching of communicative transactions in all four skills. However, as we have seen from the analysis of the ECCE, many of the test activities test real world abilities and therefore could be used as teaching materials not only to help the students with strategies for the actual test but also to help students develop genuinely useful language skills.


ECCE: Teaching tool


Many teachers who are under pressure to prepare students effectively for a particular test expose students to a number of practice tests in the hope that these will increase competency, which in turn will enable the students to pass the test. This is a misconception, particularly if the students receive limited feedback from the teacher, such as simply supplying the correct answer. The following sug- gestions for using the ECCE practice tests as a teaching tool, however, could lead to effective language skill building and development of specific test-taking strategies through active involvement of the learner.

ECCE listening section. In the following example the students listen to a short dialogue or announcement, which is then followed by a comprehension question. The students respond to the question by selecting the appropriate picture from three choices.

Students will hear:


Man: Here’s the delivery. You owe me $20. I’d prefer you pay by credit card.

Woman: But I already paid. Just a minute. I’ll show you.

Woman: What does she need to get for the man? (comprehension question)


Students will see pictures (See Figure 2 ).

The above item requires the student to listen for specific information which requires understanding of a particular structure (al-ready paid). Teachers can help students develop this skill by introducing the following procedures:

  • Make students aware of the need to predict the content which will help them to process the information more easily. This can be done by discussing the pictures before they listen.
  • Provide a pre-listening question which will help students focus on the structure. For the above item the question “Will the woman pay now?” would serve this purpose.


The following item from the same section requires the student to understand the gist of the conversations:

Students will hear:


Man: This is terrible. I had no idea the road would be so crowded.

Woman: I know. Let’s get off as soon as we can. Turn off at the next exit.

Woman: Where are they now? (test question).


Students will see pictures (See Figure 3 ).

Listening for gist can be practiced with the test materials in the following way:

  • Before listening, the teacher can provide examples of contexts on the b/w board which the students match with the pictures, e.g., someone giving instructions, a weather report, description of person or place and sequence of events. Alternatively, students can suggest contexts themselves.
  • Students then listen to the dialogues and confirm their predictions.
  • Students discuss specific language which helped them understand the gist of the conversation.


In the second part of the listening students have to understand the main points of a longer text usually in the form of an interview in which someone is talking about a place of interest. The students hear the interview in several parts, and after each part they are asked to respond to two to five questions by indicating, “Yes, that’s true,” or, “No, that’s not true.” They are allowed to take notes while they are listening. Therefore, activities which focus on note-taking and listening for specific information would help the students considerably.

Introduce the topic and elicit from the students key words related to the topic. This develops predicting skills.

Provide written statements related to the text, some of which are the main points of the text and others which are irrelevant. Students check the main points. Alternatively, provide a while-listening activity which will help students focus on the main points. An example would be a chart which indicates important dates about the place of interest. This would help students follow the sequence of events and help them to organize their notes.

Grammar. Grammar is tested in a multiple-choice format and is contextualised in short conversations or in sentences that reflect spoken discourse. Students need to understand the meaning of the item and be able to recognize the correct form.

A: “Did you go to class yesterday?” B: “Yes, and you _______ gone, too!”

  1. shouldn’t have
  2. would have
  3. wouldn’t have
  4. should have


The above item requires the student to understand the language expressing the function of criticism. Teachers can make students aware of the relationship between form and function with procedures such as the following:

Before the students look at the options, the teacher asks the following questions about the meaning of the exchange.


Did A go to class? Did B go to class?


Do you think B is agreeing with A, advising A, or criticizing A?

Then the students can check the options and match the function with the correct form.


“John went to the game -______ the bad weather.”

  1. despite
  2. in spite
  3. despite that
  4. in spite of that



The above item requires the student to recognize the correct form since all the options have the same meaning. Students can be helped with development of accuracy by working with this item in the following way:

  • Students write a sentence with each of the options before looking at the stem.
  • Students discuss in groups the word order of their sentences.
  • Students then choose the correct option for the given sentence.


Vocabulary. Vocabulary is tested at the sentence level with the focus on recognition of correct use. This test activity can be used to build up specific word-attack skills which will also be useful for reading comprehension. For example, students can be guided to guess words from context by focusing their attention on clues such as synonyms, antonyms, and reference words.

Example 1:


This is a real diamond, not _________.

  1. a compromise
  2. a conversion
  3. an alienation
  4. an imitation



Example 2:


Do you like my coat? I got it at _____ price.

  1. a bargain
  2. a bonus
  3. an easy
  4. a simple



As can be seen in example 2, high frequency collocations in everyday contexts are tested. The best way to help students with collocations in exam preparation material is to do the following:

  • Students try to complete the sentence before looking at the options.
  • Students then look at the options and make comparisons with their own choices.
  • Students discuss the distractors with the teacher and come up with appropriate contexts for them.


Reading. The first reading task is expository prose between 220 and 250 words in length. The purpose of the text is to inform the reader about a topic of interest such as environmental issues or human behavior of the type you would find in a magazine or a leaflet. The multiple-choice questions check the students’ ability to understand the main idea, specific details, and to make inferences. Multiple-choice questions can be very effective for training interpretative skills if the teacher exploits them in the following way:

  • Make students aware that distractors for reading items are written with students’ likely mistakes in mind.


  1. They contain words from the text but the overall meaning is different.
  2. They often exaggerate the meaning by using all, never, or always.
  3. They may say something that is possibly true but not stated in the text.


  • By understanding the nature of the distractors, the students can then discuss the justification for their choices in groups and then present their arguments to the whole class.
  • For further practice in interpretative skills, the teacher can produce a number of statements that can be inferred from the text and a number of statements that could be true but are not stated in the text. This activity could also generate useful group discussions.


The second and third reading tasks require the student to skim and scan the text quickly. The following guidelines could be useful for assisting students to retrieve information quickly and accurately:

  • Direct students to read the questions first so that they read with a purpose.
  • Make them aware of the importance of rubrics to locate information quickly.
  • Time the task so that students are forced to skim and scan and not waste time reading every single word.


Writing. In the writing section students need to write a letter or composition in response to a prompt (letter or short article). They have only thirty minutes to write a coherent text so there is obviously no time for the students to write more than one draft. However, students can be trained to apply a process approach to writing in which they manage their time effectively. For example, they should be encouraged to work out how much time they need for each stage of their writing in the following examination conditions:

  • reading of prompt and underlining key words (2 minutes)
  • noting down ideas and useful language (4 minutes)
  • organizing ideas into paragraphs (2 minutes)
  • writing of composition (20 minutes)
  • editing (2 minutes)


Speaking. The students’ speaking ability is assessed through a role-play performed with the examiner. Frequent practice with roleplays in the classroom helps the student to develop fluency and accuracy. Controlled practice with typical question forms preceding the activity may help to develop the students’ accuracy, but students also need to develop strategic competence for managing conversations more effectively and giving an impression of fluency. Practice with the following categories of words and expressions may help to achieve this end:

  • asking for clarification’ What do you mean exactly?
  • hesitating’ Let me see.
  • fillers’ O.K., fine, right


The above suggestions are by no means exhaustive. The important thing to remember, however, is that this approach for teaching exam practice test material is much more instructional, and teachers should spend just as much time preparing these lessons as regular lessons.


Conclusion


In this article I have analyzed the ECCE and illustrated how it could be considered an example of the movement towards achievement-oriented proficiency testing. The analysis shows that objectives of the test resemble syllabus specifications of current ELT coursebooks which aim to develop communicative competence. Furthermore, the test activities can be exploited effectively as classroom activities which not only help the students to become familiar with the format of the examination but also help them to develop useful language skills. In order to maximize the washback effect, however, teachers need to be empowered with a greater understanding of the principles involved in communicative testing. It is therefore essential that all teacher education programs contain a testing component which includes an analysis of various proficiency tests and practical ideas to illustrate the positive effects of the relationship between testing and teaching.


CHRISTINE IRVINE-NIAKARIS is Director of Education at the Hellenic American Union, Athens, Greece.
 

Return



References


  • Canale, M. and M. Swain, 1980. Theoretical basis of communicative approaches to second language teaching and testing, Applied Linguistics, 1. pp. 1–47.
  • Canale, M., 1983. On some dimensions of language proficiency. In Issues in Language Testing Research, ed. J. W Oller. Rowley, MA: Newbury House.
  • Hymes, D. H. 1972. On communicative competence. In Sociolinguistics, eds. J. B. J. Pride and Holmes. Harmondsworth: Penguin Books.
  • Weir, C. J. 1990. Communicative Language Testing. London: Prentice Hall International (UK) Ltd.
  • ———. 1995. Understanding and Developing Language Tests, Phoenix ELT.



Back to Top


Vol 35 No 2, April - June 1997,
Page 2
PREVIOUS ... CONTENTS ... SEARCH ... NEXT





Figure 1

Text task and purpose Text type
Intensive reading skills to obtain through comprehension Expository prose
Skimming and scanning to make a decision, e.g., selecting a school, holiday 4 or 5 short texts which can be compared and contrasted to find the information desired
Skimming and scanning to find relevant information A longer text in the form of a brochure or manual


Back to Article







Figure 2

PAID INVOICE
CASH
CREDIT CARD


Back to Article







Figure 3

CROWDED BUS
CROWDED THEATER
CONGESTED HIGHWAY


Back to Article

 

Vol 35 No 2, April - June 1997,
Page 2
PREVIOUS ... CONTENTS ... SEARCH ... NEXT
. .

On October 1, 1999, the Bureau of Educational and Cultural Affairs will become part of the
U.S. Department of State. Bureau webpages are being updated accordingly. Thank you for your patience.