The Oral Interview

1. Purpose
2. Interviewers
3. The Interview Process
4. Rationale for the Oral Interview
5. Background
6. Explanation of Holistic Scoring
7. Description of Final Evaluative Categories
8. Evaluation Process
9. Holistic Terms

A. Purpose

The Oral Interview is administered in order to determine whether international candidates for AI positions at Indiana University have a minimally acceptable level of general oral proficiency in English. More specifically, the test aims to assess the “readiness” of AI candidates to conduct classes in English, a matter which entails using English both to give lectures and to interact with students, many of whom may not have had previous contact with nonnative English speakers, and hence may not be accustomed to “accented” speech. The interview provides candidates with the opportunity to demonstrate that they can communicate freely in English. Although candidates are not expected to communicate in English exactly like native speakers, it is expected that their proficiency level of spoken English will not hinder undergraduate student comprehension of any given subject/lecture.

Back to top

B. Interviewers

The pool of interviewers consists of approximately 20 instructors from the the Intensive English Program in the Department of Second Language Studies. All have advanced degrees in linguistics and/or second language instruction as well as many years of experience in the field of Teaching English to Speakers of Other Languages (TESOL) and testing. Their training as interviewers, moreover, is on-going, with training and calibration sessions conducted quarterly, to coincide with each TEPAIC. Apprentice interviewers, besides undergoing the training, are paired with experienced mentors for their first two interviewing sessions. The training includes: a review of interviewing skills, including types of questions and methods of employing ”probes” (i.e., questions which progressively probe more complex topics and demand more sophisticated language skills); discussion of criteria employed in rater judgments; and calibration (i.e., focused adjustment and standardization of raters’ employment of the measuring instrument) using previous interviews to confirm that all interviewers have internalized the rating criteria and that they apply the criteria in the same way.

Back to top

C. The Interview Process

Two raters (one male and one female, whenever possible) conduct a fifteen-minute interview with each international graduate student. During an initial introduction, candidates are asked to show a picture identification and give their current contact information (address, student identification number, telephone, department, etc.). The raters also mention that the session is not a “test” per se, as there are no correct or incorrect answers. Rather, it is a conversational setting designed to give the candidates the optimum opportunity to express themselves in a wide variety of topic areas. Since candidates are often extremely nervous, interviewers attempt to establish a casual atmosphere and a modicum of rapport.

After the initial procedures, the interviewers may proceed in a number of directions, often dictated by the candidates’ responses. At some point, there is usually a discussion of the candidates’ experience with various cross-cultural issues, ranging from comparing the climate and living conditions of their own country with those in the U.S. to comparing the educational system or student conduct in each country.

A second common thread of conversation is a discussion of the candidates’ fields of study, giving them the opportunity to present some simple technical information to the raters, who, as a relatively naive audience, need simple, coherent, yet adequately detailed explanation in order to understand the specialized topics. (See Rationale below.)

Besides these two major conversational threads, candidates are given probes which require that they project themselves into both future and hypothetical situations, each of which require specific grammatical devices to indicate the difference between real and possible situations.

If interviewers notice specific problems with a particular area of competence, they may probe this topic further to discern whether this problem will be an impediment to understanding for a prospective undergraduate audience.

Back to top

D. Rationale for the Oral Interview

In an English proficiency exam for instructors of English-speaking undergraduate students, an oral proficiency measure is needed, because AI’s, in addition to lecturing, must be able to comprehend student questions and respond to them with intelligible explanations. Secondly, a measure of candidates’ spoken English, separate from a written examination, is needed, because research clearly shows that speaking ability is not accurately predictable on the basis of results from an examination that emphasizes written English (c.f. Dandonoli and Henning, 1990; ETS, 1992). This is not surprising, given that research has demonstrated that speaking and listening are separate skills (Bachman, 1981; Dandonoli and Henning, 1990).

A measure of general English proficiency, as opposed to specific or special purpose English (e.g., English for chemists, or English for mathematicians) is included in the battery for several reasons. First, a field-specific approach would require separate examinations for each area of specialization. This would be very difficult because it would require the involvement of content-area specialists who were quite knowledgeable about language use and who had extensive experience in applied linguistics and in language testing. The participation of such uniquely qualified individuals would be needed both for input in the test development stages and for interviewing and rating purposes at every single administration of the test. Furthermore, the status of such specific-purpose tests is still controversial in the language testing profession. However, the TEPAIC Oral Interview does provide opportunities for candidates to discuss topics on general themes within their specialization, in the relevant context of explaining a general concept in their field to a novice learner. Such explanations typically do not go into the technical depth required in true specific-purpose tests, and hence do not require expert judges in each content area to rate the performance. Furthermore, since the interviewers are not specialists in the candidates’ fields, a candidate would not be marked down for providing inaccurate technical information in responding to such a question. The purpose of this type of question is simply to provide a thematically relevant context in which the candidate may speak to demonstrate her or his English language proficiency.

A second reason that general oral proficiency (as opposed to field-specific language ability) is targeted by the AI test is that in many aspects of their teaching, teachers need to use general English that does not relate directly to the content of lectures. For example, they need to be able to explain what homework is to be done, when it is due, how it can be submitted, what chapters to study for a test, how much each section of the test counts toward the students’ grade, and so forth. Teachers also need to be able to comprehend questions raised by students, and students typically use general or colloquial language, as opposed to, or at least in addition to, formal or special purpose language specific to the subject being taught.

Back to top

E. Background

For the TEPAIC interview, a Holistic Rating System is used, as in well-known oral interviews such as the American Council on the Teaching of Foreign Languages Oral Proficiency Interview (ACTFL OPI) and the Foreign Service Institute (FSI) interview. Separate evaluation forms help examiners remember specific candidates after the interview. When the candidates finish the interview and leave the room, the interviewers complete the evaluation form. There is little if any discussion necessary and no “coming to terms or agreement”. Both interviewers must individually and separately decide whether or not the candidate successfully completed the interview.

Back to top

F. Explanation of Holistic Scoring

Holistic evaluation is generally used in language testing for open-ended production skills, such as composition grading and oral interviews; otherwise, given the constraints of time and material, the number of areas which discrete-item scoring would need to consider are so numerous as to make such scoring untenable. This is particularly true with oral interviews since the evaluation is being done as candidates proceed through the interview. As Buck (cited in Cohen, 1994) notes “language comprehension is, by its very nature, multidimensional, and testing it only increases the number of dimensions involved in the process. In such a case it is obviously not possible to say what each item measures” (p. 261).

Back to top

G. Description of final evaluative categories on the Oral Interview Form

The Oral Interview for the TEPAIC is adapted from several common models of Oral Proficiency Interview (OPI). It is a test of oral proficiency, as opposed to a placement, achievement, or diagnostic test. Thus, it is designed to give a measure of the overall communicative competence of the interviewee, not to test retention and comprehension of specific material (achievement tests), to assign speakers to a particular level of instruction (placement tests), or to give impressions of specific deficiencies in discrete aspects of oral production (diagnostic tests). One of the best known OPIs, the ACTFL, on which the TEPAIC Oral Interview is loosely based, is mainly a placement test, placing speakers into one of 10 levels of competence (3 intermediate levels each for novice, intermediate, and advanced, with an added category superior). In the IU TEPAIC, however, there is no need for such fine distinctions across all levels, since we are only concerned with ascertaining that candidates have the necessary communicative competence to give lectures, lead discussions, answer questions spontaneously, and negotiate communicative breakdowns. Therefore, all candidates for the TEPAIC interview should be at or above the ACTFL Advanced Level. A detailed explanation of the competence required for each of the ACTFL proficiency levels is available at www.actfl.org.

Back to top

H. Evaluation Process

Immediately after the interview, the two raters independently assign a holistic rating (high pass, pass, borderline pass, borderline fail, fail) and then compare their findings. In the majority of cases, they agree on the holistic rating. In the case of disagreement, there is no negotiation; each rater simply assigns a separate score, and the lower rating is given for the overall evaluation.

Having given the rating, interviewers then assess problem areas of individual language skills by means of the analytical assessment form. This assessment has no relation to the holistic score already assigned, and serves only to give the candidate additional information about possible areas in need of training. It must be stressed that candidates rarely fail on the basis of one particular flaw in language use; usually there are problems in many areas, some of which, such as a lack of sophistication in vocabulary or grammar, are difficult to describe, precisely because they are indicated by the absence of more sophisticated vocabulary or grammar rather than the presence of any inappropriate elements.

The categories used on the analytical instrument are:

1. Aural Proficiency (Comprehension, Appropriate Response)
2. Pronunciation (Consonants, Vowels, Intonation, Word Stress, Enunciation)
3. Language (Grammar, Vocabulary)
4. Communicative Strategies (Self-correction, Appropriate questions, Communicative Initiative, Rephrasing, Awareness of Listener)
5. Fluency (Speed, Smoothness, Sentence stress)

Back to top

I. Holistic terms (C1, C2, C3) for the Oral Interview

The holistic evaluation groups the candidates into one of five levels:

• High Pass—Proficiency is near that of native speakers; they can discuss any topic offered with ease; they produce idiomatic phrasing; they use appropriate grammatical constructions and vocabulary; they can understand questions and answer appropriately; they rarely suffer from communication breakdown.
• Pass—Although not native-like, the speaker can handle any communicative situation which presents itself; intonation and pronunciation, although non-native, do not impede understanding; grammar and vocabulary are sophisticated, with only occasional errors; communication breakdowns are easily overcome through repair strategies.
• Borderline Pass—The speaker, although clearly non-native in intonation and pronunciation, is yet able to negotiate any topic with minimal difficulty; grammar and vocabulary are adequate, although less sophisticated than with a clear pass; the speaker uses a variety of communicative strategies to overcome deficiencies in other areas; the speaker may have occasional communication breakdowns, but can successfully employ appropriate repair strategies.
• Borderline Fail—Some or all of the criteria in the categories listed above are inadequate; intonation and pronunciation are impediments to comprehension; grammar and vocabulary are rudimentary; a halting presentation style shows difficulty in accessing aspects of language structure; a speeded up style indicates lack of awareness of prosodic elements (grammatical constructions, discourse features such as question and statement intonation, etc); inappropriate response shows lack of comprehension on the speaker’s part; the speaker lacks repair strategies to negotiate communication breakdowns.
• Fail—Communication is difficult because of severe problems in some or all of the areas listed in Borderline Fail. Pronunciation, intonation, grammar, vocabulary and comprehension may all show problems significant enough to render understanding problematic; communication strategies, even if present, cannot compensate for severity of deficiencies. An interviewee who receives a Fail was probably not prepared for participation in this test and may need significant training time in order to successfully complete it.

Back to top

Last updated: 7 July 2008
URL: http://www.indiana.edu/~deanfac/tepaic/
Comments to: kjbastin@indiana.edu
Contributors to this Web site
Copyright 2001, the Trustees of Indiana University