RECONSIDERING THE USE OF INTERNATIONAL PROFICIENCY TESTS: DESIGNING ‘SELF-MADE SPEAKING PROFICIENCY TEST’

The purpose of this paper is to share how State Polytechnic of Semarang conducts regular speaking test for all the sixth semester students. Due to the different purpose of TOEFL from the students’ language learning purpose and the high cost of TOEIC that our students unlikely are able to afford, we design our own proficiency test. The program has been started by conducting a needs analysis on the graduates’ English communication competences. The survey found that there were 20 oral and 24 written topics that should be acquired by the graduates. It means that the test should cover four language skills. After long tough discussion on the practicality and the limited budget versus the validity and the reliability of the speaking test and considering the suggestions of invited experts in some workshops then it is constructed Self-made Speaking Test for Students of Polytechnic. The design of the speaking test is in short described as follows: Each speaking test taker has to do two speaking tasks, one monologue and one dialogue. The tasks are taken randomly. The dialogue is done in pairs. The test takers have their freedom to choose their partners from the same study program. Each pair is given 15 minutes to prepare and 15 minutes to perform. The performance is observed analytically. The elements observed are: task, coherence, pronunciation, grammar, vocabulary and fluency. The graders are English lecturers who have undergone training of graders. There is a comfortable room designed for the speaking test. It is found that the role of the testing officer is very crucial in this test. Meanwhile, some activities to guard its validity and reliability are: analytical marking, grader training, checking the validity of the test regularly.

Key words: Speaking test, validity, reliability, and practicality

Introduction
Testing is an important and integral part of English instructions. The proper relationship between teaching and testing is surely that of partnership (Hughes, A., 2003:2). Distracting problem is rarely met in designing a good test for reading, writing and listening skills for large number of students. But there is in designing speaking skill (Thornbury, 2005). The problem lays on its practicality versus its validity, reliability and authenticity. It is inevitable that to have valid, reliable and authentic speaking test will not only be time consuming but also energy and money absorbing, in short, when the validity, reliability and authenticity of a speaking test are put forward, the practicality is hard to attend (Harsono, 2007; Weir, 1990).
Many teachers are reluctant to test students’ speaking skill by asking them to speak. Instead, they (still) use written English proficiency test to predict the students’ true speaking skill. To predict the speaking skill of large number of students, paper-based TOEFL-Like and TOEIC-Prediction tests are likely their first choices. They avoid using the real international proficiency tests as there is very limited budget provided. Furthermore, there must be some other reasons for the fact that there is a wide use of TOEFL-Like and TOEIC-Prediction tests by many educational institutions despite the fact that the action is considered a violation of copyright law. For there are many companies use the tests for one means of their employee recruitment, the educational institutions can not neglect the reality (Language Center of State Polytechnic of Semarang, 2004). To get the graduates to have the ability to penetrate in the work world is a very crucial matter to concern. Therefore, they make their students experiencing in taking such tests. Moreover, numerous grant providers of tertiary educational development programs like, Due-Like, SP4 or PHKI usually require the level of the students’ English ability informed with the score of (paper-based) TOEFL. Additionally, the teachers’ lack of awareness of copyright law may be the other reason for using these most famous international proficiency tests’ names.
Having responsiveness to those matters, State Polytechnic of Semarang i.e. the Language Centre has come to the decision of designing the self-made speaking test.
The purpose of this paper is to share how we conduct regular speaking test for all the sixth semester students of State Polytechnic of Semarang. There are about 480 students of semester sixth from all classes of two commerce departments who have to join the speaking test and get the passing grade of 4.5 (possible 9.0). The test is centrally conducted in one season annually by the Language Centre of the institution. The other 792 students from the rest three departments are expected to join the regular speaking test.

The Rationale Underlies the Designing the Self-made Speaking Test
There are five underlying principles of the determination. First, the self-made speaking test is designed to meet the students’ learning purpose (Hughes, R., 2002). To set the purpose of the speaking instructions the language centre has done a survey on the needs analysis on the graduates’ English communication competences (Thornbury, 2005). The survey found that there are ten speaking topics which should be acquired by the graduates of State Polytechnic of Semarang (Language Center of State Polytechnic of Semarang, 2004), as it is shown in Table 1. The students need to be able to speak with confidence in order to carry out at least the tasks which are shown through the topics (Bygate, 1987). On the other hand, the TOEIC test, for example, evaluates the capacity of non-native speakers of English to communicate in the international workplace (www.toeic,or,id), and TOEFL measures the ability of non native speakers of English to use and understand English as it is spoken, written, and heard in college and university settings (www.ets.org/portal/site/ets). Those international tests have broader purposes than the students’ learning purpose.

Table 1: Topics for Speaking Test
No Topic Description
1 Introducing self and others Giving and asking for personal information; doing Introducing self and others in English/international style; doing short conversation on ‘safe topics’ after introducing.
2 Giving and following directions Giving directions for certain rooms, giving directions for certain place in an area (e.g. campus) or in a city using map.
3 Giving argument giving opinion, asking questions, giving argument.
4 Making appointment Asking and answering about days, dates, time, and places, agreeing and disagreeing, telephoning skills.
5 Business presentation Describing plans, describing graph, doing business presentation, handling questions
6 Job interview Telling the job preferences, talking about work experience,
7 Persuasive communication on product and service Describing product and service
8 Meeting Reading agenda, giving opinion, asking questions, interrupting, agreeing opinion
9 Making suggestion Making strong suggestions and giving options
10 Describing company profile Describing organisation structure, locations, product, history of a company.

The second is the design of the speaking test is adjusted to the capabilities of the institution as well as its limitations (Hughes, A., 2003). It is not easy to get the balance of the four principles of a good test, especially when there is only very limited budget. After working hard on the designing and doing careful calculation on the use of the speaking test, at present the centre needs to charge a test taker not more than Rp15.000 for once speaking test taken.
The third belief is that there is a backwash of the speaking testing; there will be more speaking in class (Hughes, A., 2003; Weir, 1990). The teachers will put more speaking in the class sessions and the students will be more motivated to practice speaking knowing that the speaking test is speaking activity not multiple-choice objective tests. It goes to what Thornbury (2005:125) remarks: “Furthermore, where teachers or students are reluctant to engage in much classroom speaking, the effect of an oral component in the final examination can be powerful incentive ‘to do more speaking’ in class. This is known as the ‘washes back’ into the coursework that precedes it.”
The fourth, the process of designing the self-made speaking test has increased teachers’ awareness and involvement in developing a good test and the quality of speaking instructions (Hughes, A., 2003). It results in, at the end, improving not only the quality of the graduates but also the teachers’. The fifth principle is that, by using the self-made speaking test, it means we do not break the copyright law- i.e. using the name of International English proficiency test without permission.

The Challenges in Designing the Self-made Speaking Test
There are two exigent challenges apart from meeting the four criteria of a good speaking test i.e. validity, reliability, authenticity and practicality, namely getting internal support and recognition from external institutions. It was not easy to convince all 28 English teachers in State Polytechnic of Semarang to accept the idea of designing the self-made speaking test. While trying out and improving the early speaking test by considering the suggestions of invited experts in some workshops, were being done long tough discussions and consistent promotion were continuously through.
At present we have not thought of the recognition from external institutions yet as we believe that it will come automatically when the test fulfilled the four basic criteria.
Designing the self-made speaking test is also a kind of education for the students and the institution to respect copyright.

The Proficiency Speaking Test for the Students of Polytechnic
When speaking is one of the language skills that students need to have, the teachers have to teach them speaking. As the students must learn speaking, to test them speaking is the teachers’ responsibility (Underhill. 1987:12). A written multiple-choice test which is still highly used by many institutions is not an appropriate way to know the students’ true speaking skill. To get the accurate picture of the skill a teacher should get the students speak. The speaking test for the students of Polytechnic is a part of the English Proficiency Test for the Students of Polytechnic. The following table shows the relationship.

Table 2: English Proficiency Test for Students of Polytechnic (Polytechnic EPT)
Part Components Number of items Time Weight Remark Administration
1 Listening 30 ± 30’ 25%
Test booklet Classical
2 Reading, grammar & Vocabulary 50 60’ 25%
3 Writing 1 30’ 25%
4 Speaking 2 (1 mono-logue & 1 dialogue) 15’ 25% Separated Individual and In pairs

Parts 1 (Listening), 2 (Reading, grammar & Vocabulary) and 3 (Speaking) are conducted classically, they are done in written. Meanwhile, part 4, the speaking test is carried out separately, it is scheduled differently. Special arrangement is made for it. The key points of the design of the speaking test are listed below.
1. Each speaking test taker has to do two speaking tasks, one monologue and one dialogue (Weir,C.J.1990:78-79). The tasks are taken randomly.
2. The dialogue is done in pairs. The test takers have their freedom to choose their partner from the same study program (Underhill. 1987:53).
3. Each pair is given 15 minutes to prepare and 15 minutes to perform (Carroll, B.J. and P.J. Hall. 1985:65; Weir,C.J.1990:75).
4. The performance is observed analytically (Weir,C.J.1990:66). The elements observed are: task, coherence, pronunciation, grammar, vocabulary and fluency.
5. The graders are English lecturers who have undergone trainings of graders (Hughes. 2003:133; Underhill. 1987:91; Weir,C.J.1990:82).
6. There is a comfortable room designed for the speaking test (Carroll, B.J. and P.J. Hall. 1985:64).
Besides those points, the other things to consider are processing scores, giving written feedback of the speaking performance, producing certificates, and continuous development of the tasks. However, we limit the discussion on the six key points.

The Speaking Tasks
The topics found from the needs analysis survey were analysed to identify the speaking competence needed by the graduates. After the topics for the speaking identified (as described in Table 1), then, they were grouped into two: monologue (extensive speaking) and dialogue- or some would describe as interpersonal and transactional speech events (Brown, 2004:167). The monologue and dialogue tasks were made based on the speaking topics (Underhill, 1987; Weir,1990; O’Malley, 1996). There were at least two tasks made for each topic to guard the test validity, reliability and anthencity.
It is hard to meet the practicality of the speaking test if we test every student all of the topics, so we assign every test taker to take the task randomly. The way is done by coding all of the provided speaking tasks. The codes are written on small cards, the cards for monologues are put in a box and for dialogue in different box. A test taker is asked to take one monologue card at random, and together with his/her partner, they, take a dialogue card at random, too. Next, the cards are exchanged with task sheets. Together with the task sheets, the presentation aids like a map or graph printed on OHP sheets are provided. An example of a dialogue task is presented below.