It must be said that the TOEIC is not very useful as a precise measure of one’s English skills - It can only indicate an approximate level of development. In fact, the TOEIC is quite ineffective for measuring short term progress. (Short term is defined here as study for approximately 150 hours or less.) Unfortunately, many companies test after only 50 to 100 study hours for the purpose of measuring a students progress. This is wrong. TOEIC has published no guidelines for minimum intervals between testings. This shocking omission may be due to the fact that the more frequently companies use the test, the more money TOEIC makes.

Although TOEIC claims an SEM of 25 (SEM = Standard Error of Measurement - an SEM of 25 indicates that 68% of all test takers fall within a range of plus or minus 25 points from their true scores), various independent studies have indicated that the TOEIC SEM more accurately lies in the range of 35 to 47. Corporate training managers who have experience using this test can readily verify this by analyzing their own score distributions.

So how many hours of instruction are needed before the TOEIC will yield meaningful results? In a large-scale study conducted by Saegusa (Saugusa 1985), 53% of students who received an average of 84 hours of instruction had made gains of less than 50 points, a number which is less than the SEM and therefore statistically meaningless. Saugusa indicates that from 100 to 200 hours of instruction are needed before meaningful TOEIC scores can be obtained.

It is appropriate, however, for corporate training managers to use the TOEIC test to determine the overall effectiveness of their company’s English training program. Using a sample of 30 students or more, it is possible, after a least 100 hours of instruction, to compare group mean pretest and post-test scores and evaluate the results next to the standards indicated on these charts.

In a Research Paper titled “Good and Bad Uses of TOEIC by Japanese Companies,” Dr. Marshall Childs reaches the below five conclusions about the TOEIC test. This information is presented here with the permission of Dr. Childs.


1) Measuring overall group gains in proficiency.

Conclusion: Good under some circumstances.
The TOEIC can be used to differentiate some group mean gains between large groups of learners with the caveats that careful measurement of statistical significance is necessary in order to distinguish real gains from illusory ones, and that even if the significance of the gains is established, the causes of the gains may remain problematic.


2) Comparing the performance of different schools or treatments.

Conclusion: Good under some circumstances.
The administrator must be aware that the difficulty of raising a TOEIC score is considerably greater at the upper end of the scale than at the lower end.


3) Gauging the progress of individual learners.

Conclusion: Bad.
The use of the TOEIC for gauging individual learning is, in general, inefficient or wrong. The variability of TOEIC results defeated their usefulness in measuring learning gains because the SEM (standard error of measurement) was in the range of expected individual gains.


4) Counseling learners on their progress.

Conclusion: Bad.
Because of the SEM of TOEIC, test-to-test differences will display very great variability. For example, differences may be negative or they may be very large-and somewhat illusory in both cases. Indeed, the lower results that are frequently encountered in successive tests can have the unfortunate side-effect of demotivating learners.


5) Guide the course of study of individual learners.

Conclusion: Bad.
TOEIC is not a diagnostic test and it can not pinpoint learners’ strengths and weaknesses. It can be a rough guide for gauging a learner’s overall level, if the administrator clearly understands the statistical variability of the results.



Sources: http://www.prolingua.co.jp/toeic.html 2