Development of a Computerized-Adaptive Test to Measure English Vocabulary Size with IRT

Andhita Dessy Wulansari(1*), Dhinuk Puspita Kirana(2), Restu Mufanti(3)

(1) Faculty of Tarbiyah and Teacher Training, Institut Agama Islam Negeri Ponorogo
(2) Faculty of Tarbiyah and Teacher Training, Institut Agama Islam Negeri Ponorogo
(3) Faculty of Teacher Training and Education, Universitas Muhammadiyah Ponorogo, Indonesia; Faculty of Education, University of Technology Sydney, New South Wales, Australia
(*) Corresponding Author

Abstract

To speak English fluently like a native speaker, having a vocabulary of 20,000 words are required. In many universities in Indonesia, Vocabulary Size Test (VST) is still held with Paper-Based Test (PBT), so it is less effective and efficient. The aims of this study were 1) to develop Computerized-Adaptive Testing (CAT), which helps get students with a standardized English vocabulary size, and 2) to implement it in measuring vocabulary size. The approach in this study is Research and Development (R&D) with product development procedures following the steps of Borg and Gall. The findings of this study are, 1) the CAT program has Item Response Theory (IRT) parameters that can change according to the test takers' answers, and the test will stop if the estimated ability of the test takers is known (Standard Error/SE close to 0.01), and 2) with the application of the CAT program it can be seen that the total vocabulary of English Language and Teaching Department students is still <20,000. Based on these findings, this study implies that for students' vocabulary mastery to be on par with native speakers' abilities, the English Language and Teaching Department needs to evaluate the learning process and determine a particular program to increase student vocabulary.

Keywords

computerized-adaptive testing; item response theory; learning innovation process; vocabulary size test

Full Text:

PDF

References

Abera, M. (2020). The Impact of Cooperative Learning on Students' Paragraph Writing Skills: The Case of Third Year Health In-formatics Stu-dents at University of Gondar. Ethiopian Renaissance Journal of Social Sciences and the Humanities, 7(2).

Ammigan, R. (2019). Institutional satisfaction and recommendation: What really matters to international students. Journal of International Students, 9(1), 262-281.

Arjmandi, M., & Aladini, F. (2020). Improving EFL Learners’ Vocabulary Learning Through Short Story Oriented Strategy (SSOS). Theory and Practice in Language Studies, 10(7), 833-841.

Bagus, H. C., Tola, B., & Tjalla, A. Achievement Tests Administration using Computerized Adaptive Testing (CAT) with Constrain Response Time Item.

Brunn, G., Freise, F., & Doebler, P. (2022). Modeling a smooth course of learning and testing individual deviations from a global course. Journal for educational research online, 14(1), 89-121.

Castillo-Cuesta, L. (2020). Using digital games for enhancing EFL grammar and vocabulary in higher education. International Journal of Emerging Technologies in Learning (iJET), 15(20), 116-129.

Egbe, C. I., Agbo, P. A., Okwo, F. A., & Agbo, G. C. (2023). Students' Perception of Computer-Based Tests in the Use of English Programme in Nigerian Universities. TechTrends, 1-12.

Fadi, A.-K. (2019). The impact of vocabulary knowledge on the reading comprehension of Saudi EFL learners. Journal of Language and Education, 5(3 (19)), 24-34.

Gall, M. D., Borg, W. R., & Gall, J. P. (1996). Educational research: An introduction. Longman Publishing.

Godfroid, A. (2019). Sensitive Measures of Vocabulary Knowledge and Processing: Expanding Nation’s Framework 1. In The Routledge handbook of vocabulary studies (pp. 433-453). Routledge.

Goulden, R., Nation, P., & Read, J. (1990). How large can a receptive vocabulary be? Applied linguistics, 11(4), 341-363.

Hartono, D. A., & Prima, S. A. B. (2021). The correlation between Indonesian university students’ receptive vocabulary knowledge and their reading comprehension level. Indonesian Journal of Applied Linguistics, 11(1), 21-29.

Hermita, N., Putra, Z. H., Alim, J. A., Wijaya, T. T., Anggoro, S., & Diniya, D. (2021). Elementary Teachers' Perceptions on Genially Learning Media Using Item Response Theory (IRT). Indonesian Journal on Learning and Advanced Education (IJOLAE), 4(1), 1-20.

Hikmat, M. H., Santos, R. F., Suharyanto, S., Maudy, A. G., & Phommavongsa, K. (2022). Toward Continuous Innovation in Teaching: Reflective Practice on English Teaching of Indonesian and the Philippine Teachers. Indonesian Journal on Learning and Advanced Education (IJOLAE), 5(1), 45-60.

Hsu, W. (2020). Can TED talk transcripts serve as extensive reading material for mid-frequency vocabulary learning? TEFLIN Journal, 31(2), 181-203.

Husnanissa, A. (2020). Measuring English Students’vocabulary Size At The First Semester Of The Eighth Grade Of Smpn 5 Bandar Lampung In The Academic Year Of 2017/2018 UIN Raden Intan Lampung].

Khoshsima, H., Toroujeni, S. M. H., Thompson, N., & Ebrahimi, M. R. (2019). Computer-based (CBT) vs. paper-based (PBT) testing: Mode effect, relationship between computer familiarity, attitudes, aversion and mode preference with CBT test scores in an asian private EFL context. Teaching English with Technology, 19(1), 86-101.

Kirana, D. P., & Basthomi, Y. (2020). Vocabulary size among different levels of university students. Universal Journal of Educational Research, 8(10), 4357-4364.

Kohnke, L., Zhang, R., & Zou, D. (2019). Using mobile vocabulary learning apps as aids to knowledge retention: Business vocabulary acquisition. Journal of Asia TEFL, 16(2), 683.

Krashen, S. (1989). We acquire vocabulary and spelling by reading: Additional evidence for the input hypothesis. The modern language journal, 73(4), 440-464.

Krishnan, I. A., Ching, H. S., Ramalingam, S., Maruthai, E., Kandasamy, P., De Mello, G., Munian, S., & Ling, W. W. (2020). Challenges of learning English in 21st century: Online vs. traditional during Covid-19. Malaysian Journal of Social Sciences and Humanities (MJSSH), 5(9), 1-15.

Kusumaningrum, S., Setyawati, I. G., Sutopo, A., Ratih, K., Badri, T. I. A., & Tawandorloh, K. A. (2019). Snowball Throwing: An English Learning Method to Improve Vocabulary Mastery and Psychomotor Ability. Indonesian Journal on Learning and Advanced Education (IJOLAE), 2(1), 10-19.

Laufer, B., & Nation, P. (1999). A vocabulary-size test of controlled productive ability. Language testing, 16(1), 33-51.

Mizumoto, A., Sasao, Y., & Webb, S. A. (2019). Developing and evaluating a computerized adaptive testing version of the Word Part Levels Test. Language testing, 36(1), 101-123.

Nation, I. (2006). How large a vocabulary is needed for reading and listening? Canadian modern language review, 63(1), 59-82.

Nation, I. (2012). Measuring vocabulary size in an uncommonly taught language. International Conference on Language Proficiency Testing in the Less Commonly Taught Languages,

Noreillie, A.-S. (2019). It's all about words. Three empirical studies into the role of lexical knowledge and use in French listening and speaking tasks.

Pramono, A. J. B., & Retnawati, H. (2020). Implementation of CAT in Indonesia school: Current challenges & strategies. Universal Journal of Educational Research, 8(11), 5599-5609.

Ratih, K., Syah, M. F. J., Nurhidayat, N., Jarin, S., & Buckworth, J. (2021). Learning patterns during the disruptive situation in informal education: Parents’ efforts and challenges in the adjustment of progressive learning. Indonesian Journal on Learning and Advanced Education (IJOLAE), 3(3), 180-193.

Ridwan, W., Wiranto, I., & Dako, R. (2020). Ability estimation in computerized adaptive test using mamdani fuzzy inference system. IOP Conference Series: Materials Science and Engineering,

Ridwan, W., Wiranto, I., & Dako, R. (2021). Computerized adaptive test based on sugeno fuzzy inference system. IOP Conference Series: Materials Science and Engineering,

Sari, S. N., & Aminatun, D. (2021). students’perception on the use of English movies to improve vocabulary mastery. Journal of English language teaching and learning, 2(1), 16-22.

Schmitt, N. (2013). An introduction to applied linguistics. Routledge.

Schmitt, N., & Schmitt, D. (2020). Vocabulary in language teaching. Cambridge university press.

Setiawan, M. R., & Wiedarti, P. (2020). The effectiveness of Quizlet application towards students’ motivation in learning vocabulary. Studies in English Language and Education, 7(1), 83-95.

Suhardi, I. (2020). Comparison of Test Scores Using Paper and Computer Media as an Indicator of Computer Self-Efficacy and Test Anxiety in Facing Computer-Based-Testing. Nusantara Journal of Social Sciences and Humanities.

Susanto, A., Halim, F. A., & Thasimmim, S. N. (2019). Vocabulary learning strategies, vocabulary skills, and integrative motivation levels among university students. Vocabulary Learning Strategies, Vocabulary Skills, and Integrative Motivation Levels among University Students, 8(5C), 323-334.

Torres, J., & Alieto, E. (2019). English learning motivation and self-efficacy of Filipino senior high school students. Asian EFL Journal, 22(1), 51-72.

van Groen, M. M., & Eggen, T. J. (2019). Educational test approaches: The suitability of computer-based test types for assessment and evaluation in formative and summative contexts. Journal of Applied Testing Technology, 21(1), 12-24.

Wilkins, D. A. (1972). Linguistics in language teaching (Vol. 111). Edward Arnold London.

Wulansari, A. D., Kumaidi, K., & Hadi, S. (2019). Two Parameter Logistics Model with Lognormal Response Time for Computer-based Testing. International Journal of Emerging Technologies in Learning (iJET), 14(15). https://doi.org/10.3991/ijet.v14i15.10580

Yosintha, R. (2020). Indonesian students’ attitudes towards EFL learning in response to industry 5.0. Metathesis: Journal of English Language, Literature, and Teaching, 4(2), 163-177.

DOI: 10.23917/ijolae.v5i3.22953

Article Metrics

Abstract view(s): 308 time(s)
PDF: 200 time(s)

Refbacks

  • There are currently no refbacks.