Education Application Testing Perspective to Empower Students' Higher Order Thinking Skills Related to The Concept of Adaptive Learning Media

Hernawan Sulistyanto(1*), Sofyan Anif(2), Sutama Sutama(3), Sabar Narimo(4), Anam Sutopo(5), Muhammad Izzul Haq(6), Gamal Abdul Nasir(7)

(1) Faculty of Teacher Training and Education, Universitas Muhammadiyah Surakarta
(2) Faculty of Teacher Training and Education, Universitas Muhammadiyah Surakarta
(3) Faculty of Teacher Training and Education, Universitas Muhammadiyah Surakarta
(4) Faculty of Teacher Training and Education, Universitas Muhammadiyah Surakarta
(5) Faculty of Teacher Training and Education, Universitas Muhammadiyah Surakarta
(6) Faculty of Arts, McGill University
(7) Sultan Hasanal Bolkiah Institute of Education, Universiti Brunei Darussalam
(*) Corresponding Author

Abstract

This article aims at arguing for the importance of the testing step when designing an educational application by taking a case study from the development of adaptive learning media. The media contains a set of in-struments that are specifically built to empower students' critical thinking skills. Three aspects that are con-sidered in testing this educational application are application validity at each stage of system development, measurement of the final system feasibility test for user needs, and system implementation by running learning media on the test sample. Implementation of testing on application products is carried out accord-ing to system requirements and models. The existence of the characteristics of adaptive media and the diver-sity of menus in the application implies the importance of doing a lot of improvisation when carrying out tests, such as determining the right test cases, choosing the appropriate test model and method, determining a suitable test environment, and considering several other aspects aimed at optimizing test results. obtained in order to ensure the quality of learning media products. This study analyzed the test data using Likert scale as an interpretation of the results of the validation assessment from the experts by referring to certain perceived standards of assessment. Meanwhile, the analysis of the data from the feasibility test results from a sample of 20 students using the system usability scale (SUS) instrument. The technique to test the effec-tiveness was using a pretest-posttest control group design with a sample of 98 students. Parametric/non-parametric data analysis was then applied to analyze the data on the results of testing the effectiveness or efficacy of adaptive media products in improving students' higher order thinking skills (HOTS). Based on the testing steps applied to the application of adaptive learning media, the results obtained that the product was considered feasible and effective in empowering students' HOTS. The study concludes that the educa-tional application testing that has been carried out is able to provide an objective and independent view of the application of adaptive learning media which will be useful in operational functions to understand the level of effectiveness in its implementation before being widely used in learning.

Keywords

adaptive learning media; education application; HOTS; measurement; system development

Full Text:

PDF

References

Afandi, A., Sajidan, S., Akhyar, M., & Suryani, N. (2018). Pre-Service Science Teachers’ Perception About High Order Thinking Skills (HOTS) in the 21st Century. International Journal of Pedagogy and Teacher Education. https://doi.org/10.20961/ijpte.v2i1.18254

Arief Maulana, Arief Kurniawan, Wini Keumala, Verdian Ramadika Sukma, A. S. (2020). Pengujian Black Box pada Aplikasi Penjualan Berbasis Web Menggunakan Metode Equivalents Partitions (Studi Kasus: PT Arap Store). Jurnal Teknologi Sistem Informasi, 3(1), 23–33. https://doi.org/DOI: http://dx.doi.org/10.32493/jtsi.v3i1.4307

Baoling, W., Xiaohui, C., Peiyi, L., Huilin, J., Hanxiang, G., Jia, L., & Huimin, L. (2020). Development, operational dilemma and tentative idea for construction mechanism of fever clinic in China. Zhonghua Wei Zhong Bing Ji Jiu Yi Xue. https://doi.org/10.3760/cma.j.cn121430-20200213-00054

Bedjou, K., Azouaou, F., & Berkani, L. (2015). Semantic recommendation of web services in the context of on-line training. 2014 4th International Symposium ISKO-Maghreb: Concepts and Tools for Knowledge Management, ISKO-Maghreb 2014. https://doi.org/10.1109/ISKO-Maghreb.2014.7033468

Bimba, A. T., Idris, N., Al-Hunaiyyan, A., Mahmud, R. B., & Shuib, N. L. B. M. (2017). Adaptive feedback in computer-based learning environments: a review. Adaptive Behavior. https://doi.org/10.1177/1059712317727590

Chang, H. Y., Wang, C. Y., Lee, M. H., Wu, H. K., Liang, J. C., Lee, S. W. Y., Chiou, G. L., Lo, H. C., Lin, J. W., Hsu, C. Y., Wu, Y. T., Chen, S., Hwang, F. K., & Tsai, C. C. (2015). A review of features of technology-supported learning environments based on participants’ perceptions. Computers in Human Behavior. https://doi.org/10.1016/j.chb.2015.06.042

Cheng-Gang, B., J. Chang-Hai, and C. Kai-Yuan. (2010) A reliability improvement predictive approach to software testing with Bayesian method, in IEEE Pro-ceeding of the 29th Chinese Control Conference, July 29-31, Beijing, China, pp. 6031-6036.

Clune, T.L., and R.B. Rood. (2011). Soft-ware testing and verification in climate model development, IEEE Journal, Fo-cus: climate change software, Septem-ber-October, pp. 49-55.

Drissi, S., & Amirat, A. (2016). An adaptive e-learning system based on student’s learning styles: An empirical study. International Journal of Distance Education Technologies. https://doi.org/10.4018/IJDET.2016070103

Gehring, J., Auli, M., Grangier, D., Yarats, D., & Dauphin, Y. N. (2017). Convolutional sequence to sequence learning. 34th International Conference on Machine Learning, ICML 2017.

Jiang, F. and Y. Lu, (2012). Software testing model selection research based on yin-yang testing theory, in IEEE Proceeding of International Conference on Computer Science and Information Processing (CISP), pp. 590-594

Jin, J., and F. Xue (2011). Rethinking soft-ware testing based on software architec-ture, in IEEE Proceeding of 7th Interna-tional Conference on Semantics, Knowledge and Grids, pp. 148-151. DOI 10.1109/SKG.2011.32

Jingyun, W., & Takahiko, M. (2015). The Reliability and Validity of Felder- Silverman Index of Learning Styles in Mandarin Version. Information Engineering Express.

Kashani-Vahid, L., Afrooz, G. A., Shokoohi-Yekta, M., Kharrazi, K., & Ghobari, B. (2017). Can a creative interpersonal problem solving program improve creative thinking in gifted elementary students? Thinking Skills and Creativity. https://doi.org/10.1016/j.tsc.2017.02.011

Khodadi, I., & Abadeh, M. S. (2016). Genetic programming-based feature learning for question answering. Information Processing and Management, 52(2), 340–357. https://doi.org/10.1016/j.ipm.2015.09.001

Kumamoto, H., et.al. (2010). Destructive testing of software systems by model checking, IEEE Journal, pp. 261-266.

Kurniawan, D., Astalini, A., Darmaji, D., Tanti, T., & Maryani, S. (2022). Innovative Learning: Gender Perception of e-Module Linear Equations in Mathematics and Physics. Indonesian Journal on Learning and Advanced Education (IJOLAE), 4(2), 92–106.

Kusuma, M. D., Rosidin, U., Abdurrahman, A., & Suyatna, A. (2017). The Development of Higher Order Thinking Skill (Hots) Instrument Assessment In Physics Study. IOSR Journal of Research & Method in Education (IOSRJRME), 07(01), 26–32. https://doi.org/10.9790/7388-0701052632

Lamas, E., A.V. Dias, and A.M. da Cunha. (2013). Applying testing to enhance software product quality, in IEEE Pro-ceeding of 10th International Confer-ence on Information Technology: New generation, pp. 349-356. DOI 10.1109/ITNG.2013.56

Lei, Y., & Jiang, Y. (2010). Chinese question classification in community question answering. Proceedings - 2010 IEEE International Conference on Service-Oriented Computing and Applications, SOCA 2010, 1–6. https://doi.org/10.1109/SOCA.2010.5707167

Lemos, O.A.L., et. al. (2011). “Evaluation studies of software testing research in the Brazilian symposium on software engineering”, in IEEE Proceeding of 25th Brazilian Symposium on Soft-ware Engineering, pp. 56-65. DOI 10.1109/SBES.2011.30

Martins, A. I., Rosa, A. F., Queirós, A., Silva, A., & Rocha, N. P. (2015). European Portuguese Validation of the System Usability Scale (SUS). Procedia Computer Science. https://doi.org/10.1016/j.procs.2015.09.273

Nagao, K., & Nagao, K. (2019). Artificial Intelligence in Education. In Artificial Intelligence Accelerates Human Learning. https://doi.org/10.1007/978-981-13-6175-3_1

Nakagawa, E.Y., and J.S. Maldonado. (2011). Contributions and perspectives in architectures of software testing envi-ronments, in IEEE Proceeding of 25th Brazilian Symposium on Software En-gineering, pp. 66-71. DOI 10.1109/SBES.2011.42

Purmadi, A., & Surjono, H. D. (2016). Pengembangan Bahan Ajar Berbasis Web Berdasarkan Gaya Belajar Siswa untuk Mata Pelajaran Fisika. Jurnal Inovasi Teknologi Pendidikan. https://doi.org/10.21831/jitp.v3i2.8285

Reza, H., and S. Lande, 2010, Model based testing using software architecture, in IEEE Proceeding of 7th International Conference on Information Technology, pp. 188-192. DOI 10.1109/ ITNG.2010.122

Sauro, J. (2011). Measuring Usability With The System Usability Scale (SUS). Measuring Usability.

Schwan, S., Dutz, S., & Dreger, F. (2018). Multimedia in the wild: Testing the validity of multimedia learning principles in an art exhibition. Learning and Instruction, 55, 148–157. https://doi.org/10.1016/j.learninstruc.2017.10.004

Seechaliao, T. (2017). Instructional Strategies to Support Creativity and Innovation in Education. Journal of Education and Learning. https://doi.org/10.5539/jel.v6n4p201

Setyaningsih, E., Agustina, P., Anif, S., Ahmad, C., Sofyan, I., Saputra, A., Salleh, W., Shodiq, D., Rahayu, S., & Hidayat, M. (2022). PBL-STEM Modul Feasibility Test for Preservice Biology Teacher. Indonesian Journal on Learning and Advanced Education (IJOLAE), 4(2), 118–127.

Seventika, S. Y., Sukestiyarno, Y. L., & Mariani, S. (2018). Critical thinking analysis based on Facione (2015) - Angelo (1995) logical mathematics material of vocational high school (VHS). Journal of Physics: Conference Series. https://doi.org/10.1088/1742-6596/983/1/012067

Sommerville, I. (2011). Software engineering, 9th Edition, Pearson Education, USA.

Sugiyono. (2017). MetodePenelitian Kuantitatif, Kualitatif dan R&D. Bandung: PT Alfabet. In Sugiyono. (2017). MetodePenelitian Kuantitatif, Kualitatif dan R&D. Bandung: PT Alfabet. https://doi.org/10.1017/CBO9781107415324.004

Sulistyanto, H., Nurkamto, J., Akhyar, M., & Asrowi. (2019). A review of determining the learning style preferences by using computer-based questionnaires on undergraduate students. Journal of Physics: Conference Series. https://doi.org/10.1088/1742-6596/1175/1/012209

Tsortanidou, X., Karagiannidis, C., & Koumpis, A. (2017). Adaptive educational hypermedia systems based on learning styles: The case of adaptation rules. International Journal of Emerging Technologies in Learning. https://doi.org/10.3991/ijet.v12i05.6967

Vong, S. A., & Kaewurai, W. (2017). Instructional model development to enhance critical thinking and critical thinking teaching ability of trainee students at regional teaching training center in Takeo province, Cambodia. Kasetsart Journal of Social Sciences, 38(1), 88–95. https://doi.org/10.1016/j.kjss.2016.05.002

Wang, J., & Mendori, T. (2016). A Study of the Reliability and Validity of Felder-Soloman Index of Learning Styles in Mandarin Version. Proceedings - 2015 IIAI 4th International Congress on Advanced Applied Informatics, IIAI-AAI 2015. https://doi.org/10.1109/IIAI-AAI.2015.284

Wen-hong, L. and W. Xin. (2012). The software quality evaluation method based on software testing, in IEEE Pro-ceeding of International Conference on Computer Science and Service System, pp. 1467-1471. DOI 10.1109/CSSS.2012.369

Wu, H., & Leung, S. O. (2017). Can Likert Scales be Treated as Interval Scales?—A Simulation Study. Journal of Social Service Research. https://doi.org/10.1080/01488376.2017.1329775

Wu, Y., Y. Zhang, and M. Lu. (2010). Software reliability accelerated testing method based on mixed testing, IEEE Journal.

Xie, X., Song, W., Liu, L., Du, C., & Wang, H. (2015). Research and implementation of automatic question answering system based on ontology. The 27th Chinese Control and Decision Conference (2015 CCDC), 1366–1370. https://doi.org/10.1109/CCDC.2015.7162131

Xu, X., Cheng, X., Tan, S., Liu, Y., & Shen, H. (2013). Aspect-level opinion mining of online customer reviews. China Communications, 10(3), 25–41. https://doi.org/10.1109/CC.2013.6488828

Yang, M. C., Lee, D. G., Park, S. Y., & Rim, H. C. (2015). Knowledge-based question answering using the semantic embedding space. Expert Systems with Applications, 42(23), 9086–9104. https://doi.org/10.1016/j.eswa.2015.07.009

Yang, Y., L. Lun, and X. Chi. (2011). Re-search on path generation for software architecture testing matrix transform-based, IEEE Journal, pp. 2483-2486.

Yoshii, A., & Nakajima, T. (2012). Study of a conversational agent system encouraging “real” answers of individuals in a group of acquaintances. Proceedings - IEEE 9th International Conference on Ubiquitous Intelligence and Computing and IEEE 9th International Conference on Autonomic and Trusted Computing, UIC-ATC 2012, 143–150. https://doi.org/10.1109/UIC-ATC.2012.124

DOI: 10.23917/ijolae.v4i3.19432

Article Metrics

Abstract view(s): 475 time(s)
PDF: 420 time(s)

Refbacks

  • There are currently no refbacks.