Combining Usability Testing with In-Depth Interview for Online Credit Hour Website Evaluation

Pristi Sukmasetya(1*), Endah Ratna Arumi(2), Agus Setiawan(3),

(1) Universitas Muhammadiyah Magelang
(2) Universitas Muhammadiyah Magelang
(3) Universitas Muhammadiyah Magelang
(*) Corresponding Author


The Online Credits Hour (KRS) application is one of the existing systems at the University of Muhammadiyah Magelang where the academic community, including lecturers, students, and education staff, produces a large amount of student academic data. Evaluation is necessary to find out whether the system runs well. This study is an effort to evaluate the system employing a usability test consisting of five indicators, namely learning ability, memory, efficiency, errors, and satisfaction. The five indicators materialized in the form of questionnaires to online KRS users. The survey involved 118 respondents. The method of study includes validity and reliability tests. The validity test employs a correlation test. The reliability test uses a simple linear regression analysis test and the comparison uses the significance value Alpha equals 0.05. The validity test for all five indicators results in value above 0.05. The reliability test of all the answers in the questionnaire has a Cronbach's Alpha value of 0.93, which suggests that the whole has correspondence. The evaluation results show that the KRS Online application has the highest value in the memorability indicator at 3.97, indicating that the KRS Online application is easily accessible with easy procedures to obtain data. The lowest value is on the error indicator at a level of 3, which means that the KRS Online website shows many errors, such as a broken link or poor navigation.


usability; usability testing; usability evaluation

Full Text:



N. Binti and N. Rozali, “Usability Testing on Government Agencies Web Portal : A Study on Ministry of Education Malaysia ( MOE ) Web Portal,” 2015 9th Malaysian Softw. Eng. Conf., pp. 37–42, 2015.

A. Setiawan, R. A. Widyanto, P. Studi, T. Informatika, F. Teknik, U. M. Magelang, and P. H. Bersama, “Evaluasi Website Perguruan Tinggi Menggunakan Metode Usability Testing,” vol. 3, no. 3, pp. 295–299, 2018.

Y. F. Qiu, Y. P. Chui, and M. G. Helander, “Usability Analysis of Mobile Phone Camera Software Systems,” pp. 1–6, 2006.

P. Sukmasetya, H. B. Santoso, and D. I. Sensuse, “Current E-Government Public Service on User Experience Perspective in Indonesia,” 2018 Int. Conf. Inf. Technol. Syst. Innov., pp. 159–164, 2018.

J. Wang, “Usability testing on a government training platform: A case study,” 2010 2nd Int. Conf. Educ. Technol. Comput., vol. 2, pp. V2-211-V2-214.

D. I. Sensuse, P. Pudy, and P. Sukmasetya, “Improving e-Leaming through Knowledge Management,” 2017.

Z. Luo, Y. Wang, P. Wang, and A. Participants, “Usability Study on Railway Self-service Terminal Interface for the Elderly,” 2018 11th Int. Symp. Comput. Intell. Des., vol. 1, pp. 157–160, 2018.

A. Lodhi, “Usability Heuristics as an Assessment Parameter : for performing Usability Testing,” 2010 2nd Int. Conf. Softw. Technol. Eng., vol. 2, pp. V2-256-V2-259, 2010.

W. K. H. Zhevlwh, K. D. V Ehhq, G. Dqg, G. Ghshqgv, R. Q. Lwv, L. Dwwulexwhv, Z. Fryhuv, W. K. H. Ghwdlov, R. I. Kwpo, V. D. Q. G. Dovr, W. Duh, and F. Dqg, “Analysis of Website Usability Evaluation Methods,” pp. 1043–1046, 2016.

M. K. Othman, M. N. S. Sulaiman, and S. Aman, “Heuristic evaluation: comparing generic and specific usability heuristics for identification of usability problems in a living museum mobile guide app,” Adv. Human-Computer Interact., vol. 2018, 2018.

H. M. Az-zahra, “Usability Evaluation of User Interface in Badan Narkotika Nasional East Java Province Website,” 2018 Int. Conf. Sustain. Inf. Eng. Technol., pp. 262–265, 2018.

D. Quiñones and C. Rusu, “Applying a methodology to develop user experience heuristics,” Comput. Stand. Interfaces, vol. 66, p. 103345, 2019.

R. Kaur, “Comparative Study for Evaluating the Usability of Web Based Applications,” 4th Int. Conf. Comput. Sci., 2018.

E. Palagi, F. Gandon, A. Giboin, and R. Troncy, “A model-based heuristic evaluation method of exploratory search,” 2018.

Z. Ebnehoseini, M. Tara, M. Meraji, K. Deldar, F. Khoshronezhad, and S. Khoshronezhad, “Usability evaluation of an admission, discharge, and transfer information system: a heuristic evaluation,” Open access Maced. J. Med. Sci., vol. 6, no. 11, p. 1941, 2018.

Y. Forster, S. Hergeth, F. Naujoks, J. F. Krems, and A. Keinath, “Empirical validation of a checklist for heuristic evaluation of automated vehicle HMIs,” in International Conference on Applied Human Factors and Ergonomics, 2019, pp. 3–14.

R. Khajouei, M. Zahiri Esfahani, and Y. Jahani, “Comparison of heuristic and cognitive walkthrough usability evaluation methods for evaluating health information systems,” J. Am. Med. Informatics Assoc., vol. 24, no. e1, pp. e55–e60, 2017.

M. D. C. Tongco, “Purposive sampling as a tool for informant selection.,” Ethnobot. Res. Appl., vol. 5, pp. 147–158, 2007.

Article Metrics

Abstract view(s): 686 time(s)
PDF: 644 time(s)


  • There are currently no refbacks.