Measuring Usability on User-Centered Mobile Web Application: Case Study on Financial Mathematics Calculator

-Financial literacy is a person's skills regarding financial knowledge and behavior. In 2019, the National Survey of Financial Literacy and Inclusion (SNLIK) stated that Indonesia had a low level of financial literacy with a percentage of 38.03%. In 2016, the Indonesian government incorporated financial literacy into the K13 curriculum for the high school level to improve financial literacy at the student level. In line with that, students need facilities to help them learn financial literacy. This study aims to build an interaction design of financial mathematics calculators using the User-Centered Design method. The limitation of this study is the design uses a mobile web platform that targets high school students. The results of usability testing on effectiveness get an average task-completion rate score of 92.3%, an average efficiency of 92.0% overall relative efficiency, user satisfaction using the System Usability Scale (SUS) gets a score of 93.2 for the usability factor, and a score of 74 for the learnability factor. The findings of this study indicate that the application is easy to use and provides a positive experience in learning financial literacy. The current study's results will support this application's use in future research as a new tool for providing financial literacy.


Introduction
Financial literacy is a fundamental need for everyone to avoid financial problems. Financial problems can arise if something goes wrong in managing finances (mismanagement) [1]. In 2019, the Financial Services Authority (OJK), through the National Survey of Financial Literacy and Inclusion (SNLIK), found that Indonesia has a low level of financial literacy with a percentage of 38.03% [2]. The value means that the Indonesians do not understand the characteristics of financial products and services [2]. The President of the Republic of Indonesia has established the National Strategy for Indonesian Financial Literacy (SNLKI) to increase public trust [3]. In 2016, the Indonesian government included financial literacy in the K13 curriculum for the Senior High School level to improve financial literacy at the student level. In line with this, students need learning media to help the learning process of financial literacy. One of the tools that students can use to learn financial literacy is a financial math calculator.
One of the crucial factors in the development of a website is usability. Usability is one of the quality attributes that measure the ease of use of an application [4] [5]. Users will leave a website if the website is hard to use, the homepage fails to display information, the user is misguided on the website, or when the website information is difficult to read or does not answer the user's essential questions [4] [6]. Usability testing is one way to ensure a high level of usability of a system. According to the usability.gov website, usability testing evaluates a product or service by testing it against representative users [7]. Usability testing provides benefits such as: learning whether participants completed the specified task, identifying the completion time of specific tasks, finding out how satisfied users are with the website, and identifying changes needed to improve performance and user satisfaction [7][8] [9].
Currently, several financial math calculator applications have been developed as learning aids, especially in Indonesia [10]- [12]. Waritasari and Rofiki's research [10] found that applying a financial math calculator can improve critical thinking skills at the highereducation student level. Purnama et al. [11] and Khomariyah et al. [12] provide retirement fund planning features. The financial math calculator application has increased financial literacy for the general public and students. However, the application material has yet to follow the 2013 curriculum and the needs of high school students. This gap creates a void in understanding students' needs in financial literacy material, especially at the high school level.
Mat-Keu is a mobile web-based financial math calculator used as a student learning aid. The Mat-Keu Calculator is a media tool for learning financial mathematics used by class X students of SMKN 1 Kota Besi since two years ago. However, even though it has been used for two years, research on this calculator has never been done.
This study aims to redesign the interaction design of the Mat-Keu application to match the characteristics of high school students. This interaction design produces a high-fidelity prototype. Then we use usability testing to evaluate the interaction design. The research questions that guide this study are how users perceive (1) the effectiveness, efficiency, and user satisfaction of mobile web-based financial math calculators; and (2) possible improvements that shape strategies to improve usability for better users. The investigation results indicate whether the app needs to be improved or redesigned before deployment.
The method used to redesign this Mat-Keu calculator app is User-Centered Design (UCD). UCD is a systematic design process where developers focus on the user's needs at each phase of the design process [13]. In this method, the designer engages the user during the design process through various research and design techniques to create a product that is extremely useful and accessible to the user [13] [14]. UCD may not be sequential but still focuses on user understanding and context in all stages of development design.
The UCD method in this study was more efficient in understanding user needs. This is due to the clear target users of the Mat-Keu application (i.e., teachers and students). The development of learning media using the UCD method can provide good design results in terms of usability and user experience [15], [16]. Using the UCD method in developing learning media apps can lead to good usability and user experience with the condition that material experts and students must be involved in the development process to ensure that the product developed follows the needs of potential users [17].
The results of this interaction design are then tested using usability testing. Usability testing is one technique that ensures good user-centered design [9]. When it comes to the UCD method, it is closely related to the value of usability that ensures its users feel happy and will return to using the system. Based on ISO 9241-11 standard [18] [19], usability goals and attributes measured in usability testing are effectiveness, efficiency, and satisfaction.

Methods
The participants in this study were 13 people consisting of 12 students in grades 10 and 11 with majors in Financial and Institutional Accounting and Computer and Network Engineering, 15-18 years old, and living in Kota Besi District, East Kotawaringin Regency, Central Kalimantan, Indonesia. And a 45year-old math teacher in SMKN 1 Kota Besi. These thirteen participants were active and took part from the beginning of this study which started from designing the user persona until the final testing stage. Participants in this study were chosen deliberately with the help of a math teacher in SMKN 1 Kota Besi. The sample participants in this study were selected using the purposive sampling method. Purposive sampling selects samples based on unique characteristics tailored to the research aims and objectives to increase the accuracy of the research and the trustworthiness of the data and results [20]. Participation was voluntary.
The development of the Mat-Keu interaction design was carried out using the UCD method. The flow of this research is divided into four main stages: understanding and determining the context of use, determining user requirements, generating design solutions, and evaluating designs [21] [22]. The implementation of the four stages describes in the following paragraphs.
The first stage is to collect data from potential users, summarize it into a user persona and then analyze the basic needs of potential users of the Mat-Keu application. At this stage, potential users are observed and given an initial survey. Observations were made by asking participants to use the previous Mat-Keu v.1 calculator. Then prospective users fill out the SUS questionnaire and provide feedback on their experience using Mat-Keu v.1. The survey was conducted through a Google form online questionnaire. The survey results showed that the Mat-Keu v.1 calculator got a usability score of 58.3 and a learnability score of 48.95. This score is below the minimum eligibility value of SUS, which is 68 [23]. In addition, participants stated that the Mat-Keu calculator does not have a menu feature for switching calculator functions. The appearance design is less attractive and has not provided information about financial mathematics concepts. The results of this observation are used as an initial hypothesis to improve the features and appearance of the Mat-Keu calculator application that students and teachers previously used at SMKN 1 Kota Besi. After getting the results of the initial survey, then a follow-up survey was conducted to understand the desires and habits of students in teaching and learning activities. This survey involved 12 participants who were high school students. Teachers did not take part in this follow-up survey.
Based on the observations, this study designs user personas to represent users and analyze user requirements. User personas aim to make it easier for developers to briefly represent system users based on several variables. In this study, the user persona category is divided into high and low user personas. Categorization is based on the prospective user's understanding of financial mathematics material, active use of digital devices, and habits when accessing the browser. Table 1 is the user persona in this study. Have smartphone only, but one of these users also have a desktop. Some of this user active using internet 2-8 hours a day but some of them using internet less than 2 hours a day. This targeted user using google chrome as their primary browser.
It has multi-device smartphones, desktops (PCs), and tablets. Most users actively use the internet for 2-8 hours, and some actively use the internet throughout the day. These targeted users are actively using google chrome as their primary browser.

Habit
The main concern when accessing a website is the content and design, if faced with confusing feature some of this user chooses to move to another web that has similar information but some users choose to learn each function and feature if needed. This targeted user prefer to collect information from more than one website page to add references.
The main concern when accessing a website is the content and design; if faced with a confusing feature, some users choose to move to another web with similar information and prefer to collect information from more than one website page to add references.

Goals
Learning, discipline, and take responsibility. User needs a website that has a good look and good to be used in multi-devices. This targeted user also like a website that fast, easy to use, and have to the point information.
Learning and take responsibility. Users likes a website that has a good appearance and suitable to be used both on desktop or on smartphone. Most of this targeted user like a fast and an easy to use website but, some of them concern on the main content also. frustration Does not like websites with a messy and confusing look.
Doesn't like websites with a messy and confusing look Objective Low user persona High user persona Users have problem in annuities and interest topic. and some users have problem in annuities and interest topic.
User requirements are analyzed based on observations and user personas to determine design solutions. Table 2 shows the user's need to use this study's financial math calculator. Conceptual models are needed to facilitate the interaction design process. The conceptual model is a subjective interpretation created as a starting point in further development [24]. The conceptual model helps provide an overview of the User interface (UI) elements needed in the design solution design process later. Table 3 is the conceptual model in this study.
The second stage is to make a design solution based on the requirements formulated in the first stage. Based on the conceptual model mapped according to user needs, the interface design ideas for each feature were designed using low-fidelity and high-fidelity prototypes In the low-fidelity prototype design, the number of text inputs adjusts according to the number of variables contained in the mathematical financial formula. This causes the input lines on the calculator menu to be different; some have three input lines and four input lines. Figure 1 is the result of the low-fidelity prototype in this study.

Figure 1. Low-fidelity prototype
After having a low-fidelity prototype, the next stage is creating the interactive design (high-fidelity prototype). Figma is a tool used to make prototypes. Interface (UI) design focuses on mobile devices to support the user persona data, whereas all respondents use smartphones more often. The color palette used in this design prototype combines blue and white. According to psychological science, blue can stimulate clear thinking, increase concentration, and help calm the mind.
In contrast, the white color as the background in this design was chosen to give a minimalist, clean impression and can provide contrasting colors [25]. This color selection was supported by the statement of respondents from the survey who mostly like the blue color. Figure 2 illustrates the appearance of a prototype design created using Figma.

Figure 2. High-fidelity prototype
After making a high-fidelity prototype, this research adds expert validation in the second stage. We use expert validation to validate the formula or material used in the application. This validation prevents errors in implementing mathematical formulas into the application. The validation process involved two math teachers at SMKN 1 Kota Besi. Both validators were asked to try every function and feature of the Mat-Keu calculator application. Table 4 shows expert validation for each feature in the solution design. Based on the results of expert validation, it can be concluded that all formulas and materials used in the Mat-Keu prototype are suitable for use and further development. The third stage is the evaluation process by doing usability testing. After completing the high-fidelity prototype, we perform usability testing to evaluate the UI designs. Usability testing is based on ISO 9241-11, which measures three usability metrics: effectiveness, efficiency, and satisfaction. The test data were obtained and analyzed from screen recording when participants conducted prototype testing and the System Usability Scale (SUS) questionnaire via Google forms. In some conditions, there is an opportunity to interview the participants who seem to be experiencing difficulties.
The three usability metrics of the ISO 9241-11 standard describes as follows.

Effectiveness
One way to measure effectiveness is to calculate the level of completion when the user successfully achieves the goal in the task scenario [19]. Refers to (1), the completion rate is a percentage of the total number of tasks completed with the total number of tasks taken by the users [18], [19].
Sauro [26] states that the average task completion rate of 1200 usability tasks are 78%. This average completion rate was used as a reference to determine whether the design made meets the effectiveness standards or not.

Efficiency
Efficiency is how much effort or time takes a user to complete a task [18]. According to Arthana et al. [18], efficiency can be measured using Overall Relative Efficiency (ORE). ORE calculates the ratio of time it takes for a user to complete a task concerning the total time taken by all users [18], [27], [28]. The formula used to calculate the percentage of efficiency using the ORE method describes in (2) [27], [28].
Where N is the total number of scenarios (goals), R is the number of users, nij is the result of coming through scenario i by user j, tij is the time spent by user j to come through scenario i.

Satisfaction
Satisfaction refers to comfort, pleasure, acceptance, and how positive user feedback is towards a product [18], [19]. There are considerable ways to measure satisfaction levels, such as System Usability Scale (SUS), Computer System Usability Questionnaire (CSUQ), Questionnaire for User Interaction Satisfaction (QUIS), Software Usability Measurement Inventory (SUMI), SUPR, etc. [18], [19]. This study's evaluation stage of satisfaction level measurement uses a System Usability Scale (SUS) questionnaire. SUS has advantages in terms of its being easy to use, cheap, and reliable with a small sample size [29], [30].
The SUS questionnaire consists of 10 items with five positive items (odd-numbered items) and five negative items (evennumbered items) [31]. The questionnaire used a Likert scale with five choices of answers from "Strongly disagree" to "Strongly Agreed." SUS has two factors: usable (8 items) and learnable (2 items) [31]. Table 5 shows the ten items owned by SUS. Lewis and Sauro [31] describe how to calculate SUS scores to extract usable and learnable scores.
1. Assume that X is the raw score of odd questions then X = X -1 2. Assume that Y is the raw score of even question then Y = 5 -Y 3. Usable score = sum all X and Y from question number (1,2,3,5,6,7,8,9) then multiple it with 3.125 4. Learnable score = sum X from question number (4 and 10) then multiple it with 12.5 The above calculation results will give each respondent a SUS score with a range of 0-100. An adjective rating scale analysis [32] can interpret an individual's SUS score and help explain the user's perspective on the application. The adjective rating scale consists of 5 criteria as follows [32]: Excellent (score > 80), Good (score between 69 to 80), Okay (score = 68), Poor (score between 51 to 67), and Awful (score < 51).
The last stage is to implement the design prototype into a ready-to-use mobile web. At this stage, the prototype is implemented using the next.JS framework. The use of next.JS in developing this application aims to make the work process behind the scenes lighter so that it does not burden the system work on the user's device.

Result and Discussion
The average task completion rate is used to measure effectiveness. Table 6 shows the participants' average task completion rate when testing each feature of the Mat-Keu prototype. Table 6 also shows failures in completing task scenarios. Failure to complete tasks in scenario one was caused by a missed click due to a lack of accuracy when selecting the calculator menu feature, so the prototype stopped functioning. Users are still not familiar with the use of prototypes. The bugs found fixed the prototype, so missed clicks did not cause the prototype to stop working. In task scenarios 2, 3, and 4, the failure to complete is caused by the user not understanding the questions, so assistance needs to be given. In task scenario 5, the user understood the question and no longer found the missed click. Based on table 6, the average task completion rate is 92.3%. This indicates that, on average, the participants have achieved the objectives of each given scenario. Therefore, the quality of the effectiveness of the highfidelity prototype is excellent. We use Overall Relative Efficiency (ORE) to measure efficiency in this study. Figure 3 displays the results of efficiency calculations using the ORE method from each test scenario in the Mat-Keu calculator prototype design. Based on the ORE results in figure 3, the average percentage efficiency of the Mat-Keu prototype is 92%. On average, participants consumed more time on task scenario 1 (compound interest). This is because, in the scenario one task, participants are still learning how to use the prototype. Participants feel they need clarification when some features or components cannot be accessed (clicked), the value that appears does not match the question (because of using the wrong feature), and guesses how to use the prototype. In task scenarios 2 to 5, participants have begun to understand how to use the prototype and its features, so the graph of the total time needed to complete the task scenario also increases. To measure user satisfaction, we use A system usability scale (SUS) with two factors: usable and learnable. Figure 4 shows the satisfaction measurement results using SUS on the high-fidelity prototype. The average SUS score of 13 respondents for usable factors is 93.27 out of 100, which is in the "Excellent" category [32]. These results indicate that participants are satisfied with the Mat-Keu prototype because of its ease of use in solving financial math problems. The average SUS score of 13 respondents for the learnable factor is 74.04 out of 100, which is in the "Good" category [32]. These results indicate that participants are satisfied with the Mat-Keu prototype because of the ease of learning the features provided, although some still need help.

Conclusion
This study aims to redesign the interaction design of the Mat-Keu application to match the characteristics of high school students using the User-Centered Design (UCD) method. By approaching the user's needs, the goals in the development of Mat-Keu were identified: including providing a new interface that is simple and easy to use, provides menu features, explaining how to spell mathematics, and providing a mathematical calculator function that can answer problems according to the national curriculum.
The presence of this application can be a tool for students to understand financial mathematics concepts according to the curriculum. The results of usability testing on effectiveness get an average task-completion rate score of 92.3%, an average efficiency of 92.0% overall relative efficiency, user satisfaction using the System Usability Scale (SUS) gets a score of 93.2 for the usability factor, and a score of 74 for the learnability factor. The findings of this study show that the Mat-Keu calculator application is easy to use and provides a positive experience in finding information related to financial literacy according to the curriculum at the high school level. This study's results will support this application's use in future research as a new tool for providing financial literacy. Further research can add control group gradations and experiments so that the impact of using the Mat-Keu calculator application can be known on the learning process and financial literacy.