doi: 10.56294/saludcyt2024.1000

ORIGINAL

Design and evaluation a mobile augmented reality to enhance critical thinking skills for vocational high schools

Diseño y evaluación de una realidad aumentada móvil para mejorar las habilidades de pensamiento crítico en escuelas secundarias vocacionales

Fatma Sukmawati¹ , Suparmi¹ , Relly Prihatin¹ , Eka Budhi Santosa¹

¹Faculty of Teacher Training and Education, Universitas Sebelas Maret, Surakarta, Indonesia.

Citar como: Sukmawati F, Suparmi, Prihatin R, Santosa EB. Design and evaluation a mobile augmented reality to enhance critical thinking skills for vocational high schools. Salud, Ciencia y Tecnología. 2024; 4:.1000. https://doi.org/10.56294/saludcyt2024.1000

Submitted: 10-02-2024                   Revised: 18-05-2024                   Aceptted: 01-10-2024                 Published: 02-10-2024

Editor: Dr. William Castillo-González

ABSTRACT

This study aimed to examine the impact of mobile augmented reality (MAR) implementations on critical thinking in vocational high school students. This study employed quantitative methodology, particularly quasi-experiment with a pre-experiment method. 120 vocational high school students, who have enrolled the accounting lesson in Class X, were examined to obtain answers to the two research questions. The findings indicate that students in the experimental class, who utilized the AR in their learning process, had significant enhancements in their critical thinking skills compared with the control class. This improvement illustrates that the implementation of AR has potential to promote the critical thinking skills of the students. To better understand these results, this study also examines the enhancement of critical thinking skills based on the six indicators of critical thinking skills—interpretation, analysis, conclusions, evaluations, explaining, and self-regulations. By comparing each indicator of the critical thinking skills, there are differences between both the experimental and the control class. The experimental class showed enhancement in all indicators but with significant improvement in conclusions, evaluations, and explaining. More crucially, through the learning process, implementation of AR technologies also instills the aspect of developing critical thinking skills.

Keywords: Mobile Augmented Reality; Critical Thinking; Vocational High School.

RESUMEN

Este estudio tuvo como objetivo examinar el impacto de la implementación de la realidad aumentada móvil (MAR) en el desarrollo de las habilidades de pensamiento crítico de los estudiantes de educación secundaria técnica. Este estudio utilizó una metodología cuantitativa, en particular un diseño cuasi-experimental con un método de pre-experimento. Se examinaron a 120 estudiantes de secundaria técnica que habían tomado la asignatura de contabilidad en el Grado X, para obtener respuestas a dos preguntas de investigación. Los hallazgos indican que los estudiantes en la clase experimental, quienes utilizaron la realidad aumentada en su proceso de aprendizaje, mostraron mejoras significativas en sus habilidades de pensamiento crítico en comparación con la clase de control. Esta mejora ilustra que la implementación de la realidad aumentada tiene el potencial de promover las habilidades de pensamiento crítico en los estudiantes. Para comprender mejor estos resultados, este estudio también examina el desarrollo de las habilidades de pensamiento crítico basándose en los seis indicadores de estas habilidades: interpretación, análisis, conclusiones, evaluaciones, explicación y autorregulación. Al comparar cada uno de los indicadores de las habilidades de pensamiento crítico, se observan diferencias entre la clase experimental y la clase de control. La clase experimental mostró mejoras en todos los indicadores, pero con un progreso significativo en conclusiones, evaluaciones y explicación. Más importante aún, a través del proceso de aprendizaje, la implementación de tecnologías de realidad aumentada también fomenta el desarrollo de las habilidades de pensamiento crítico.

Palabras clave: Realidad Aumentada Móvil; Pensamiento Crítico; Escuela Secundaria Técnica.

INTRODUCTION

Vocational education is one type of educational process that prepares learners by obtaining definite skills, knowledge, and competencies for a specific occupation. In this regard, the learning process in vocational high schools has to face several problems: outdated curricula; low interest of students; and inappropriate competences of teachers.^(1,2) This problem is further exacerbated by the weak support the teachers receive in improving their competency in teaching students effectively. Another problem that exists in vocational learning is the conventional teaching method, such as lectures. The students are primarily presented with explanations of the learning materials without its relevance to real-world problems.^(3,4) It might be applied widely but often do not cater adequately with the complexity of the material, specifically, making students disengage and to be unmotivated. This contributes to the high unemployment rate among vocational school graduates.⁽⁵⁾

Hence, there is an urgent need to incorporate more active and student-centered learning that can help students better understand, and as well, improve their critical thinking skills when addressing complicated materials.⁽⁶⁾ Vocational schools also prioritize the development of practical skills and theories compared to soft skills such as critical thinking. The development of critical thinking is often difficult to integrate into technical and vocational education and training because the main focus is on mastering practical skills and theories that can be directly applied in the workplace. The vocational school system has not succeeded in producing students that are able to think critically and gives them opportunity to explore their imaginative ideas in the learning process.⁽⁷⁾ Their thinking abilities remain at a low level because teachers often spoon-fed them with materials but seldom taught them how to learn. They struggle to integrate the skills, knowledge and also attitude as it needs complex cognitive skills. Consequently, the students face difficulties in solving real life problems, making decisions, and thinking critically and creatively.

Meanwhile, to produce high-quality graduates from vocational high schools, it is essential to have a common understanding between the competencies that must be achieved by the students and the needs of the industry in the country.⁽⁸⁾ Along with the economic and industrial revolution, there are 6 main competencies that students must have in the 21st century, collectively known as the 6Cs of 21st century education. They are critical thinking, communication, collaboration, creativity, character and culture/citizenship.⁽⁹⁾ Therefore, it is crucial not only to master practical skills but also to instill soft skills such as critical thinking abilities in vocational high school students. This approach will enable students to be able to solve real-world problems effectively. Incorporating critical thinking into vocational education leads to sharper ability of students for logical reasoning and applying creativity when solving tasks in any professional conditions.⁽¹⁰⁾ Students can gain critical thinking skills from activities such as involving them in problem solving-based activities encouraging them to carry out lots of exploration and questioning activities so that they are ready to face the complex and complicated world of modern work. Hence, we cannot neglect the importance of critical thinking skills in a vocational high school setting. Critical thinking skills can be categorized under six indicators.⁽¹¹⁾ The indicators include interpretation, analysis, evaluation, inference, explanation, and self-regulation. This method gives a comprehensive account of the complex process of thinking critically, where logic, evidence judging, and self-awareness are seen as key main features.

Research indicates that the infusion of technology in vocational high schools setting develops students’ learning outcomes noticeably. For example, it can enhance students’ academic performance by making them enter more innovative and interactive environments and improving engagement and motivation for students studying in vocational high schools.⁽¹²⁾ It creates an interactive learning condition, dealing with challenges of inadequate resources and passive learning. Additionally, the use of technology in vocational learning plays a significant role in the student’s development of technical and social skills.⁽¹³⁾ Furthermore, advancement in technology ensures that higher order thinking skills, such as critical thinking, are developed in the process of learning if the right technological tools are employed.⁽¹⁴⁾ In general, technology in vocational high schools has learning benefits and helps prepare the students for the tasks in the contemporary job market through the enhancement of skills and proficiency.

Mobile augmented reality (MAR) combines computer-generated virtual elements with the actual world via technology, resulting in an immersive and interactive learning environment.⁽¹⁵⁾ MAR is used to apply immersive learning for educational purposes in vocational high schools. There is evidence that the utilization of mobile augmented reality can increase student motivation and better understanding of the subject matter. The studies carried out in vocational high schools are found to be effective with augmented reality. Academic research has underlined effectiveness with augmented reality in career education, upgrading the quality of practical learning experiences, and enhancing student achievement in vocational high schools.⁽¹⁶⁾ Teachers have incorporated augmented reality technology to create realistic learning settings and trigger the interest and increased intellectual potential of students.⁽¹⁷⁾ The technology applies interactive and attractive learning activities to connect theoretical knowledge with the reality of life.^(18,19) By adopting Augmented Reality (AR) in vocational education, self-efficacy among the students and cognitive load can be increased, thus further improving the learning process effectiveness of students.^(20,21) MAR provides a more profound educational experience for students’ skills to be applied in the real world, therefore transforming vocational education. AR-based learning increases student engagement because of more interactive activities that allow students to learn with instructional resources actively.⁽²²⁾

MAR provides the opportunity for vocational schools’ students to receive contextualized and direct learning. During the course of an automotive engineering class, learners would view a three-dimensional model of an automobile engine, guided through the working process, create discussions, and deliver an assignment with help from augmented reality’s interactive simulations.⁽²³⁾ The underlined effectiveness of this learning has both theoretical and practical significance that just can’t be provided by traditional teaching techniques. One of the benefits of augmented reality is that it can visualize complicated topics.⁽²⁴⁾ In marketing, for instance, students can use augmented reality to visualize effective marketing strategies in simulated market environments. This enhances the understanding and application of theory among students. MAR could also encourage teacher-student cooperation in vocational institutions. Through AR applications, vocational school instructors are able to give more vivid and efficient lectures while students are allowed to make joint problem-solving efforts.⁽²⁵⁾

The nature of the studies in vocational schools generally demands personal and shared practical teamwork skills. And according to further studies, AR raises students’ interests in what they are being taught. In one vocational school research, the students who were studying the accounting topic through augmented reality showed significant levels of motivation, compared to the traditional way of learning. Academic results are, in fact, linked with such a level of motivation.⁽²⁶⁾ There are different advantages that mobile Augmented reality (MAR) presents when implemented in the context of vocational high school. One of the primary strengths of AR is in increasing the engagement of students in the learning process. The use of AR motivates and engages learners during the learning process, delivering engaging and dynamic graphics.⁽²⁷⁾ This technology enables interaction with virtual elements, which smoothens blends into the real world, making learning interesting and pleasurable. Besides that, AR enables not only to see theories and abstract topics more clearly but in a more concrete way. For example, in mechanical engineering, it is possible to design 3D models of the machinery used, and students will be able to manipulate them better for understanding the material.

Better attentiveness is not just the only advantage of MAR for information retention and understanding. Since it’s displayed using an interactive visual instrument, students can hold onto learning and also make real-life applications of learned concepts.⁽²⁸⁾ Additionally, MAR encourages social interaction because students can work together on AR-related activities; these interactions will allow them to vividly express their gained knowledge.⁽²⁹⁾ AR makes the learning of the students more enhanced, better, and memorable toward better practical skills of what they study theoretically.

Learning using MAR makes knowledge interactive and contextual for students. That is, students of vocational high schools can use augmented reality to understand natural phenomena that otherwise will be hard or impossible for them to experience by themselves, such as structures on the molecular level and biological processes.⁽³⁰⁾ This makes it possible for the learners to obtain a more detailed understanding of content and to critically appraise knowledge.

There have been a number of MAR-based learning media applications that could encourage critical thinking in various learning settings.^(17,25) It has been used with primary school students to improve critical thinking.^(18,31) Being realized by interactive and contextual learning, respectively, to stimulate the skills of critical thinking and solving problems.

Furthermore, its use has been conducted in several fields, such as mathematics and science.^(21,32) On the other hand, AR combined with the right learning model, such as guided inquiry learning, can help increase critical thinking among students by enhancing their ability to engage in higher mental processes and, therefore, develop their own critical thinking skills more effectively.^(23,33) An argumentation exercise again incorporated with AR can help students do critical-thinking tasks like decision-making, clarification, and inference.^(24,34)

Research indicates that learning with the help of MAR can boost students’ interest and involvement in the learning process, assisting them to develop their critical thinking ability. An increase in motivation was reported by those pupils who learned through the augmented reality system, while they felt much more intellectually challenged than those who learned via traditional techniques. Such enthusiasm on the part of a pupil enables him/her to participate in the learning process quite actively and improve his/her critical thinking skills. MAR can also help students enhance their skills to evaluate and synthesize information coming from different sources. The students’ diversification of information formats, such as text, graphics, animation, among others is brought into touch with MAR-based learning. The inclusion and assessment of information are basic premises of critical thinking.⁽²⁸⁾ Thus, augmented reality can equip students with the set of skills that will allow them to easily navigate in a complex, information-rich environment.

In summary, the research detailed above portrays the positive effect of MAR implementation toward students’ critical thinking in vocational education. Despite the several studies that have explored this matter, further accounting study is still needed to expand current literature that is still limited. So, there is a need for further inquiry regarding these six markers of critical thinking ability to elucidate how augmented reality affects each of the dimensions of critical thinking skills.

The purpose of this study is to see how the use of Mobile Augmented Reality (MAR) technology affects students’ critical thinking abilities in vocational high schools. Based on Facione’s critical thinking indicators, this study will compare the critical thinking abilities of vocational high school students in experimental classes using MAR versus control classes that do not utilize MAR media. After evaluating the hypothesis, N-Gain assessments were performed to determine the changes between before and after MAR treatment in each indicator of critical thinking abilities, including interpretation, analysis, drawing conclusions, evaluation, explanation, and self-regulation. Interpretation refers to pupils’ capacity to grasp and categorize information acquired through MAR’s 3D visualization. Analysis relates to pupils’ ability to recognise relationships between concepts and ideas, which is aided by interactive simulations in MAR.

Furthermore, the ability to form inferences will be assessed using virtual experiments that allow students to derive conclusions from data provided in an augmented reality environment. Evaluation, or the capacity to assess the credibility of information sources and the validity of arguments, will be examined by presenting multiple sources of information in AR for students to analyze and evaluate. For example, interpretation concerns the ability to express ideas and arguments clearly, which will be operationalized in terms of the ability to use interactive 3D models in augmented reality for teaching technical topics.⁽³⁵⁾

Self-regulation, or the monitoring, assessment, and correction of one’s mental processes, will be operationalized in AR by providing rapid feedback so that students can recognize and correct errors on their own. The research will be conducted by using an experimental design approach, with a control group and an experimental group. Therefore, the t test will be employed to analyze significant differences in critical thinking skills between these groups. By this method, research is expected to provide thorough knowledge of the usefulness of MAR in developing students’ critical thinking abilities in vocational schools.

Research objective is to identify the level of the impact of the adoption of mobile augmented reality (MAR) on vocational high school students’ critical thinking skills. This work analyzes students’ critical thinking skills based on 6 indicators from Facione⁽³⁶⁾ namely interpretation, analysis, evaluation, inference, explanation, and self-regulation. The six components of critical thinking skills of vocational school students are measured following the implementation of a developed Mobile Augmented Reality (MAR) application. Therefore, the research questions for this study are:

· Is there a significant difference in the Critical Thinking Skills of students who learn using MAR (experimental group) compared to those who do not use MAR (control group)?

· How do the six indicators of critical thinking skills differ between the experimental group and the control group?

This study offers multifaceted contributions. Firstly, this research has developed and integrated advanced technology, a Mobile Augmented Reality (MAR) learning platform for vocational high school students, particularly those majoring in accounting. Secondly, this study analyzes the often-overlooked skill, critical thinking, in vocational high school settings in Indonesia, as the focus tends to be predominantly on practical skills. Thirdly, in researching critical thinking skills, we not only examine students who used the MAR platform but also conduct a comparative analysis with students who applied the conventional learning methods. The result of this study is beneficial for educators and policymakers in vocational education settings. As it aims to prepare students for specific careers, vocational high school settings must be able to adeptly integrate technology into their learning to equip students with skills pertinent in the current era.

METHOD

Research Design

This research utilizes a quantitative analysis method, specifically quasi-experiment with a pre-experiment method with a one-group pretest-posttest design as shown in table 1. This study consisted of two groups of participants:  the control group and the experimental groups. The experimental group was taught MAR-based accounting subjects for 4 weeks, while the control group was taught using the traditional teaching approach. Pretest and posttest will be administered for both groups. A pretest was used to compare the critical thinking skills of the control and experimental groups before the teaching intervention. After the intervention, a post-test was conducted on both groups to identify the impact of using MAR teaching on the development of students’ critical thinking skills.

Table 1. One Group Pretest Posttest Design
Pretest	Treatment	Postest
O1	X	O2

Information:

O₁: initial test (pretest), before administering treatment

X: treatment using MAR

O₂: final test (posttest) after administering treatment

Research Population

The research population for this study comprised vocational high school students enrolled in accounting majors at State Vocational School 1 Surakarta, Central Java. In this research study, 4 classes of first-year vocational high students were selected. With each class consisting of 30 students, a total of 120 students participated voluntarily. These classes were divided equally into two groups, using random sampling. Two classes were assigned as the experimental group and taught using MAR. The remaining two classes were assigned as the control group and taught using the traditional teaching approach. A pre-test with and post-test was conducted to compare the existing knowledge level of both the groups prior to the intervention.

Instrument

The data collection technique employed in this research is a test, which is an essay-type question with 15 questions that can measure students’ critical thinking skills in the cognitive domain. This study’s starting point is 6 indicators of critical thinking according to Facione’s.⁽³⁷⁾ Based on the academic’s work, Facione’s listed 6 indicators, including: 1) interpretation 2) analysis, 3) making conclusions, 4) making evaluations, 5) explaining, 6) self-regulation. Critical thinking skills indicators in the cognitive domain using interval scale This approach allows us to compare the level of improvement between the experimental and control groups. The t-test is employed to determine the significance of the difference between two paired averages.⁽³⁸⁾ In this study, the Independent Sample T Test analysis in the SPSS version 18 was utilized to support the hypothesis testing.

The variables in this study include the six critical thinking indicators adapted from Facione’s critical thinking framework serve as the dependent variables. Meanwhile, the N-Gain is not a variable, but a method of analysis used to measure the extent of improvement in critical thinking skills following the intervention. The N-gain test is used to quantify the extent of enhancement in critical thinking skills before and after treatment. The improvement in critical thinking skills is analyzed using the N-gain test equation by using the following formula:

Where:

Post-Test: score after MAR intervention

Pre-Test: score before MAR intervention

Max Score: maximum test score

The results of the analysis are then adjusted to the criteria for students’ critical thinking skills in accordance with table 2 below.

Table 2. N-gain values and category
N-gain (g)	Category
0,71≤ g≤ 1	High
0,31≤ g≤ 0,7	Medium
0 ≤ g≤ 0,3	Low

Validity and Reliability Test

The prerequisite tests for analysis in this research consist of the normality and homogeneity tests of the data. Both data normality and data homogeneity tests are essential procedures in statistics that allow researchers to validate fundamental assumptions before applying certain statistical analysis. The Shapiro-Wilk test is employed as the normality test, which is designed to measure whether the data sample is derived from a normal population distribution or not. If the data are not normally distributed, then parametric statistical analysis is not suitable and non-parametric alternatives should be examined. On the other hand, a homogeneity test is employed to determine whether the variability between groups of data is equal or not. This also refers to whether the variance between data groups is homogeneous or not. Above all, the Shapiro-Wilk test in this study aids in determining if the fundamental assumptions of a particular statistical method are met. Also, it ensures a more accurate and valid interpretation of analysis results. Therefore, the normally distributed data in this study have to meet the criteria in the hypothesis being tested, which are as follows:

Table 3. Data Normality Test Criteria
Test Data	Significant Value (p)	Criteria
Normality	p < 0,05	Not Normally distributed
	p > 0,05	Normally distributed
Homogeneity	p < 0,05	Variances are not equal (Not homogeneous)
	p > 0,05	Variances are equal (homogeneous)

RESULTS

The research findings, based on 120 students who enrolled in accounting lessons and participated in the research, illustrate a notable increase in the students’ critical thinking skills. The findings indicate the increasing indicators of students’ critical thinking skills by Facione’s, before and after learning using MAR. Information about a students’ starting point in terms of critical thinking skills is obtained from their test scores. Before hypothesis testing analysis is conducted, the data obtained have to meet the prerequisite tests for parametric testing including normality and homogeneity. The following presents a discussion of each prerequisite test result and research hypothesis examined.

Parametric Prerequisite Test Results

The results of this study refer to findings that have fulfilled all the prerequisites and criteria previously established for analysis. The prerequisite tests used ensure the reliability and validity of the data while addressing the significant variables that impact the outcomes. Prerequisite tests in this research include data normality and homogeneity tests. If the analysis prerequisite tests are fulfilled, then the data have to meet the normal and homogeneous categories. Table 4 below presents the results of the analysis prerequisite tests for the experimental and control classes from this study.

Table 4. Data Normality Test dan Homogeneity Test
Class	Sig. Value	Normality Description	Sig. value	Homogenity Description
Eksperimental	0,356	Normally Distributed	0,625	Homogeneously Distributed
Control	0,274		0,674

The table 4 indicates that the normality test result for the experimental class is 0,356, while for the control class it is 0,274. Based on the normality test data in the experimental and control classes, it is evident that the results of the normality test in these classes have a significant value higher than the minimum required (p>0,05). Therefore, it can be concluded that the result data in both the experimental and control classes derived from a normally distributed data population. Moreover, in order to assess the data homogeneity, the same criteria are used as those employed for the normality test. If the significance value is higher than 0,05 (p>0,05), then the data obtained fulfills the homogeneous criteria. Based on table 3, the significance values in the experimental and control classes are 0,625 and 0,674, respectively. Since these scores are higher than 0,05 (p>0,05), the data obtained from the pretest and posttest results in both classes are homogeneously distributed.

Hypothesis testing

Hypothesis testing is a statistical technique used to verify claims or assumptions made regarding the characteristics of a population by using sample data. In this study, the following hypotheses are tested; H₀: there is no significant difference in critical thinking skills between students who learn using MAR and those who learn using traditional methods; H1: there is a significant difference in critical thinking skills between students who learn using MAR and those who learn using traditional methods. The results of hypothesis testing are examined as follows:

Table 5. Hypothesis Test Results
Classes	Hypothesis Test (T)
	Sig. (2 tailed)	Description
Eksperimental	0,00	H1 is Accepted

The table 5 presents the results of the hypothesis test for the experimental class. Based on the hypothesis test (T-Test) with a Sig. (2-tailed) value of 0,00, it can be concluded that the alternative hypothesis (H1) is accepted. This indicates that there is a significant difference between the experimental group, which utilized Mobile Augmented Reality (MAR)-based learning, and the control group that employed traditional teaching methods. A significance value less than 0,05 (in this case, 0,00) supports the acceptance of the alternative hypothesis, meaning that the use of MAR has a significant impact on improving students’ critical thinking skills.

The discussion of the critical thinking skills in this study is conducted based on Facione’s 6 indicators of critical thinking. These indicators include:

1) interpretation

2) analysis

3) making conclusions

4) making evaluations

5) explaining

6) self-regulation

The research results show the percentage of each indicator in the pretest, as presented in the following figure:

Figure 1. Bar Diagram of Percentage of Critical Thinking Skills Indicators in the Pretest

According to figure 1, the pretest results indicate that the highest indicator percentages are interpretation, explaining, and conclusion with 75 % and 70 % respectively. These indicators are followed by conclusion and evaluations indicators that have the same percentage at 65 %. Then, the lowest indicators are analysis and self-regulation indicators which have 60 % and 50 % respectively.

Figure 2. Bar diagram of posttest critical thinking skills indicators

Furthermore, figure 2 illustrates the percentage of achievement for each critical thinking skills indicator (post-test). These include the interpretation indicator at 80 %, the analysis indicator at 85 %, the conclusion making indicator at 86 %, the evaluation indicator at 87 %, the explanation indicator at 90 %, and the self-regulation indicator at 90 %.

The critical thinking skills data from both pretest and posttest shown at figure 3 are collected by using an essay-type question. These questions were administered on participants which were followed by data analysis. The pretest was given first to the participants before the learning process began. This test has the purpose of assessing the level of students’ critical thinking skills in answering critical thinking questions related to teaching material. Once the pretest has been administered, the participants in the experimental class are given a treatment using MAR. When the treatment has been completed, students are then given posttest that are identical to the pretest question. The table 6 below presents the pretest and posttest results, including the lowest score (X-min), the highest score (X-max), the average score (X-average) and the standard deviation (s).

Table 6. Pretest and Posttest Score Description
Description	Pretest	Posttest
Lowest Score	50	60
Highest Score	70	90
Average Score	64	86
Standard Deviation	1,999	1,476
The number of Student	120	120

Meanwhile, the difference in pretest and posttest scores, which reflect the students’ critical thinking skills, is visually represented in the following figure.

Figure 3. Line Graph of Pretest and Posttest Critical Thinking Skills Scores

The results of the N-Gain analysis reveal a notable enhancement in students’ critical thinking abilities. As depicted in figure 3, the minimum pre-test score was 50, which rose to 70 in the post-test. Similarly, the maximum score increased from 70 to 90, while the average score improved from 64 to 86. These findings clearly indicate that students’ critical thinking skills improved following the integration of MAR. The observed increase in scores strongly suggests that the application of MAR positively influenced the students’ learning outcomes. 

Table 7. N-gain Test Results
N	Posttest Average	Pretest Average	N-gain	Category
120	86	64	0.753	High

According to table 7, the participants’ N-gain score category is 0,753 which lies within the High group. Therefore, the implementation of MAR has the capacity to enhance the critical thinking skills of highly proficient students. The analysis results and impact of MAR-assisted learning on enhancing each indicator of critical thinking skills based on the N-gain score criteria are detailed in table 8.

Table 8. N-Gain Results for Each Indicator of Critical Thinking Skills for the Experimental Class and the Control Class
Experimental Class				Control Class
No	Critical Thinking Indicators	Score	Category	Critical Thinking Indicators	Score	Category
1	Interpretation	0,78	High	Interpretation	0,60	Medium
2	Analysis	0,65	Medium	Analysis	0,55	Medium
3	Conclusions	0,80	High	Conclusions	0,70	High
4	Evaluations	0,83	High	Evaluations	0,73	High
5	Explaining	0,80	High	Explaining	0,65	Medium
6	Self-Regulations	0,73	High	Self-Regulations	0,60	Medium
Average		0,76	High	Averages	0,63	Medium

According to table 8, the N-Gain test results indicate that the average student’s critical thinking skills have increased in each indicator. The N-Gain test results in the experimental class were higher than the control class. The experimental class achieved a score of 0,76 (High), whereas the control class only achieved a score of 0,63 (Medium). The difference in treatment between the two classes is the underlying cause for the variation in results shown in the N-Gain test. While the experimental class employs the use of MAR in its learning, the control class solely focused on conventional learning methods.

The independent sample t-test results reveal a notable difference in critical thinking skills between students in the experimental group, who utilized MAR, and those in the control group. With a significance level (2-tailed) of 0,00, which is below the threshold of 0,05, the alternative hypothesis (H₁) is supported. This suggests that the application of MAR has a statistically significant and positive influence on students’ critical thinking abilities when compared to conventional teaching methods. Thus, this finding addresses the first research question by confirming the substantial improvement observed in the experimental group.

The N-gain analysis indicates greater improvement across all six critical thinking indicators in the experimental group compared to the control group. For instance, the conclusion indicator in the experimental group achieved a high improvement category (N-gain = 0,80), whereas the control group attained a medium improvement category (N-gain = 0,70). Similarly, the self-regulation indicator in the experimental group recorded an N-gain of 0,73 (high), while the control group displayed an N-gain of 0,60 (medium). These findings address the second research question, highlighting that the experimental group experienced more significant gains in all six critical thinking indicators, particularly in conclusion, evaluation, and explanation.

In conclusion, the results demonstrate that employing MAR notably enhances students’ critical thinking abilities across all six indicators, with the most significant improvements seen in conclusion, evaluation, and explanation skills. These findings effectively address both research questions and further validate the positive impact of MAR in fostering critical thinking among vocational high school students.

DISCUSSION

The impact on every indicator of critical thinking skill can be deeply examined in order to discuss these results more comprehensively. This study identifies six indicators of critical thinking skills based on Facione’s academic works.⁽³⁶⁾ The first is Interpretation. Accounting lessons that are using AR helps students in understanding information through interactive visualization. For this reason, AR enables students to interpret financial transactions and reports by giving a simpler approach and thus enhancing students’ comprehension and evaluation of financial scenarios. This statement is in line with research the learners find it easier to remember or learn complex visualized-based materials given with MAR.⁽³⁹⁾

The next indicator is analyzing, which pertains to analyzing. Students can virtually simulate many accounting scenarios by using AR. It also offers them a chance to explore in a visually immersive and contextual manner how different financial moves may influence their lives. This captivity is useful in building up analytical abilities that might be of use when assessing financial options and alternatives of varying kinds. Combined with appropriate teaching strategies, AR could enhance such cognitive processes as decision making, clarification and inferences-making among others related to critical thinking skills.^(40,41) Then, the third indicator is drawing conclusions. In the context of evaluating the use of AR, students find it easier to make conclusions from data that are presented visually and interactively. Also, making decisions based on simulations and real-time data can improve students’ critical skill in accounting to draw appropriate and quick conclusions. In line with studies, AR is indeed effective in improving student performance by providing hands-on learning experiences.⁽⁴²⁾

After that, the next indicator is Evaluates. Evaluating the accuracy and efficacy of various accounting strategies and procedures become easier by analyzing the application of AR. Students can observe the consequences of various financial actions and learn to assess options using more accurate criteria. This supports studies that AR reduces cognitive strain and bridges the gap between theoretical and practical knowledge.^(43,44) The fifth critical thinking indicator is explaining. AR can assist students in both understanding accounting concepts and clearly expressing these ideas to others. The ability to communicate complex concepts in a way that is easy to understand is a crucial component of critical thinking. The last indicator of critical thinking skills is Self-Regulation. With AR, students may adjust their learning process independently and receive immediate feedback.⁽⁴⁵⁾ Students can determine their own pace and learning path as well as acknowledge and correct their mistakes in real time.⁽⁴⁶⁾ This aids development of students’ self-regulation.

Nowadays, there is an immersive growth in Augmented Reality (AR) technology in educational settings, particularly in accounting lessons. This rapid growth cannot be separated from what AR has to offer, which is providing the visual and sensory experience. The students can easily understand the complex material with an interactive approach. AR in education provides rich visual stimuli, which helps students understand abstract concepts more easily.^(47,48) This is particularly relevant in accounting lessons, where concepts such as financial records, data analysis, and report interpretation can become more intuitive with AR visualization.

The reason for this is that student motivation and engagement are influenced by AR stories, which directly relate to improving critical thinking skills.⁽⁴⁹⁾ Among the critical thinking abilities are analysis, evaluation, and decision-making – all of which are vital for accounting. A study about an accounting lesson using AR for teaching basic financial reporting.⁽²²⁾ Improvement in conceptual understanding and practical application of knowledge was great among those students who used AR while it was not so with traditional methods as indicated by results. Similarly looked into whether AR can improve critical thinking skills or not.⁽⁵⁰⁾ They discovered that those learners who learned with augmented reality techniques had superior capabilities to appraise diverse fiscal circumstances based on the visual data given to them. The literature thus shows that AR technology enhances visual learning experiences as well as strengthening students’ critical thinking in accounting lessons. Henceforth, educational institutions should use AR technology within their accounting curriculum as it will facilitate improved academic achievements thereby equipping the learner with crucial skill sets that are fundamental in their careers.⁽⁵¹⁾

In an educational context, the N-Gain test in an educational setting helps measure the impact of learning interventions by comparing pretest and posttest scores. An increase in the N-Gain value indicates that the intervention had been successful and there was a great improvement in understanding. The findings from this study on N-Gains showed a 0,76 for experimental class and 0,63 for control class. These numbers suggested that experimental group had more positive result due to the intervention educational research. From these results, the experimental group achieved an N-Gain of 0,76 which falls under ‘High’ category while the control group had an N-Gain of 0,63 falling under ‘Medium’ category. This suggests that, in comparison to the control class, students in the experimental class experienced a more substantial increase in understanding.

Also, the learning methods or technology applied in experimental classes support students to understand the learning materials deeply and enhance their critical thinking skills. The use of such innovative learning methods evidently increased the N-Gain score. This is in line, who studied how interactive learning methods can generate a higher N-Gain score in comparison with traditional lecture methods.⁽⁵²⁾

The learning methods employed in experimental classes may include components that stimulate deeper cognitive engagement. Examples of pedagogical strategies are the flipped classroom, game-based learning, or collaborative learning. These strategies may not inescapably calculate on slice- edge technology, yet they’ve a profound impact on pupil engagement. In addition, the N- Gain can be told by other factors, including the quality of relations between preceptors and scholars, scholars’ literacy styles, and the social environment of literacy. constantly, the classroom dynamics that grease discussion and reflection can promote critical thinking and enhance appreciation.⁽⁵³⁾

This discussion highlights the significance of employing a comprehensive approach to tutoring. It isn’t only limited to the use of technology. But it also involves a broader and deeper understanding of the dynamics of literacy and social commerce. By examining the increase in N- Gain in the experimental class, it becomes apparent that multitudinous aspects can contribute to enhanced pupil understanding. All of these factors are good of being integrated into educational practice.

CONCLUSIONS

The use of mobile augmented reality (MAR) shows the positive impact in perfecting the critical thinking chops of vocational high school students. When compared with the control group, the experimental group showed more significant improvements in all indicators: interpretation, analysis, evaluation, drawing conclusions, explanation, and self-regulation. Based on these results, it means that MAR has proven to be effective in improving critical thinking skills in vocational education settings. Students can understand complex material through the visualization provided by MAR. The students’ learning experience is also interesting and enjoyable. Through MAR, learning becomes more interactive, immersive, and more focused on practical hands-on experiences. Overall, the transformational potential of MAR in improving vocational high school students’ educational experiences and cognitive skills become this research highlights.

REFERENCES

1. Arakawa T, Yamabe S, Suzuki T. Practice of Game Development Project-Based Learning Classes for Improving Disaster Management. Educ Sci (Basel). 2023 Oct 1;13(10).

2. Zhan Q, Li G, Zhan W. Measurement of the coupling coordination relationship between the structures of secondary vocational school programs and industries in China. Humanit Soc Sci Commun. 2023 Dec 1;10(1).

3. Hong J. The Application Optimization of Project based Learning Design in Ideological and Political Education in Colleges and Universities Based on ARCS Model. Applied Mathematics and Nonlinear Sciences. 2023 Jun 6;

4. Mičiulienė R, Kovalčikienė K. Motivation to Become a Vocational Teacher As a Second Career: A Mixed Method Study. Vocations and Learning. 2023 Dec 1;

5. Yusuf M, Syahputra FP, Lubis AA, Aswani A, Tambunan RW. Developing Genre-based English Worksheet for Vocational Students by Using 4-D Development Model to Improve Writing Skill. World Journal of English Language. 2023 Nov 8;14(1):34.

6. Arrasyid R, Ruhimat M, Setiawan I, Ihsan HM, Darmawan W, Mulyana A, et al. Design, Development, and Evaluation of a Mobile Learning Application for Geography Education. Journal of Advanced Research in Applied Sciences and Engineering Technology. 2024 Aug 1;38(1):109–34.

7. Kim KG, Oertel C, Dobricki M, Olsen JK, Coppi AE, Cattaneo A, et al. Using immersive virtual reality to support designing skills in vocational education. British Journal of Educational Technology. 2020 Nov 1;51(6):2199–213.

8. Yanto DTP, Kabatiah M, Zaswita H, Giatman G, Effendi H. Development of Virtual Learning using Problem-Based Learning Models for Vocational Education Students. Elinvo (Electronics, Informatics, and Vocational Education). 2023 Feb 19;7(2):163–72.

9. Suharno, Pambudi NA, Harjanto B. Vocational education in Indonesia: History, development, opportunities, and challenges. Vol. 115, Children and Youth Services Review. Elsevier Ltd; 2020.

10. Widiaty I, Ana A, Rifqi Mubaroq S, Ana A, Kuswardhana D, Achdiani Y. Development of Augmented Reality Technology in Vocational School [Internet]. Vol. 16, Journal of Engineering Science and Technology. 2021. Available from: https://www.researchgate.net/publication/363319811

11. Chao Xiang O, Ann TT, Hui CY, Yew LT. Effectiveness of Gamification in Vocational Technical Education.

12. Widiaty I, Yulia C, Gafar Abdullah A. The Application of Virtual Reality (VR) in Vocational Education: A Systematic Review. 2022.

13. Maulana MA, Sujana Y, Budianto A. Development of Module and Augmented Reality Based Android Aplication Computer Assembly For Vocational High School Student. Journal of Informatics and Vocational Education. 2019 Jul 27;2(2).

14. Ayaz A, Karacan Özdemir N. A case study of a Turkish vocational high school, and the challenges for teachers. Journal of Vocational Education and Training. 2023;75(3):459–78.

15. Singh G, Ahmad F. An interactive augmented reality framework to enhance the user experience and operational skills in electronics laboratories. Smart Learning Environments. 2024 Dec 1;11(1).

16. Wong KT, Hosshan H, Hanafi HF, Mudiono A. Augmented Reality (AR): An Assistive Technology for Special Education Needs. Journal of Advanced Research in Applied Sciences and Engineering Technology Journal homepage [Internet]. 2024; 35:97–105. Available from: https://semarakilmu.com.my/journals/index.php/applied_sciences_eng_tech/index

17. Singh G, Ahmad F. An interactive augmented reality framework to enhance the user experience and operational skills in electronics laboratories. Smart Learning Environments. 2024 Dec 1;11(1).

18. Li H, Li B. The state of metaverse research: a bibliometric visual analysis based on CiteSpace. J Big Data. 2024 Dec 1;11(1).

19. Alkan IB, Basaga HB. Augmented reality technologies in construction project assembly phases. Vol. 156, Automation in Construction. Elsevier B.V.; 2023.

20. Guo L, Li S, Xie S, Bian L, Shaharudin S. The impact of digital healthcare systems on pain and body function in patients with knee joint pain: a systematic review and meta-analysis. Sci Rep. 2024 Dec 1;14(1):3310.

21. Singh G, Ahmad F. An interactive augmented reality framework to enhance the user experience and operational skills in electronics laboratories. Smart Learning Environments. 2024 Dec 1;11(1).

22. Tuli N, Singh G, Mantri A, Sharma S. Augmented reality learning environment to aid engineering students in performing practical laboratory experiments in electronics engineering. Smart Learning Environments. 2022 Dec 1;9(1).

23. Chiou R, Phi T. Exploring Virtual Reality for Student Learning Enhancement on Environmentally Sustainable Manufacturing with Renewable Energy. In. Available from: www.slayte.com

24. Koumpouros Y. Revealing the true potential and prospects of augmented reality in education. Vol. 11, Smart Learning Environments. Springer; 2024.

25. Drouot M, Le Bigot N, Bricard E, Bougrenet JL de, Nourrit V. Augmented reality on industrial assembly line: Impact on effectiveness and mental workload. Appl Ergon. 2022 Sep 1;103.

26. Sukmawati F, Santosa EB, Rejekiningsih T. Design of Virtual Reality Zoos Through Internet of Things (IoT) for Student Learning about Wild Animals. Revue d’Intelligence Artificielle [Internet]. 2023 Apr 30;37(2):483–92. Available from: https://iieta.org/journals/ria/paper/10.18280/ria.370225

27. Zulherman* Z, Amirulloh G, Purnomo A, Aji GB, Supriansyah S. Development of Android-Based Millealab Virtual Reality Media in Natural Science Learning. Jurnal Pendidikan Sains Indonesia. 2021;9(1):1–10.

28. Marmoah S, Sukmawati F, Poerwanti JIS, Duca DS. Teacher Challenges in Designing the Learning after Curriculum Change: An Analysis of Learning Management System. 2023;13(6).

29. Estrada Villalba É, San Martín Azócar AL, Jacques-García FA. State of the art on immersive virtual reality and its use in developing meaningful empathy. Computers and Electrical Engineering. 2021;93(July).

30. Heining SM, Raykov V, Wolff O, Alkadhi H, Pape HC, Wanner GA. Augmented reality-based surgical navigation of pelvic screw placement: an ex-vivo experimental feasibility study. Patient Saf Surg. 2024 Dec 1;18(1).

31. Affendi FA, Junaini SN. Exploring the Impact of Mobile Augmented Reality on COVID-19 Prevention Education in Primary Schools. Journal of Advanced Research in Applied Sciences and Engineering Technology. 2024 Sep 1;39(2):231–41.

32. Jang C, Bang K, Chae M, Lee B, Lanman D. Waveguide holography for 3D augmented reality glasses. Nat Commun. 2024 Dec 1;15(1).

33. Pérez-Pachón L, Sharma P, Brech H, Gregory J, Lowe T, Poyade M, et al. Augmented reality headsets for surgical guidance: the impact of holographic model positions on user localisation accuracy. Virtual Real. 2024 Jun 1;28(2).

34. Tiwari AS, Bhagat KK, Lampropoulos G. Designing and evaluating an augmented reality system for an engineering drawing course. Smart Learning Environments. 2024 Dec 1;11(1).

35. Sari IP, Permana FC, Firmansyah FH, Hernawan AH. Computer-based learning: 3D visualization and animation as content development for digital learning materials for traditional Indonesian cloth (Songket Palembang). In: Journal of Physics: Conference Series. IOP Publishing Ltd; 2021.

36. Facione PA. Critical Thinking : What It Is and Why It Counts. Pearson Education: Englewood Cliffs, NJ,; 2011.

37. Facione P. Critical Thinking: What It Is and Why It Counts [Internet]. Pearson Education. NJ: Pearson Education: Englewood Cliffs; 2011. Available from: https://www.researchgate.net/publication/251303244

38. Lee H jung, Kim H, Byun H. Are high achievers successful in collaborative learning ? An explorative study of college students ’ learning approaches in team project-based learning explorative study of college students ’ learning approaches in. 2015;3297(January 2016).

39. Maulana MA, Sujana Y, Budianto A. Development of Module and Augmented Reality Based Android Aplication Computer Assembly For Vocational High School Student. Journal of Informatics and Vocational Education. 2019 Jul 27;2(2).

40. Nidhom AM, Putra ABNR, Smaragdina AA, Dyah KNG, Habibi MA, Yunos JM. The Integration of Augmented Reality into MOOC’s in Vocational Education to Support Education 3.0. International Journal of Interactive Mobile Technologies. 2022;16(3):20–31.

41. Lee CJ, Hsu Y. Sustainable education using augmented reality in vocational certification courses. Sustainability (Switzerland). 2021 Jun 1;13(11).

42. Ibrahim N, Saari EM, Azmi NH, Abbas MA, Hasin K, Ibrahim MD. The Evaluation of Augmented Reality Dictionary to Improve English Vocabulary. Journal of Advanced Research in Applied Sciences and Engineering Technology. 2024 Aug 1;38(2):129–41.

43. Yousefi Babadi S, Daneshmandi H. Effects of virtual reality versus conventional balance training on balance of the elderly. Exp Gerontol [Internet]. 2021;153(July):111498. Available from: https://doi.org/10.1016/j.exger.2021.111498

44. Fedorko G. Application possibilities of virtual reality in failure analysis of conveyor belts. Eng Fail Anal [Internet]. 2021;128(July):105615. Available from: https://doi.org/10.1016/j.engfailanal.2021.105615

45. Kohli V, Tripathi U, Chamola V, Rout BK, Kanhere SS. A Review on Virtual Reality and Augmented Reality Use-Cases of Brain Computer Interface Based Applications for Smart Cities. Microprocess Microsyst [Internet]. 2021;104392. Available from: https://doi.org/10.1016/j.micpro.2021.104392

46. Vidal-Balea A, Blanco-Novoa O, Picallo-Guembe I, Celaya-Echarri M, Fraga-Lamas P, Lopez-Iturri P, et al. Analysis, Design and Practical Validation of an Augmented Reality Teaching System Based on Microsoft HoloLens 2 and Edge Computing †. Engineering Proceedings. 2020;2(1).

47. Lee JD, Chien JC, Wang KC, Wu CT. Collaborative diagnosis in mixed-reality using deep-learning networks and RE-WAPICP algorithm. ICT Express. 2023 Nov;

48. CALVO PMH, FERNÁNDEZ JG. Dialogue, participation and social innovation initiatives for improving territorial governance in rural environments. Ciudades. 2021;(24):45–64.

49. Vidal-Balea A, Blanco-Novoa O, Picallo-Guembe I, Celaya-Echarri M, Fraga-Lamas P, Lopez-Iturri P, et al. Analysis, Design and Practical Validation of an Augmented Reality Teaching System Based on Microsoft HoloLens 2 and Edge Computing †. Engineering Proceedings. 2020;2(1).

50. Zhou C, Li JQ. The development of aesthetic experience through virtual and augmented reality. Sci Rep. 2024 Dec 1;14(1).

51. Mansor M, Sulaiman S, Wahid RA, Khan SBO, Ariffin AH, Shaberi AHA. AR-Speaking Buddy: Designing Content and User Interface of An Augmented Reality (AR)-Based Application for Malaysian Children Speaking Practice. Journal of Advanced Research in Applied Sciences and Engineering Technology. 2024 May 1;35(2):57–68.

52. Rolland JP, Goodsell J. Waveguide-based augmented reality displays: a highlight. Light Sci Appl. 2024 Dec 1;13(1).

53. Zhao X, Zhao H, Zheng W, Gohritz A, Shen Y, Xu W. Clinical evaluation of augmented reality-based 3D navigation system for brachial plexus tumor surgery. World J Surg Oncol. 2024 Dec 1;22(1).

FINANCING

This research is supported by Universitas Sebelas Maret

CONFLICT OF INTEREST

The authors declare that they have no competing interests

AUTHORSHIP CONTRIBUTION

Conceptualization: Fatma Sukmawati, Suparmi, Relly Prihatin, Eka Budhi Santosa.

Data curation: Fatma Sukmawati, Suparmi, Relly Prihatin, Eka Budhi Santosa.

Formal analysis: Fatma Sukmawati, Suparmi, Relly Prihatin, Eka Budhi Santosa.

Research: Fatma Sukmawati, Suparmi, Relly Prihatin, Eka Budhi Santosa.

Methodology: Fatma Sukmawati, Suparmi, Relly Prihatin, Eka Budhi Santosa.

Project management: Fatma Sukmawati, Suparmi, Relly Prihatin, Eka Budhi Santosa.

Resources: Fatma Sukmawati, Suparmi, Relly Prihatin, Eka Budhi Santosa.

Software: Fatma Sukmawati, Suparmi, Relly Prihatin, Eka Budhi Santosa.

Supervision: Fatma Sukmawati, Suparmi, Relly Prihatin, Eka Budhi Santosa.

Validation: Fatma Sukmawati, Suparmi, Relly Prihatin, Eka Budhi Santosa.

Display: Fatma Sukmawati, Suparmi, Relly Prihatin, Eka Budhi Santosa.

Drafting - original draft: Fatma Sukmawati, Suparmi, Relly Prihatin, Eka Budhi Santosa.

Writing - proofreading and editing: Fatma Sukmawati, Suparmi, Relly Prihatin, Eka Budhi Santosa.