An AI-Powered Teaching Performance Evaluation System: Technical Implementation and Pilot Testing at Universidad Técnica de Ambato
DOI:
https://doi.org/10.56294/saludcyt20252441Keywords:
Artificial intelligence, teaching evaluation, higher education, automation, multimodal analysis, quality assuranceAbstract
The article detailed the design, implementation and pilot test of an artificial intelligence-based system for evaluating teaching performance in higher education. The researcher aimed to create an automatic evaluation process by using speech recognition, semantic analysis and emotional detection, within an artificial intelligence architecture developed at Universidad Técnica de Ambato. The process was executed using open-source tools such as AssemblyAI, GPT-4o-mini, DeepFace and the n8n workflow platform which allowed autonomous analysis of recorded classroom sessions. A quasi experimental validation was actioned using 36 class recordings from 18 teachers from three disciplines. Overall, the findings indicated transcription accuracy of 96.4 %, inter-rater reliability above 90 % rubric agreement and substantial agreement with human raters (Cohen’s κ > 0.65; ICC > 0.80). Time for evaluation was reduced by greater 95 % and cost by 97 % compared with other peer review methods. These results confirmed the feasibility and reliability of the system for institutional quality assurance in teaching evaluation. The study concluded that artificial intelligence-based approaches could provide institutions with an open, efficient and scalable mechanism to assess the pedagogical performance that enhances innovation in higher education.
References
1. Kreitzer RJ, Sweet-Cushman J. 2021. Evaluating Student Evaluations of Teaching: a Review of Measurement and Equity Bias in SETs and Recommendations for Ethical Reform. J Acad Ethics. https://doi.org/10.1007/s10805-021-09400-w
2. Otu N, Otu NE. 2023. Student Evaluations of Teaching Are Mostly Awfully Wrong. Universal J Educ Res, 2:1–16.
3. Rubie-Davies CM. 2006. Teacher expectations and student self-perceptions: Exploring relationships. Psychol Sch, 43(5):537–52. https://doi.org/10.1002/pits.20169
4. Sigurdardottir MS, Rafnsdottir GL, Jónsdóttir AH, Kristofersson DM. 2023. Student evaluation of teaching: gender bias in a country at the forefront of gender equality. High Educ Res Dev, 42(4):954–67. https://doi.org/10.1080/07294360.2022.2087604
5. Adams S, Bekker S, Fan Y, Gordon T, Shepherd LJ, Slavich E, et al. 2022. Gender Bias in Student Evaluations of Teaching: ‘Punish[ing] Those Who Fail To Do Their Gender Right.’ High Educ (Dordr), 83(4):787–807. https://doi.org/10.1007/s10734-021-00704-9
6. Lawson RA, Stephenson EF. 2005. Easiness, attractiveness, and faculty evaluations: Evidence from RateMyProfessors.com. Atl Econ J, 33(4):485–6. https://doi.org/10.1007/s11293-005-2873-z
7. Bartlett AD, Um IS, Luca EJ, Krass I, Schneider CR. 2020. Measuring and assessing the competencies of preceptors in health professions: A systematic scoping review. BMC Med Educ, 20(1):1–9. https://doi.org/10.1186/s12909-020-02082-9
8. Kumar A, Jain R. 2018. Faculty Evaluation System. Procedia Comput Sci, 125:533–41. https://doi.org/10.1016/j.procs.2017.12.069
9. Watson D, Amin SN, Pino N. 2022. Self-evaluating performance: an analysis of police integrity, professionalism and service provision from the South Pacific. Policing Soc, 32(1):89–102. https://doi.org/10.1080/10439463.2021.1888950
10. Chionidou-Moskofoglou M, Doukakis S, Lappa A. 2021. The use of e-portfolios in teaching and assessment. https://doi.org/10.48550/arXiv.2105.15114
11. Rodgers WJ, Kennedy MJ, VanUitert VJ, Myers AM. 2019. Delivering Performance Feedback to Teachers Using Technology-Based Observation and Coaching Tools. Interv Sch Clin, 55(2):103–12. https://doi.org/10.1177/1053451219837640
12. Zhai X, Yin Y, Pellegrino JW, Haudek KC, Shi L. 2020. Applying machine learning in science assessment: a systematic review. Stud Sci Educ, 56(1):111–51. https://doi.org/10.1080/03057267.2020.1735757
13. González-Calatayud V, Prendes-Espinosa P, Roig-Vila R. 2021. Artificial Intelligence for Student Assessment: A Systematic Review. Appl Sci 2021, 11(12):5467. https://doi.org/10.3390/app11125467
14. Kaswan KS, Dhatterwal JS, Ojha RP. 2024. AI in personalized learning. En: Adv Technol Innov High Educ Theory Pract. p. 103–17.
15. Romero C, Ventura S. 2020. Educational data mining and learning analytics: An updated survey. Wiley Interdiscip Rev Data Min Knowl Discov, 10(3):e1355. https://doi.org/10.1002/widm.1355
16. Perez-Ortiz M, Novak E, Bulathwela S, Shawe-Taylor J. 2021. An AI-based Learning Companion Promoting Lifelong Learning Opportunities for All. https://doi.org/10.48550/arXiv.2112.01242
17. Merino-Campos C. 2025. The Impact of Artificial Intelligence on Personalized Learning in Higher Education: A Systematic Review. Trends High Educ 2025, 4(2):17. https://doi.org/10.3390/higheredu4020017
18. Ma K, Zhang Y, Hui B. 2024. How Does AI Affect College? The Impact of AI Usage in College Teaching on Students’ Innovative Behavior and Well-Being. Behav Sci 2024, 14(12):1223. https://doi.org/10.3390/bs14121223
19. Chai F, Ma J, Wang Y, Zhu J, Han T. 2024. Grading by AI makes me feel fairer? How different evaluators affect college students’ perception of fairness. Front Psychol, 15:1221177. https://doi.org/10.3389/fpsyg.2024.1221177
20. Niculescu AI, Ehnen J, Yi C, Jiawei D, Pin TC, Zhou JT, et al. 2025. On the development of an AI performance and behavioural measures for teaching and classroom management. https://doi.org/10.48550/arXiv.2506.11143
21. OpenAI, Achiam J, Adler S, Agarwal S, Ahmad L, Akkaya I, et al. 2023. GPT-4 Technical Report. https://doi.org/10.48550/arXiv.2303.08774
22. An Y, Kolanupaka S, An J, Ma M, Chhatwal U, Kalinowski A, et al. 2024. Is the Lecture Engaging for Learning? Lecture Voice Sentiment Analysis for Knowledge Graph-Supported Intelligent Lecturing Assistant (ILA) System. https://doi.org/10.48550/arXiv.2408.10492
23. Chen J, Lai P, Chan A, Man V, Chan CH. 2022. AI-Assisted Enhancement of Student Presentation Skills: Challenges and Opportunities. Sustainability 2023, 15(1):196. https://doi.org/10.3390/su15010196
24. Giese S. 2021. Automatisierungsservice Zapier. En: Das automatisierte Maklerbüro. p. 65–71. https://doi.org/10.1007/978-3-658-33275-4_5
25. Jeong C. 2025. Beyond Text: Implementing Multimodal Large Language Model-Powered Multi-Agent Systems Using a No-Code Platform. J Intell Inf Syst, 31(1):191–231. https://doi.org/10.13088/jiis.2025.31.1.191
26. Owoc ML, Sawicka A, Weichbroth P. 2021. Artificial Intelligence Technologies in Education: Benefits, Challenges and Strategies of Implementation. IFIP Adv Inf Commun Technol, 599:37–58. https://doi.org/10.1007/978-3-030-85001-2_4
27. Liu Q, Geertshuis S, Grainger R. 2020. Understanding academics’ adoption of learning technologies: A systematic review. Comput Educ, 151:103857. https://doi.org/10.1016/j.compedu.2020.103857
28. Al-Zahrani AM. 2024. Unveiling the shadows: Beyond the hype of AI in education. Heliyon, 10(9):e30696. https://doi.org/10.1016/j.heliyon.2024.e30696
29. Cha H, Park T, Seo J. 2020. What Should Be Considered when Developing ICT-Integrated Classroom Models for a Developing Country? Sustainability 2020, 12(7):2967. https://doi.org/10.3390/su12072967
30. Zou Y, Kuek F, Feng W, Cheng X. 2025. Digital learning in the 21st century: trends, challenges, and innovations in technology integration. Front Educ (Lausanne), 10:1562391. https://doi.org/10.3389/feduc.2025.1562391
31. Salas-Pilco SZ, Yang Y. 2022. Artificial intelligence applications in Latin American higher education: a systematic review. Int J Educ Technol High Educ, 19(1):1–20. https://doi.org/10.1186/s41239-022-00326-w
32. Hilliger I, Ortiz-Rojas M, Pesántez-Cabrera P, Scheihing E, Tsai YS, Muñoz-Merino PJ, et al. 2020. Towards learning analytics adoption: A mixed methods study of data-related practices and policies in Latin American universities. Br J Educ Technol, 51(4):915–37. https://doi.org/10.1111/bjet.12933
33. Dolan EL, Elliott SL, Henderson C, Curran-Everett D, St. John K, Ortiz PA. 2018. Evaluating Discipline-Based Education Research for Promotion and Tenure. Innov High Educ, 43(1):31–9. https://doi.org/10.1007/s10755-017-9406-y
34. Southwell R, Pugh S, Perkoff EM, Clevenger C, Bush JB, Lieber R, et al. 2022. Challenges and Feasibility of Automatic Speech Recognition for Modeling Student Collaborative Discourse in Classrooms. En: Proc 15th Int Conf Educ Data Mining, EDM 2022. International Educational Data Mining Society.
35. Li W, Mohamad M, You HW. 2025. Integrating automatic speech recognition and automated writing evaluation to reduce speaking anxiety and enhance speaking competence among Chinese EFL learners. Cogent Educ, 12(1). https://doi.org/10.1080/2331186X.2025.2559161
36. McGuire M, Larson-Hall J. 2025. Assessing Whisper automatic speech recognition and WER scoring for elicited imitation: Steps toward automation. Res Methods Appl Linguist, 4(1):100197. https://doi.org/10.1016/j.rmal.2025.100197
37. Kortemeyer G, Nöhl J. 2025. Assessing confidence in AI-assisted grading of physics exams through psychometrics: An exploratory study. Phys Rev Phys Educ Res, 21(1):010136. https://doi.org/10.1103/PhysRevPhysEducRes.21.010136
38. Ngoc Vu N, Thi Hong Lien N. 2023. Leveraging AI for higher education: The role and implications of chatGPT: THE ROLE AND IMPLICATIONS OF CHATGPT. Tạp chí Khoa học HUFLIT, 7(4):29–29.
39. CACES. 2023. Modelo de evaluación externa con fines de acreditación para el aseguramiento de la calidad de las Universidades y Escuela Politécnicas. https://www.caces.gob.ec/universidades-y-escuelas-politecnicas-3/
40. Ahlawat H, Aggarwal N, Gupta D. 2025. Automatic Speech Recognition: A survey of deep learning techniques and approaches. Int J Cogn Comput Eng, 6:201–37. https://doi.org/10.1016/j.ijcce.2024.12.007
41. Zewei, Tian, Liu A, Esbenshade L, Sarkar S, Zhang Z, et al. 2025. Implementation Considerations for Automated AI Grading of Student Work. https://doi.org/10.48550/arXiv.2506.07955
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Wilber Orlando Romero Villarroel, Sara Nidhya Camacho Estrada, Héctor Santiago López Zurita, Danilo Fabricio Trujillo Ronquillo, Carlos Patricio Rodríguez Hurtado, Edison Gerardo Llerena Medina (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
The article is distributed under the Creative Commons Attribution 4.0 License. Unless otherwise stated, associated published material is distributed under the same licence.
