A Systematic Literature Review of Automated Feedback Generation in Education
DOI:
https://doi.org/10.46328/ijte.5347Keywords:
adapted learning, automated feedback generation, systematic literature review, data-driven education, feedback, technology-enhanced learningAbstract
Feedback that is individualized and immediate is essential to improving learning outcomes but providing it to every learner is difficult. Automatic feedback generation (AFG) aims to alleviate this problem, especially with technology-enhanced learning environments. This systematic literature review of AFG in education, following the PRISMA framework, examines 34 peer-reviewed publications. The findings revealed that the reviewed studies (1) gained momentum after 2019; (2) often used secondary cognitive data to evaluate AFG approaches; (3) mainly targeted computer science domain; (4) frequently combined multiple methods to generate feedback; (5) employed multiple performance evaluations; and (6) mostly provided written feedback aimed at correcting student errors. This review also highlighted several gaps, including the lack of (1) in-depth cognitive and affective data from user studies to evaluate feedback and understand how students interpret it; (2) research on feedback use and strategies to close feedback loop; (3) AFG systems for ill-defined domains with strong transferability; (4) elaborated feedback that scaffolds problem-solving rather than giving answers; (5) feedback using multiple modalities and valences; and (6) integration of learning theories in AFG design. This review advances understanding of current AFG practices, evaluates and extends conceptual frameworks of AFG, and provides insights for future AFG design and evaluation.
References
Ahmed, U., Fan, Z., Yi, J., Al-Bataineh, O., & Roychoudhury, A. (2022). Verifix: Verified repair of programming assignments. ACM Transactions on Software Engineering and Methodology, 31(4), 1–31. https://doi.org/10.1145/3510418
Akobeng, A. K. (2005). Understanding systematic reviews and meta-analysis. Archives of Disease in Childhood, 90(8), 845–848. https://doi.org/10.1136/adc.2004.058230
Bangert-Drowns, R. L., Kulik, C.-L. C., Kulik, J. A., & Morgan, M. (1991). The instructional effect of feedback in test-like events. Review of Educational Research, 61(2), 213–238.
Behzad, S., Kashefi, O., & Somasundaran, S. (2024). LEAF: Language learners’ English essays and feedback corpus. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2, 433–442. https://doi.org/10.18653/v1/2024.naacl-short.36
Bellhäuser, H., Liborius, P., & Schmitz, B. (2022). Fostering self-regulated learning in online environments: Positive effects of a web-based training with peer feedback on learning behavior. Frontiers in Psychology, 13, 813381. https://doi.org/10.3389/fpsyg.2022.813381
Beltrami, P., Crescenzi, P., Gensini, G., Innocenti, A., Lippi, P., & Saccone, N. (2006). Automatic feedback generation in scenario-based e-learning with an application to the healthcare sector. Journal of E-Learning and Knowledge Society, 2(2), 229–240. https://doi.org/10.20368/1971-8829/716
Bhatia, S., Kohli, P., & Singh, R. (2018). Neuro-symbolic program corrector for introductory programming assignments. In Proceedings of the 40th International Conference on Software Engineering, 60–70. https://doi.org/10.1145/3180155.3180219
Boud, D., & Molloy, E. (2013). Rethinking models of feedback for learning: The challenge of design. Assessment & Evaluation in Higher Education, 38(6), 698–712. https://doi.org/10.1080/02602938.2012.691462
Buckingham Shum, S., Lim, L.-A., Boud, D., Bearman, M., & Dawson, P. (2023). A comparative analysis of the skilled use of automated feedback tools through the lens of teacher feedback literacy. International Journal of Educational Technology in Higher Education, 20(1), 40. https://doi.org/10.1186/s41239-023-00410-9
Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65(3), 245–281. https://doi.org/10.3102/00346543065003245
Carless, D., & Boud, D. (2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315–1325. https://doi.org/10.1080/02602938.2018.1463354
Cavalcanti, A. P., Barbosa, A., Carvalho, R., Freitas, F., Tsai, Y.-S., Gašević, D., & Mello, R. F. (2021). Automatic feedback in online learning environments: A systematic literature review. Computers and Education: Artificial Intelligence, 2, 100027. https://doi.org/10.1016/j.caeai.2021.100027
Covidence (2025). Covidence systematic review software. [Computer software]. Veritas Health Innovation. www.covidence.org
Dai, W., Tsai, Y.-S., Lin, J., Aldino, A., Jin, H., Li, T., Gašević, D., & Chen, G. (2024). Assessing the proficiency of large language models in automatic feedback generation: An evaluation study. Computers and Education: Artificial Intelligence, 7, 100299.
Deeva, G., Bogdanova, D., Serral, E., Snoeck, M., & De Weerdt, J. (2021). A review of automated feedback systems for learners: Classification framework, challenges and opportunities. Computers & Education, 162, 104094. https://doi.org/10.1016/j.compedu.2020.104094
Dzikovska, M., Steinhauser, N., Farrow, E., Moore, J., & Campbell, G. (2014). BEETLE II: Deep natural language understanding and automatic feedback generation for intelligent tutoring in basic electricity and electronics. International Journal of Artificial Intelligence in Education, 24(3), 284–332. https://doi.org/10.1007/s40593-014-0017-9
Ertmer, P. A., & Newby, T. J. (2013). Behaviorism, cognitivism, constructivism: Comparing critical features from an instructional design perspective. Performance Improvement Quarterly, 26(2), 43–71. https://doi.org/10.1002/piq.21143
Evans, C. (2013). Making sense of assessment feedback in higher education. Review of Educational Research, 83(1), 70–120. https://doi.org/10.3102/0034654312474350
Filighera, A., Tschesche, J., Steuer, T., Tregel, T., & Wernet, L. (2022). Towards generating counterfactual examples as automatic short answer feedback. In M. M. Rodrigo, N. Matsuda, A. I. Cristea, & V. Dimitrova (Eds.), Artificial Intelligence in Education (Vol. 13355, pp. 206–217). Springer International Publishing. https://doi.org/10.1007/978-3-031-11644-5_17
Fossati, D., Di Eugenio, B., Ohlsson, S., Brown, C., & Chen, L. (2015). Data driven automatic feedback generation in the iList intelligent tutoring system. Technology, Instruction, Cognition and Learning, 10(1), 5–26.
Guid, M., Možina, M., Pavlič, M., & Turšič, K. (2019). Learning by arguing in argument-based machine learning framework. In A. Coy, Y. Hayashi, & M. Chang (Eds.), Intelligent Tutoring Systems (Vol. 11528, pp. 112–122). Springer International Publishing. https://doi.org/10.1007/978-3-030-22244-4_15
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487
Haut, K., Wohn, C., Kane, B., Carroll, T., Guigno, C., Kumar, V., Epstein, R., Schuber, L., & Hoque, E. (2023). Validating a virtual human and automated feedback system for training doctor-patient communication skills. In 2023 11th International Conference on Affective Computing and Intelligent Interaction (ACII), 1–8. https://doi.org/10.1109/ACII59096.2023.10388213
Heo, J., Jeong, H., Choi, D., & Lee, E. (2023). REFERENT: Transformer-based feedback generation using assignment information for programming course. In 2023 IEEE/ACM 45th International Conference on Software Engineering: Software Engineering Education and Training, 101–106. https://doi.org/10.1109/ICSE-SEET58685.2023.00035
Hossain, S., Kamzin, A., Amperayani, V. N. S. A., Paudyal, P., Banerjee, A., & Gupta, S. K. S. (2021). Engendering trust in automated feedback: A two step comparison of feedbacks in gesture based learning. In I. Roll, D. McNamara, S. Sosnovsky, R. Luckin, & V. Dimitrova (Eds.), Artificial Intelligence in Education (pp. 190–202). Springer International Publishing. https://doi.org/10.1007/978-3-030-78292-4_16
Huang, W., Stephens, J. M., & Brown, G. T. L. (2025). Feedback assisted by technology: A systematic review of empirical research. International Journal of Technology in Education, 8(2), 421–444. https://doi.org/10.46328/ijte.1061
Ice, P., Swan, K., Diaz, S., Kupczynski, L., & Swan-Dagen, A. (2010). An analysis of students’ perceptions of the value and efficacy of instructors’ auditory and text-based feedback modalities across multiple conceptual levels. Journal of Educational Computing Research, 43(1), 113–134. https://doi.org/10.2190/EC.43.1.g
Jia, Q., Young, M., Xiao, Y., Cui, J., Liu, C., Rashid, P., & Gehringer, E. (2022). Automated feedback generation for student project reports: A data-driven approach. Journal of Educational Data Mining, 14(3), 132–161. https://doi.org/10.5281/zenodo.7304954
Jonsson, A. (2013). Facilitating productive use of feedback in higher education. Active Learning in Higher Education, 14(1), 63–76. https://doi.org/10.1177/1469787412467125
Keuning, H., Jeuring, J., & Heeren, B. (2018). A systematic literature review of automated feedback generation for programming exercises. ACM Transactions on Computing Education, 19(1), 1–43. https://doi.org/10.1145/3231711
Kim, D., Kwon, Y., Liu, P., Kim, I. L., Perry, D. M., Zhang, X., & Rodriguez-Rivera, G. (2016). Apex: Automatic programming assignment error explanation. ACM SIGPLAN Notices, 51, 311–327. https://doi.org/10.1145/2983990.2984031
Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254–284. https://doi.org/10.1037/0033-2909.119.2.254
Kulhavy, R. W., & Stock, W. A. (1989). Feedback in written instruction: The place of response certitude. Educational Psychology Review, 1(4), 279–308. https://doi.org/10.1007/BF01320096
Kylvaja, M., Kumpulainen, P., & Konu, A. (2019). Application of data clustering for automated feedback generation about student well-being. In Proceedings of the 1st ACM SIGSOFT International Workshop on Education Through Advanced Software Engineering and Artificial Intelligence, 21–26. https://doi.org/10.1145/3340435.3342720
Laine, T. H., & Lindberg, R. S. N. (2020). Designing engaging games for education: A systematic literature review on game motivators and design principles. IEEE Transactions on Learning Technologies, 13(4), 804–821. https://doi.org/10.1109/TLT.2020.3018503
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. JSTOR. https://doi.org/10.2307/2529310
Larrondo, P., Frank, B., & Ortiz, J. (2021). The state of the art in providing automated feedback to open-ended student work. In Proceedings of the Canadian Engineering Education Association (CEEA), 1–8. https://doi.org/10.24908/pceea.vi0.14854
Lasserson, T. J., Thomas, J., & Higgins, J. P. (2019). Starting a review. In J. P. Higgins, J. Thomas, J. Chandler, M. Cumpston, T. Li, M. J. Page, & V. A. Welch (Eds.), Cochrane handbook for systematic reviews of interventions (pp. 1–12). Wiley Online Library. https://doi.org/10.1002/9781119536604.ch1
Lipnevich, A. A., Berg, D. A., & Smith, J. K. (2016). Toward a model of student response to feedback. In Handbook of Human and Social Conditions in Assessment (pp. 169–185). Routledge.
Lipnevich, A. A., & Panadero, E. (2021). A review of feedback models and theories: Descriptions, definitions, and conclusions. Frontiers in Education, 6, 720195. https://doi.org/10.3389/feduc.2021.720195
Lu, C., & Cutumisu, M. (2021). Integrating deep learning into an automated feedback generation system for automated essay scoring. In Proceedings of The 14th International Conference on Educational Data Mining (EDM21), 573–579. https://educationaldatamining.org/edm2021/
Luo, H., Yang, T., Xue, J., & Zuo, M. (2019). Impact of student agency on learning performance and learning experience in a flipped classroom. British Journal of Educational Technology, 50(2), 819–831. https://doi.org/10.1111/bjet.12604
Máñez, I., Vidal-Abarca, E., Kendeou, P., & Martínez, T. (2019). How do students process complex formative feedback in question-answering tasks? A think-aloud study. Metacognition and Learning, 14(1), 65–87. https://doi.org/10.1007/s11409-019-09192-w
Marwan, S., Lytle, N., Williams, J. J., & Price, T. (2019). The impact of adding textual explanations to next-step hints in a novice programming environment. In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education, 520–526. https://doi.org/10.1145/3304221.3319759
Mason, B. J., & Bruning, R. (2001). Providing feedback in computer-based instruction: What the research tells us? (No. 9). Center for Instructional Innovation.
Molloy, E. K., & Boud, D. (2014). Feedback models for learning, teaching and performance. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of Research on Educational Communications and Technology (pp. 413–424). Springer New York. https://doi.org/10.1007/978-1-4614-3185-5_33
Murtagh, L. (2014). The motivational paradox of feedback: Teacher and student perceptions. The Curriculum Journal, 25(4), 516–541. https://doi.org/10.1080/09585176.2014.944197
Narciss, S. (2008). Feedback strategies for interactive learning tasks. In Handbook of research on educational communications and technology (3rd Edition, pp. 125–143). Routledge.
Narciss, S., & Huth, K. (2004). How to design informative tutoring feedback for multi-media learning. In H. M. Niegemann, D. Leutner, & R. Brünken (Eds.), Instructional Design for Multimedia Learning (pp. 181–195). Waxmann.
Nayak, S., Agarwal, R., Khatri, S. K., & Mohammadian, M. (2024). Student outcome assessment on structured query language using rubrics and automated feedback generation. International Journal of Advanced Computer Science and Applications, 15(3), 728. https://doi.org/10.14569/IJACSA.2024.0150374
Nicol, D., & Macfarlane-Dick, D. (2006). Formative assessment and self‐regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. https://doi.org/10.1080/03075070600572090
Obaido, G., Ade-Ibijola, A., & Vadapalli, H. (2020). TalkSQL: A tool for the synthesis of SQL queries from verbal specifications. In 2020 2nd International Multidisciplinary Information Technology and Engineering Conference (IMITEC), 1–10. https://doi.org/10.1109/IMITEC50163.2020.9334088
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. International Journal of Surgery, 88, 105906. https://doi.org/10.1016/j.ijsu.2021.105906
Panadero, E., & Lipnevich, A. A. (2022). A review of feedback models and typologies: Towards an integrative model of feedback elements. Educational Research Review, 35, 100416. https://doi.org/10.1016/j.edurev.2021.100416
Pardo, A., Jovanovic, J., Dawson, S., Gašević, D., & Mirriahi, N. (2019). Using learning analytics to scale the provision of personalised feedback. British Journal of Educational Technology, 50(1), 128–138. https://doi.org/10.1111/bjet.12592
Piech, C., Huang, J., Nguyen, A., Phulsuksombati, M., Sahami, M., & Guibas, L. (2015). Learning program embeddings to propagate feedback on student code. In Proceedings of the 32nd International Conference on Machine Learning, 1093–1102. http://proceedings.mlr.press/v37/piech15.html
Poulos, A., & Mahony, M. J. (2008). Effectiveness of feedback: The students’ perspective. Assessment & Evaluation in Higher Education, 33(2), 143–154. https://doi.org/10.1080/02602930601127869
Rivers, K., & Koedinger, K. R. (2013). Automatic generation of programming feedback: A data-driven approach. In The First Workshop on AI-Supported Education for Computer Science (AIEDCS 2013), 50, 50–59.
Rose, K. J. (2023). Using classification of instructional program codes in human resource development. Human Resource Development Review, 22, 428–444. https://doi.org/10.1177/15344843231184101
Rudolph, E., Seer, H., Mothes, C., & Albrecht, J. (2024). Automated feedback generation in an intelligent tutoring system for counselor education. In Proceedings of the 19th Conference on Computer Science and Intelligence Systems (FedCSIS), 501–512. https://doi.org/10.15439/2024F1649
Ruiz, J., & Snoeck, M. (2021). Automatic feedback generation for supporting user interface design. In H. Fill, M. VanSinderen, & L. Maciaszek (Eds.), In Proceedings of the 16th International Conference on Software Technologies (ICSOFT 2021) (pp. 23–33). https://doi.org/10.5220/0010513400230033
Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18(2), 119–144. https://doi.org/10.1007/BF00117714
Sargeant, J. M., & O’Connor, A. M. (2020). Scoping reviews, systematic reviews, and meta-analysis: Applications in veterinary medicine. Frontiers in Veterinary Science, 7, 11. https://doi.org/10.3389/fvets.2020.00011
Serral Asensio, E., Ruiz, J., Elen, J., & Snoeck, M. (2019). Conceptualizing the domain of automated feedback for learners. In Proceedings of the XXII IberoAmerican Conference on Software Engineering, CIbSE 2019, 223–236.
Shadiev, R., & Feng, Y. (2023). Using automated corrective feedback tools in language learning: A review study. Interactive Learning Environments, 1–29. https://doi.org/10.1080/10494820.2022.2153145
Singh, R., Gulwani, S., & Solar-Lezama, A. (2013). Automated feedback generation for introductory programming assignments. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation, 15–26. https://doi.org/10.1145/2462156.2462195
Sondergaard, H., & Thomas, D. (2004). Effective feedback to small and large classes. In 34th Annual Frontiers in Education, 2004. FIE 2004., F1E-9. https://doi.org/10.1109/FIE.2004.1408573
Sychev, O. (2022). Write a line: Tests with answer templates and string completion hints for self-learning in a CS1 course. In Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Software Engineering Education and Training, 265–276. https://doi.org/10.1109/ICSE-SEET55299.2022.9794157
Thorndike, E. L. (1927). The law of effect. American Journal of Psychology, 39(1/4), 212–222. JSTOR. https://doi.org/10.2307/1415413
Toma, I., Marica, A., Dascalu, M., & Trausan-Matu, S. (2021). Readerbench—Automated feedback generation for essays in Romanian. University Politehnica of Bucharest Scientific Bulletin Series C-Electrical Engineering and Computer Science, 83(2), 21–34.
Tunstall, P., & Gsipps, C. (1996). Teacher feedback to young children in formative assessment: A typology. British Educational Research Journal, 22(4), 389–404. https://doi.org/10.1080/0141192960220402
Vittorini, P., & Galassi, A. (2023). rDSA: An intelligent tool for data science assignments. Multimedia Tools and Applications, 82(9), 12879–12905. https://doi.org/10.1007/s11042-022-14053-x
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes (M. Cole, Ed.). Harvard University Press; WorldCat.
Wiener, N. (1954). The human use of human beings: Cybernetics and society. New Haven: Houghton Mifflin.
Wingate, U. (2012). ‘Argument!’ helping students understand what essay writing is about. Journal of English for Academic Purposes, 11(2), 145–154. https://doi.org/10.1016/j.jeap.2011.11.001
Wisniewski, B., Zierer, K., & Hattie, J. (2019). The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology, 10, 1664–1708. https://doi.org/10.3389/fpsyg.2019.03087
Wisniewski, B., Zierer, K., & Hattie, J. (2020). The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology, 10, 1664–1708. https://doi.org/10.3389/fpsyg.2019.03087
Wolcott, M. D., & McLaughlin, J. E. (2024). Exploring user experience (UX) research methods in health professions education. Currents in Pharmacy Teaching and Learning, 16(2), 144–149. https://doi.org/10.1016/j.cptl.2023.12.010
Yu, S., Jiang, L., & Zhou, N. (2020). Investigating what feedback practices contribute to students’ writing motivation and engagement in Chinese EFL context: A large scale study. Assessing Writing, 44, 100451. https://doi.org/10.1016/j.asw.2020.100451
Downloads
Published
Issue
Section
License
Copyright (c) 2026 International Journal of Technology in Education
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Articles may be used for research, teaching, and private study purposes. Authors alone are responsible for the contents of their articles. The journal owns the copyright of the articles. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of the research material.
The author(s) of a manuscript agree that if the manuscript is accepted for publication in the International Journal of Technology in Education (IJTE), the published article will be copyrighted using a Creative Commons “Attribution 4.0 International” license. This license allows others to freely copy, distribute, and display the copyrighted work, and derivative works based upon it, under certain specified conditions.
Authors are responsible for obtaining written permission to include any images or artwork for which they do not hold copyright in their articles, or to adapt any such images or artwork for inclusion in their articles. The copyright holder must be made explicitly aware that the image(s) or artwork will be made freely available online as part of the article under a Creative Commons “Attribution 4.0 International” license.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
