Psychometric Properties of Generative AI Literacy Scale for Preservice Teachers: Evidence from a Scoping Review and Factor Analytic Validation
DOI:
https://doi.org/10.46328/ijte.6082Keywords:
Generative AI, AI literacy, Preservice Teachers, Psychometric PropertiesAbstract
Amid the rapid diffusion of generative artificial intelligence in teacher education, there is a pressing need for a validated measure that captures how future educators understand, apply, and govern these systems in instructional contexts. This study developed and validated the Generative Artificial Intelligence Literacy Scale for Preservice Teachers (GAIL-PT), a psychometrically grounded instrument designed to assess preservice teachers’ understanding, application, and ethical engagement with generative artificial intelligence (AI) in educational contexts. Guided by established frameworks for instrument development, the research followed six sequential stages: conceptualization through a systematic scoping review, item generation, expert validation using the Content Validity Index (CVI), pilot testing and refinement, exploratory factor analysis (EFA), and confirmatory factor analysis (CFA). The initial instrument consisted of 42 items distributed across six conceptual domains: conceptual understanding, use and application, evaluation and verification, ethics and responsibility, pedagogical integration, and affective readiness. Data were collected from 402 preservice teachers across various education programs at Bukidnon State University. Exploratory factor analysis using maximum likelihood extraction with Promax rotation confirmed sampling adequacy and factorability, resulting in a refined two-factor model explaining nearly half of the total variance. The emergent factors, Applied Engagement and Instructional Integration and Responsible and Reflective Use, encapsulate both the practical and ethical-pedagogical dimensions of AI literacy. Confirmatory factor analysis supported the adequacy of this two-factor structure, demonstrating acceptable reliability and overall model fit across key indices. The results affirm that generative AI literacy among preservice teachers is a multidimensional construct encompassing cognitive, behavioral, and ethical competencies. The validated GAIL-PT offers a theoretically coherent and empirically robust tool for assessing generative AI literacy among preservice teachers.
References
Abdulayeva, A., Zhanatbekova, N., Andasbayev, Y., & Boribekova, F. (2025). Fostering AI literacy in pre-service physics teachers: Inputs from training and co-variables. Frontiers in Education. https://doi.org/10.3389/feduc.2025.1505420
Al-Abdullatif, A. (2025). Auditing AI literacy competency in K–12 education: The role of awareness, ethics, evaluation, and use in human–machine cooperation. Systems. https://doi.org/10.3390/systems13060490
Al-Abdullatif, A. M. (2024). Modeling teachers’ acceptance of generative artificial intelligence use in higher education: The role of AI literacy, intelligent TPACK, and perceived trust. Education Sciences, 14(11), 1209. https://doi.org/10.3390/educsci14111209
Allen, L. K., & Kendeou, P. (2023). ED-AI Lit: An interdisciplinary framework for AI literacy in education. Policy Insights from the Behavioral and Brain Sciences, 11(1), 3–10. https://doi.org/10.1177/23727322231220339
Alqarni, A. (2025). Artificial Intelligence-Critical Pedagogic: Design and psychologic validation of a teacher-specific scale for enhancing critical thinking in classrooms. Journal of Computer Assisted Learning. https://doi.org/10.1111/jcal.70039
Arksey, H., & O’Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19–32. https://doi.org/10.1080/1364557032000119616
Bahroun, Z., Anane, C., Ahmed, V., & Zacca, A. (2023). Transforming education: A comprehensive review of generative artificial intelligence in educational settings through bibliometric and content analysis. Sustainability. https://doi.org/10.3390/su151712983
Bauer, E., Greiff, S., Graesser, A., Scheiter, K., & Sailer, M. (2025). Looking beyond the hype: Understanding the effects of AI on learning. Educational Psychology Review. https://doi.org/10.1007/s10648-025-10020-8
Boateng, G. O., Neilands, T. B., Frongillo, E. A., Melgar-Quiñonez, H. R., & Young, S. L. (2018). Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontiers in Public Health, 6, 149. https://doi.org/10.3389/fpubh.2018.00149
Bower, M., Torrington, J., Lai, J., Petocz, P., & Alfano, M. (2024). How should we change teaching and assessment in response to increasingly powerful generative artificial intelligence? Outcomes of the ChatGPT teacher survey. Education and Information Technologies, 29, 15403–15439. https://doi.org/10.1007/s10639-023-12405-0
Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). Guilford Press.
Carolus, A., Koch, M. J., Straka, S., Latoschik, M. E., & Wienrich, C. (2023). MAILS—Meta AI Literacy Scale: Development and testing of an AI literacy questionnaire. Computers in Human Behavior: Artificial Humans, 1(2), 100014. https://doi.org/10.1016/j.chbah.2023.100014
Celik, I. (2022). Towards Intelligent-TPACK: An empirical study on teachers’ professional knowledge to ethically integrate artificial intelligence (AI)-based tools into education. Computers in Human Behavior, 138, 107468. https://doi.org/10.1016/j.chb.2022.107468
Chan, K., & Tang, W. (2024). Evaluating English teachers’ artificial intelligence readiness and training needs with a TPACK-based model. World Journal of English Language. https://doi.org/10.5430/wjel.v15n1p129
Chiu, T. (2023). The impact of generative AI (GenAI) on practices, policies and research direction in education: A case of ChatGPT and Midjourney. Interactive Learning Environments, 32, 6187–6203. https://doi.org/10.1080/10494820.2023.2253861
Chiu, T. K. F. (2024). Define, foster, and assess AI literacy in education: Implications for teaching and learning. Computers & Education: Artificial Intelligence, 5, 100223. https://doi.org/10.1016/j.caeai.2024.100223
Chiu, T. K. F., Leung, K. L., & Wong, Y. Y. (2025). Teacher AI competence and self-efficacy (TAICS): Development and validation. Education and Information Technologies. https://doi.org/10.1007/s10639-024-13094-z
Coşgun, G. (2025). Artificial intelligence literacy in assessment: Empowering pre-service teachers to design effective exam questions for language learning. British Educational Research Journal. https://doi.org/10.1002/berj.4177
DeVellis, R. F. (2016). Scale development: Theory and applications (4th ed.). Sage Publications.
Ding, L., Kim, S., & Allday, R. (2024). Development of an AI literacy assessment for non-technical individuals: What do teachers know? Contemporary Educational Technology. https://doi.org/10.30935/cedtech/14619
Du, H., Jin, S., & Liu, W. (2024). Exploring the effects of AI literacy on teacher learning and self-efficacy. Humanities and Social Sciences Communications, 11(1), 125. https://doi.org/10.1057/s41599-024-03101-6
Fabrigar, L. R., & Wegener, D. T. (2012). Exploratory factor analysis. Oxford University Press.
Fan, O., Zheng, L., & Jiao, P. (2022). Artificial intelligence in online higher education: A systematic review of empirical research from 2011 to 2020. Education and Information Technologies, 27, 7893–7925. https://doi.org/10.1007/s10639-022-10925-9
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.1177/002224378101800104
Giannakos, M., Azevedo, R., Brusilovsky, P., Cukurova, M., Dimitriadis, Y., Leo, D., Järvelä, S., Mavrikis, M., & Rienties, B. (2024). The promise and challenges of generative AI in education. Behaviour & Information Technology, 44, 2518–2544. https://doi.org/10.1080/0144929X.2024.2394886
Grassini, S. (2024). A psychometric validation of the PAILQ-6: Perceived Artificial Intelligence Literacy Questionnaire. In Proceedings of NordiCHI 2024. https://doi.org/10.1145/3679318.3685359
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis (8th ed.). Cengage.
Hava, K., Demir, S., & Aktepe, A. (2025). Digital proficiency and teachers’ AI-TPACK competencies: A structural equation approach. Education and Information Technologies. https://doi.org/10.1007/s10639-024-12939-x
Hornberger, M., Bewersdorff, A., & Nerdel, C. (2023). What do university students know about artificial intelligence? Development and validation of an AI literacy test. Computers & Education: Artificial Intelligence, 5, 100165. https://doi.org/10.1016/j.caeai.2023.100165
Hornberger, M., Bewersdorff, A., & Nerdel, C. (2025). Development and validation of a short AI literacy test (AILIT). Computers in Human Behavior: Artificial Humans. https://doi.org/10.1016/j.chbah.2025.100060
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
Jia, K., Sun, X., & Li, T. (2025). Developing a holistic AI literacy framework for children: A design-based research approach. In Proceedings of the ACM Interaction Design and Children Conference (IDC ’25). https://doi.org/10.1145/3727986
Joanna Briggs Institute (JBI). (2020). JBI manual for evidence synthesis. The University of Adelaide. https://doi.org/10.46658/JBIMES-20-01
Kamalov, F., Calonge, D., & Gurrib, I. (2023). New era of artificial intelligence in education: Towards a sustainable multifaceted revolution. Sustainability. https://doi.org/10.3390/su151612451
Kim, S. W., & Lee, Y. (2022). The artificial intelligence literacy scale for middle school students. Journal of the Korea Society of Computer and Information, 27(3), 225–238. https://doi.org/10.9708/jksci.2022.27.03.225
Kline, R. B. (2021). Principles and practice of structural equation modeling (5th ed.). Guilford Press.
Koch, M. J., Carolus, A., Wienrich, C., & Latoschik, M. E. (2024). Meta AI literacy scale: Further validation and development of a short version. Heliyon, 10(21), e39686. https://doi.org/10.1016/j.heliyon.2024.e39686
Kong, S. C., et al. (2023). Evaluating an AI literacy programme for secondary students: Impacts and implications. Education and Information Technologies, 28, 5031–5053. https://doi.org/10.1007/s10639-022-11408-7
Laupichler, M. C., Aster, A., Haverkamp, N., & Raupach, T. (2023). Development of the Scale for the Assessment of Non-Experts’ AI Literacy: An exploratory factor analysis. Computers in Human Behavior Reports, 12, 100338. https://doi.org/10.1016/j.chbr.2023.100338
Laupichler, M. C., Carolus, A., & Koch, M. J. (2023). Establishing a core set of AI literacy items through a Delphi study. Computers & Education: Artificial Intelligence, 4, 100126. https://doi.org/10.1016/j.caeai.2023.100126
Lee, S., & Park, G. (2024). Development and validation of ChatGPT literacy scale. Current Psychology, 43(21), 18992–19004. https://doi.org/10.1007/s12144-024-05723-0
Levac, D., Colquhoun, H., & O’Brien, K. K. (2010). Scoping studies: Advancing the methodology. Implementation Science, 5, 69. https://doi.org/10.1186/1748-5908-5-69
Lim, W., Gunasekara, A., Pallant, J., Pallant, J., & Pechenkina, E. (2023). Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators. The International Journal of Management Education. https://doi.org/10.1016/j.ijme.2023.100790
Lintner, T. (2024). A systematic review of AI literacy scales. NPJ Science of Learning, 9. https://doi.org/10.1038/s41539-024-00264-4
Lintner, T., Carolus, A., & Koch, M. J. (2024). A systematic review of AI literacy instruments: Dimensions, validity, and challenges. NPJ Science of Learning, 9, 1–18. https://doi.org/10.1038/s41539-024-00264-4
Liu, Y., Chen, X., Wang, L., & Zhang, H. (2025). Measuring generative AI literacy: Scale development and validation in workplace contexts. Computers & Education, 210, 105001. https://doi.org/10.1016/j.compedu.2025.105001
Long, D., & Magerko, B. (2020). What is AI literacy? Competencies and design considerations. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1–16). https://doi.org/10.1145/3313831.3376727
Lynn, M. R. (1986). Determination and quantification of content validity. Nursing Research, 35(6), 382–386. https://doi.org/10.1097/00006199-198611000-00017
Malhotra, N. K., & Dash, S. (2011). Marketing research: An applied orientation (6th ed.). Pearson Education.
Mishra, P., Warr, M., & Islam, R. (2023). TPACK in the age of ChatGPT and generative AI. Journal of Digital Learning in Teacher Education, 39, 235–251. https://doi.org/10.1080/21532974.2023.2247480
Monib, W., Qazi, A., Apong, R., Azizan, M., De Silva, L., & Yassin, H. (2024). Generative AI and future education: A review, theoretical validation, and authors’ perspective on challenges and solutions. PeerJ Computer Science, 10. https://doi.org/10.7717/peerj-cs.2105
Msambwa, M., Wen, Z., & Daniel, K. (2025). The impact of AI on the personal and collaborative learning environments in higher education. European Journal of Education. https://doi.org/10.1111/ejed.12909
Ng, D. T. K., Wu, J., Leung, J. K. L., Chu, S. K. W., Qiao, M. S., & Li, X. (2024). Design and validation of the AI Literacy Questionnaire: The affective, behavioral, cognitive, and ethical approach. British Journal of Educational Technology, 55(3), 915–936. https://doi.org/10.1111/bjet.13411
Ng, W. (2012). Can we teach digital natives digital literacy? Computers & Education, 59(3), 1065–1078. https://doi.org/10.1016/j.compedu.2012.04.016
Ning, Y., Lin, Y., & Chen, J. (2024). Teachers’ AI-TPACK: Integrating technological, pedagogical, and ethical competencies for AI use. Sustainability, 16(3), 978. https://doi.org/10.3390/su16030978
Park, H. (2025). Development of a teacher’s AI literacy scale based on meta-analysis. Korea University Institute of Educational Research. https://doi.org/10.24299/kier.2025.381.41
Peters, M. D. J., Godfrey, C. M., Khalil, H., McInerney, P., Parker, D., & Soares, C. B. (2015). Guidance for conducting systematic scoping reviews. International Journal of Evidence-Based Healthcare, 13(3), 141–146. https://doi.org/10.1097/XEB.0000000000000050
Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research, 8(2), 23–74. (No DOI available.)
Sperling, K., Stenberg, C., McGrath, C., Åkerfeldt, A., Heintz, F., & Stenliden, L. (2024). In search of artificial intelligence (AI) literacy in teacher education: A scoping review. Computers and Education Open. https://doi.org/10.1016/j.caeo.2024.100169
Tapalova, O., Zhiyenbayeva, N., & Gura, D. (2022). Artificial intelligence in education: AIEd for personalised learning pathways. Electronic Journal of e-Learning. https://doi.org/10.34190/ejel.20.5.2597
Tenberga, I., & Daniela, L. (2024). Artificial intelligence literacy competencies for teachers through self-assessment tools. Sustainability. https://doi.org/10.3390/su162310386
Walter, Y. (2024). Embracing the future of artificial intelligence in the classroom: The relevance of AI literacy, prompt engineering, and critical thinking in modern education. International Journal of Educational Technology in Higher Education, 21, 1–29. https://doi.org/10.1186/s41239-024-00448-3
Wang, H., Rau, P. L. P., & Yuan, Y. (2022). Measuring artificial intelligence literacy: Scale development and validation. Computers & Education, 187, 104515. https://doi.org/10.1016/j.compedu.2022.104515
Wang, X., Xu, X., Zhang, Y., Hao, S., & Jie, W. (2024). Exploring the impact of artificial intelligence application in personalized learning environments: Thematic analysis of undergraduates’ perceptions in China. Humanities and Social Sciences Communications. https://doi.org/10.1057/s41599-024-04168-x
Worthington, R. L., & Whittaker, T. A. (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34(6), 806–838. https://doi.org/10.1177/0011000006288127
Wu, D., & Zhang, J. (2025). Generative artificial intelligence in secondary education: Applications and effects on students’ innovation skills and digital literacy. PLOS ONE, 20. https://doi.org/10.1371/journal.pone.0323349
Yan, L., Sha, L., Zhao, L., Li, Y., Maldonado, R., Chen, G., Li, X., Jin, Y., & Gašević, D. (2023). Practical and ethical challenges of large language models in education: A systematic scoping review. British Journal of Educational Technology, 55, 90–112. https://doi.org/10.1111/bjet.13370
Yang, H., Lee, W., & Kim, J. (2025). Identification of key factors influencing teachers’ self-perceived AI literacy: An XGBoost and SHAP-based approach. Applied Sciences. https://doi.org/10.3390/app15084433
Yim, I. H. Y. (2024). Developing an intelligence-based AI literacy framework for primary education. Computers & Education: Artificial Intelligence, 6, 100319. https://doi.org/10.1016/j.caeai.2024.100319
Younis, B. (2025). The artificial intelligence literacy (AIL) scale for teachers: A tool for enhancing AI education. Journal of Digital Learning in Teacher Education, 41(1), 37–56. https://doi.org/10.1080/21532974.2024.2441682
Zhai, X. (2024). Transforming teachers’ roles and agencies in the era of generative AI: Perceptions, acceptance, knowledge, and practices. arXiv. https://doi.org/10.48550/arxiv.2410.03018
Zhang, H., Lee, I., Ali, S., DiPaola, D., & Breazeal, C. (2024). Developing and validating the AI Literacy Concept Inventory (AI-CI): An instrument to assess AI literacy among middle-school students. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-024-00398-x
Zhao, L., Wu, X., & Luo, H. (2022). Developing AI literacy for primary and middle school teachers in China: Based on a structural equation modeling analysis. Sustainability. https://doi.org/10.3390/su142114549
Downloads
Published
Issue
Section
License
Copyright (c) 2026 International Journal of Technology in Education
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Articles may be used for research, teaching, and private study purposes. Authors alone are responsible for the contents of their articles. The journal owns the copyright of the articles. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of the research material.
The author(s) of a manuscript agree that if the manuscript is accepted for publication in the International Journal of Technology in Education (IJTE), the published article will be copyrighted using a Creative Commons “Attribution 4.0 International” license. This license allows others to freely copy, distribute, and display the copyrighted work, and derivative works based upon it, under certain specified conditions.
Authors are responsible for obtaining written permission to include any images or artwork for which they do not hold copyright in their articles, or to adapt any such images or artwork for inclusion in their articles. The copyright holder must be made explicitly aware that the image(s) or artwork will be made freely available online as part of the article under a Creative Commons “Attribution 4.0 International” license.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
