Identifying Whether a Short Essay was written by a University Student or ChatGPT
Abstract
This study seeks to clarify whether teachers are able to distinguish between essays written by English L2 students or generated by ChatGPT. 47 instructors who hold experience teaching English to native speakers of Japanese in universities or other higher education institutions were tested on whether they could identify between human written essays and ChatGPT generated essays. The ICNALE written corpus (Ishikawa, 2013) was used to find and randomly select the essays of four Japanese university students’ written work who studied English at roughly CEFR A2 level. The AI chatbot, ChatGPT, was used to generate four essays utilizing prompts which directed the chatbot to mimic grammar mistakes common to nonnative speakers of English. Teachers were requested to identify which of the eight essays they believed to be human written or ChatGPT generated. On average, the teachers were able to identify 54.25% of items accurately. This result is slightly better than random chance, and implies that most teachers cannot make an accurate assessment on a ChatGPT generated essay when ChatGPT is prompted to make grammar mistakes.
Keywords
Full Text:
PDFReferences
Saarna, C. (2024). Identifying whether a short essay was written by a university student or ChatGPT. International Journal of Technology in Education (IJTE), 7(3), 611-633. https://doi.org/10.46328/ijte.773
DOI: https://doi.org/10.46328/ijte.773
Refbacks
- There are currently no refbacks.
![Creative Commons License](http://licensebuttons.net/l/by-nc-sa/4.0/88x31.png)
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Abstracting/Indexing
International Journal of Technology in Education (IJTE) - ISSN:2689-2758
affiliated with
International Society for Technology, Education and Science (ISTES)
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.