Reimagine essay-based assessment in the wake of ChatGPT: insights from corpus-based analyses of ChatGPT - assisted writing, university student writing, and IELTS writing and human feedback on them

This project investigates how postgraduate students use ChatGPT to assist with their written coursework and its impact on writing quality. It will reveal the different word choices and critical thinking skills in ChatGPT-assisted, student-only, and IELTS test-taker writing, built upon our previous funded project on different language demands of university student writing and IELTS writing tasks. We employed corpus analysis of writing scripts, chat histories, and lecturers' feedback on writing with/without ChatGPT. We also held focus groups with students regarding their experience and suggestions for training in GenAI use. The findings will inform essay-based assessment and benefit stakeholders including language proficiency test designers, university subject tutors, learners, language tutors, and policymakers on academic integrity and exam fraud in the context of generative AI.

Project aims

Help educators and learners understand the impact of LLMs on writing processes and products, and develop their GenAI literacies to inform teaching, learning, and assessment and future professional literacy in a GenAI-empowered world. 

Impact

Our presentations at conferences have sparked discussions on better understanding and more effective use of GenAI in education based on research evidence, at and beyond conferences We anticipate that our forthcoming publications will offer further valuable insights into this evolving field.

Publications and outputs

BERA blog: Reimagine essay-based assessment in higher education in the era of generative artificial intelligence | BERA

Reimagining assessment and language education in higher education in an AI-powered era, in Open crowdsourced collection: Towards AI Literacy: 101+ Creative and Critical Practices, Perspectives and Purposes

Conferences

• 30th May – 2nd June 2024: invited keynote speaker at ELT (English language teaching) Research Conference, Ankara, Turkey

• 20th-21st June 2024: mini-keynote speaker at the Assessment in Higher Education annual conference, Manchester

• 24th-25th June 2024: presentation at the AI Summit at the University of Leeds, Leeds

• 9th – 12th September 2024: presentation at BERA (British Educational Research Association)/WERA (World Educational Research Association) annual conference, Manchester