Generative Agents for Pre-Assessment Question Evaluation

Date: 21 July 2026, Tuesday

Time: 1pm - 2pm

 Synopsis

Ensuring the quality of multiple-choice questions remains a challenge in higher education, as traditional post-hoc psychometric analyses often identify flaws only after students have been negatively impacted. This talk proposes the use of generative agents to simulate student performance to validate items before delivery. In addition, a teacher role-playing prompt is introduced to mitigate high-accuracy bias by prompting the model to anticipate student errors from an educator's perspective. Empirical findings provide evidence of the potential of this approach and suggest a decision matrix to guide educators in selecting the optimal LLM configuration for assessment evaluation.

Keynote Speaker
A/P Lo Siaw Ling (SMU)

Siaw Ling is an Associate Professor of Information Systems (Education) at the School of Computing and Information Systems, Singapore Management University (SMU). She is one of the SMU’s Education Research (ER) Fellows and her primary research interests focus on the application of generative AI and large language models in educational contexts, including AI Agent-based assessment evaluation, the use of AI for instructional and pedagogical approach, and personalized learning. She has received multiple grants from institutions such as ST Engineering and the Singapore Ministry of Education.