AI-ALOE External Advisory Board Meeting
May 16, 2025
Our lab attended the AI-ALOE external advisory board (EAB) meeting at the Coda Building, Georgia Tech on May 16. There, our members had multiple presentations as well as posters.
Our first presentation was during the Impact on Learning session, where our AI-ALOE graduate associate, Yoojin Bae shared extending SMART into workforce education. She shared about our deployments expanding to nursing and it related courses, and the results from our case study from experimental setting English courses.
Dr. Min Kyu Kim led the Contributions to Theories of Learning and of AI Agents session as the chair, introducing speakers and facilitating the following discussion. In the same section, our graduate associate, Jinho Kim, also presented on A Theory of Learning, sharing our SMART team’s approach in revisiting and reinterpreting CoI, ICAP, and Self Determination Theory in SMART.
The final presentation was during the Design Heuristics for Building AI Agents for Learning and Education section, where Jinho Kim introduced our top down approach from theory to design guidelines using a whole person approach.
During the poster session, we also had four posters to share.
Automated Writing Evaluation in English and Foreign Language Argumentative Writing: Opportunities, Challenges, and Future Directions
Golnoush Haddadian, Min Kyu Kim, and Nooshin Haddadian
This systematic review investigates the use of Automated Writing Evaluation (AWE) systems in supporting argumentative writing among English as a Foreign Language (EFL) learners. Drawing from publications between 2014 and 2023, the study synthesizes findings from empirical research to identify key features, design principles, and pedagogical implications of AWE tools in EFL contexts. Through rigorous coding and thematic analysis, the review uncovers emerging opportunities, persistent challenges, and gaps in the current literature. Findings underscore the need to align AWE feedback with learners’ higher-level argumentative writing needs (e.g., claim development, evidence integration) rather than focusing primarily on surface-level issues (e.g., grammar, spelling), by embedding personalized learning opportunities within the writing process. The study concludes with recommendations for future research and design guidelines to inform the development of future AWE tools that effectively support argumentative writing.
Design-Based Research for Scenario-Based, Generative AI-Augmented Simulation in Nursing Education
Jinho Kim, Maria Feliciana Galindo-Parra, Seora Kim, Yoojin Bae, Terry Morris Hendry, Sujay Saphire Galen, Min Kyu Kim
Simulation-based learning is vital in nursing education but faces scalability and authenticity challenges. This study explores using generative AI as a role-playing agent in nursing simulations. Using a design-based research approach, we developed an evaluation framework and question set to evaluate AI model performance. Findings highlight accurate role-based dialogue but limitations in common-sense reasoning. Recommendations include clearer evaluation criteria, diverse question sets to comprehensively evaluate ambiguity handling, and refining scenarios iteratively for consistency and clarity.
Cheating or Adapting? Plagiarism Behaviors in AI-Augmented Physics Course
Hyunkyu Han, Golnoush Haddadian, Mohamed Shameer Abdeen, Min Kyu Kim
The integration of artificial intelligence (AI) in education poses challenges and opportunities for academic integrity. This study examines the intersection of plagiarism and learning behaviors in an AI-augmented introductory physics course. Using the Student Mental Model Analyzer for Research and Teaching (SMART), 35 undergraduate students submitted 101 summary revisions, which were analyzed for local and global plagiarism using Copyleaks and Turnitin. Findings reveal that high Global Plagiarism Index (GI) scores, often linked to misconduct, can also indicate meaningful learning behaviors such as conceptual engagement and iterative refinement. Bayesian Linear Mixed-Effects models show GI scores positively correlated with concept similarity and paraphrasing efforts, while multiple revisions were associated with reduced reliance on external sources. Additionally, a case review illustrates how high GI scores may reflect learning progress rather than unethical practices. The results suggest a potential re-interpretation of typical plagiarism indices within the context of AI-augmented summary writing assignments.
AI-Scaffolded Pre-Classroom Learning in an Introductory Undergraduate Physics
Seora Kim, Jinho Kim, Hyunkyu Han, Mohamed Shameer Abdeen, Min Kyu Kim
This study examines the impact of AI-scaffolded formative feedback on pre-class learning and its relationship with classroom task scores, STEM interest, self-efficacy, and motivation in an introductory undergraduate physics course, focusing on the first assignment. Students revised summaries of assigned materials using AI-generated feedback and their actions were conceptualized based on the theory of engagement. Data from 55 participants were analyzed through descriptive statistics, independent T-test, and linear mixed-effects models. We found differences in concept learning effectiveness between students who demonstrated engaged learning behaviors and those who did not. However, no significant relationships emerged between concept learning and other learning outcomes or affective factors. These findings highlight the importance of learner engagement in conceptual learning in the early stage of tool usage. Further research is needed to explore how more engagement in AI-supported learning relates to students’ academic achievement and affective factors.
Overall, the EAB meeting was a chance for us to share our contributions with AI-ALOE, have discussions, and get valuable feedback.
For more information about the External Advisory Board Meeting, please visit the following link: AI-ALOE Hosts External Advisory Board Meeting to Showcase Year-Four Achievements
Lia Haddadian has been awarded the AIVO Fellowship and selected for the QuEST Program 2025
May 15, 2025
Lia Haddadian, our graduate associate, has been awarded the AIVO Fellowship and selected for the QuEST Program 2025.
The AI Institutes Virtual Organization (AIVO), managed by UC Davis, offers the Graduate Research Fellowship as part of the AI4Ed Summer 2025 Program. Lia will be engaged in this 12-week program from May 7 to August 24, contributing 40 hours per week to AIVO’s AI4Ed initiatives.
In addition, Lia has been selected to participate in the Quantitative Evidence Synthesis Training (QuEST) Program 2025, a highly competitive and fully funded training hosted by the American Institutes for Research® (AIR). This intensive program will be held from August 4–8, 2025, at AIR’s Chicago office and brings together emerging scholars from across the country for hands-on training in advanced methods of quantitative evidence synthesis. Learn more about the QuEST Program here: https://www.air.org/QuEST-for-p3/in-person-training
Yoojin Featured as a Graduate Researcher in the AI-ALOE Spring Newsletter
May 6, 2025
We are delighted to share that Yoojin Bae, our graduate associate, has been recognized in the AI-ALOE Spring Newsletter!
Featured in the Student Highlights section, Yoojin was acknowledged for her outstanding achievements and contributions to the AI-ALOE project. Her research lies at the intersection of education and computer science, with a focus on leveraging artificial intelligence to support adaptive learning environments. Currently, she is working with multimodal large language models to analyze video data in educational contexts.
Be sure to check out the full newsletter here: AI-ALOE Newsletter
Research Collaboration: AI-Powered Nursing Education
May 1, 2025
Since Fall 2024, our lab has been collaborating with the School of Nursing at Georgia State University (https://lewis.gsu.edu/nursing/) to integrate AI technologies into both undergraduate and graduate nursing education. Our efforts focus on two key areas: enhancing teaching and learning by embedding AI agents into nursing courses, and developing advanced AI-driven simulations for the Clinical Skills and Simulation Center (https://lewis.gsu.edu/sim-center/) within the Byrdine F. Lewis College of Nursing and Health Professions.
Regarding AI-powered teaching and learning, we have been implementing the SMART system across various courses in the School of Nursing at Georgia State University. We began by applying SMART to Medical Surgical I, an undergraduate course required for third-year nursing students that integrates theoretical instruction with clinical practice. A refined integration involving four SMART assignments was implemented and tested in Spring 2025. Additionally, this Spring semester, we extended the AI deployment to Epidemiology, a graduate-level course focused on conceptual and theoretical content.
Looking ahead to the summer, we plan to implement SMART in both undergraduate and graduate Pathophysiology courses. Our goal is to continue identifying effective strategies for AI integration that support instructors’ teaching practices and enhance students’ learning and engagement. Notably, we will introduce multiple AI tools within the same instructional setting. For instance, we aim to create a confluence of SMART—a knowledge-based AI system—and Jill Watson, a GenAI-powered conversational agent. This dual deployment will allow stakeholders to benefit from SMART’s assignment-specific assessment and feedback features, alongside Jill Watson’s interactive, conversation-based learner support. Ultimately, we hope to explore how these tools can be combined to create a task-oriented, interactive, and engaging concept learning platform that advances nursing education.
Additionally, we are developing an AI-augmented simulation-based learning platform, tentatively titled The AI-Powered Nursing Simulation Framework (AI-NSF). This effort is a collaboration with the Clinical Skills and Simulation Center (https://lewis.gsu.edu/sim-center/) within the Byrdine F. Lewis College of Nursing and Health Professions and the Future of Learning Lab at Cornell University (https://learning.cis.cornell.edu/). The AI-NSF aims to establish a flexible framework for supporting simulation-based learning through a conversational AI-augmented simulator.
The platform will provide a hybrid simulation environment that connects an online simulator with physical simulation rooms. Within this setup, learners can engage with either a virtual or manikin patient across various clinical scenarios, allowing them to demonstrate and refine their critical thinking, communication, and clinical skills in a realistic, adaptive, and authentic learning context.
