Tophel, Amir, Chen, Liuxin, Hettiyadura, Umidu, and Kodikara, Jayantha (2025) Towards an AI tutor for undergraduate geotechnical engineering: a comparative study of evaluating the efficiency of large language model application programming interfaces. Discover Computing, 28. 76.

Preview

PDF (Published Version) - Published Version Available under License Creative Commons Attribution. Download (1MB) | Preview

Abstract

This study investigates the efficiency of large language model (LLM) application programming interfaces (APIs)—specifically GPT-4 and Llama-3—as AI tutors for undergraduate Geotechnical Engineering education. As educational needs in specialised fields like Geotechnical Engineering become increasingly complex, innovative teaching tools that provide personalised learning experiences are essential. Unlike previous studies on AI-driven education, our research uniquely focuses on assessing the role of retrieval-augmented generation (RAG) in improving the accuracy of LLM-generated solutions to Geotechnical problems. A dataset of 391 questions from the related textbook written by Das and Sobhan (Das B, Sobhan K. Principles of Geotechnical engineering, Eight Edition. In: Cengage Learning. 2014) was used for evaluation, with solutions sourced from the textbook’s manual. Performance benchmarking focused on 20 challenging questions previously identified by Chen et al. (Chen et al. in Geotechnics 4:470–498, 2024) as problematic for GPT-4 in Zero Shot tasks. GPT-4 with API support demonstrated superior accuracy, achieving accuracy rates of 95% at a temperature setting of 0.1, 82.5% at 0.5, and 60% at 1. In comparison, Llama-3 achieved an accuracy of 25% in Zero Shot tasks and 45% with API support at a temperature setting of 0.1. The findings highlight GPT-4’s potential as an AI tutor for Geotechnical Engineering education while demonstrating the need for domain-specific optimisation and advanced formula integration techniques. This study contributes to the ongoing discourse on AI in education by providing empirical evidence supporting the deployment of LLMs as personalised, adaptive teaching aids in engineering disciplines. Future work should explore optimised formula integration strategies, expanded domain knowledge bases, and long-term student learning outcomes.

Item ID:	85636
Item Type:	Article (Research - C1)
ISSN:	2948-2992
Copyright Information:	© The Author(s) 2025. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Date Deposited:	28 May 2025 02:46
FoR Codes:	39 EDUCATION > 3904 Specialist studies in education > 390499 Specialist studies in education not elsewhere classified @ 50% 39 EDUCATION > 3999 Other Education > 399999 Other education not elsewhere classified @ 50%
SEO Codes:	16 EDUCATION AND TRAINING > 1603 Teaching and curriculum > 160399 Teaching and curriculum not elsewhere classified @ 50% 16 EDUCATION AND TRAINING > 1601 Learner and learning > 160102 Higher education @ 50%
Downloads:	Total: 1 Last 12 Months: 1
	More Statistics