Design and implementation of large language model retrieval-augmented generation-based computer network experiment course QA system
[Objective]With the development of online course learning and online experimental teaching,numerous challenges to the online-offline blended teaching mode in computer network experiment courses have emerged.On one hand,online experiments meet the demand for students to conduct experiments anytime and anywhere;on the other hand,these experiments require immediate support and clarification for students.There are two main obstacles to overcome in this teaching mode.First,a lot of frequently asked questions related to common experimental issues are often raised,which leads to inefficient and limited human responses.Second,students often struggle to quickly locate experimental guidance within the abundant and complex reference materials,resulting in untimely solutions and delayed experimental progress,ultimately impacting the effectiveness of course learning.[Methods]A computer network experiment course QA system based on retrieval-augmented generation using a large language model is designed,comprising a large language model,an external knowledge base,and a normative detection system.The system utilizes the versatility of large language models to provide highly customized responses with relevant context from the external knowledge base,which consists of over 200,000 words from experiment-related materials,including textbooks,frequently asked questions,and chat history from the course WeChat group.The multilevel normative detection system includes word-level keyword retrieval and semantic-level large language model assessment,which rejects the generation of responses to problems not related to the course,thereby ensuring the system's robustness.To generate responses for specific questions,the system initially applies a large-language-model-based query enhancement to the original query,rewriting it into a similar query,and answers both queries with the large language model directly without any external knowledge.These query-answer pairs retrieve context from the external knowledge base using both BM25 and vector retrieval,providing query-related context that helps offer guidance.The context is then sent to a prompt template with chat history to generate a chat message,which is finally fed into the large language model to produce a customized answer.In addition,a WeChat mini-program is developed for information display and user interaction to enhance the system's usability,providing user feedback for further performance improvement in the future.[Results]The system has been in use for two semesters since October 2023.In-class testing results indicate that the system can accurately answer students'questions,including those that can be directly answered with the textbook and those that require further inference,showcasing the system's in-domain and out-of-domain capabilities.Tests on unreasonable or unsafe content attempts,to which the system refuses to respond,also demonstrate the system's robustness.During the computer network experiment course in 2024,the system successfully provided question-answering services for over 500 students within a month,averaging 46 students per course day,resolving approximately 72.4%of problems,and alleviating the teacher's workload during class.[Conclusions]This system,with its wide application range,fast response speed,and high-performance responses,represents a successful integration of large language models into the computer network experiment course.It significantly enhances the student experience and improves course quality,providing high-precision,all-day-long intelligent responses.This represents an effective teaching practice combining artificial intelligence and education.
large language modelcourse question answering systemretrieval-augmented generation
万方数据
维普
