Interacting with educational chatbots: A systematic review Education and Information Technologies

Specifically, chatbots have demonstrated significant enhancements in learning achievement, explicit reasoning, and knowledge retention. The integration of chatbots in education offers benefits such as immediate assistance, quick access to information, enhanced learning outcomes, and improved educational experiences. However, there have been contradictory findings related to critical thinking, learning engagement, and motivation.

Additionally, AICs today can also incorporate emerging technologies like AR and VR, and gamification elements, to enhance learner motivation and engagement (Kim et al., 2019). The first one delves into the effects of AICs on language competence and skills. These studies showed how AICs can manage personal queries, correct language mistakes, and offer linguistic support in real-time. Chatbot technology has evolved rapidly over the last 60 years, partly thanks to modern advances in Natural Language Processing (NLP) and Machine Learning (ML) and the availability of Large Language Models (LLMs). Today chatbots can understand natural language, respond to user input, and provide feedback in the form of text or audio (text-based and voice-enabled).

However, it is essential to address concerns regarding the irrational use of technology and the challenges that education systems encounter while striving to harness its capacity and make the best use of it. The traditional education system faces several issues, including overcrowded classrooms, a lack of personalized attention for students, varying learning paces and styles, and the struggle to keep up with the fast-paced evolution of technology and information. As the educational landscape continues to evolve, the rise of AI-powered chatbots emerges as a promising solution to effectively address some of these issues. Some educational institutions are increasingly turning to AI-powered chatbots, recognizing their relevance, while others are more cautious and do not rush to adopt them in modern educational settings. Consequently, a substantial body of academic literature is dedicated to investigating the role of AI chatbots in education, their potential benefits, and threats. Chatbots can help educational institutions in data collection and analysis in various ways.

After defining the criteria, our search query was performed in the selected databases to begin the inclusion and exclusion process. Initially, the total of studies resulting from the databases was 1208 studies. The metadata of the studies containing; title, abstract, type of article (conference, journal, short paper), language, and keywords were extracted in a file format (e.g., bib file format). Subsequently, it benefits of chatbots in education was imported into the Rayyan tool Footnote 6, which allowed for reviewing, including, excluding, and filtering the articles collaboratively by the authors. With its human-like writing abilities and OpenAI’s other recent release, DALL-E 2, it generates images on demand and uses large language models trained on huge amounts of data. The same is true of rivals such as Claude from Anthropic and Bard from Google.

An example of this is the chatbot in (Sandoval, 2018) that answers general questions about a course, such as an exam date or office hours. After the first, second, and third filters, we identified 505 candidate publications. We continued our filtering process by reading the candidate publications’ full texts resulting in 74 publications that were used for our review. Compared to 3.619 initial database results, the proportion of relevant publications is therefore about 2.0%. In the case of Google Scholar, the number of results sorted by relevance per query was limited to 300, as this database also delivers many less relevant works. The value was determined by looking at the search results in detail using several queries to exclude as few relevant works as possible.

The study by Pérez et al. (2020) reviewed the existing types of educational chatbots and the learning results expected from them. Smutny and Schreiberova (2020) examined chatbots as a learning aid for Facebook Messenger. Thomas (2020) discussed the benefits of educational chatbots for learners and educators, showing that the chatbots are successful educational tools, and their benefits outweigh the shortcomings and offer a more effective educational experience. Okonkwo and Ade-Ibijola (2021) analyzed the main benefits and challenges of implementing chatbots in an educational setting.

I believe the most powerful learning moments happen beyond the walls of the classroom and outside of the time boxes of our course schedules. Authentic learning happens when a person is trying to do or figure out something that they care about — much more so than the problem sets or design challenges that we give them as part of their coursework. It’s in those moments that learners could benefit from a timely piece of advice or feedback, or a suggested “move” or method to try. So I’m currently working on what I call a “cobot” — a hybrid between a rule-based and an NLP bot chatbot — that can collaborate with humans when they need it and as they pursue their own goals.

Deng and Yu (2023) found that chatbots had a significant and positive influence on numerous learning-related aspects but they do not significantly improve motivation among students. Contrary, Okonkwo and Ade-Ibijola (Okonkwo & Ade-Ibijola, 2021), as well as (Wollny et al., 2021) find that using chatbots increases students’ motivation. Much more than a customer service add-on, chatbots in education are revolutionizing communication channels, streamlining inquiries and personalizing the learning experience for users. For institutions already familiar with the conversational sales and support landscapes, harnessing the potential of chatbots could catapult their educational services to the next level.

Criteria were determined to ensure the studies chosen are relevant to the research question (content, timeline) and maintain a certain level of quality (literature type) and consistency (language, subject area). It is expected that as these models become more widely available for commercial use, research on the benefits of their use will also increase. Because chatbots using LLMs have vastly more capabilities than their traditional counterparts, it is expected that there are additional benefits not currently identified in the literature. Therefore, this section outlines the benefits of traditional chatbot use in education. AI aids researchers in developing systems that can collect student feedback by measuring how much students are able to understand the study material and be attentive during a study session. The way AI technology is booming in every sphere of life, the day when quality education will be more easily accessible is not far.

Associated Data

As another example, the SimStudent chatbot is a teachable agent that students can teach (Matsuda et al., 2013). In terms of the medium of interaction, chatbots can be text-based, voice-based, and embodied. Text-based agents allow users to interact by simply typing via a keyboard, whereas voice-based agents allow talking via a mic. Voice-based chatbots are more accessible to older adults and some special-need people (Brewer et al., 2018). An embodied chatbot has a physical body, usually in the form of a human, or a cartoon animal (Serenko et al., 2007), allowing them to exhibit facial expressions and emotions. Initial use of chatbots can be challenging, and some students may not understand how to prompt them correctly to achieve the desired result (Kaur et al., 2021).

By grouping the resulting relevant publications according to their date of publication, it is apparent that chatbots in education are currently in a phase of increased attention. The release distribution shows slightly lower publication numbers in the current than in the previous year (Figure 6), which could be attributed to a time lag between the actual publication of manuscripts and their dissemination in databases. Educational Technologies enable distance learning models and provide students with the opportunity to learn at their own pace. Some studies mentioned limitations such as inadequate or insufficient dataset training, lack of user-centered design, students losing interest in the chatbot over time, and some distractions. The results show that the chatbots were proposed in various areas, including mainly computer science, language, general education, and a few other fields such as engineering and mathematics.

In an experiment in which the chatbot is asked to design a trendy women’s shoe, it offers several possible alternatives and then, when asked, serially and skillfully refines the design. The first article describes how a new AI model, Pangu-Weather, can predict worldwide weekly weather patterns much more rapidly than traditional forecasting methods but with comparable accuracy. The second demonstrates how a deep-learning algorithm was able to predict extreme rainfall more accurately and more quickly than other methods. The authors would like to express their gratitude to all the college students from both institutions for their invaluable participation in this project. Chatbots must be designed with strict privacy and security controls to safeguard sensitive information. A strategic plan is essential to organize and present this data through the chatbot without overwhelming the user.

This assessment was aligned with the CHISM scale, which was completed in a post-survey. A minimum interaction of three hours per week with each AIC, or 48 h over a month across all AICs, was requested from each participant. Firstly, it aims to investigate the current knowledge and opinions of language teacher candidates regarding App-Integrated Chatbots (AICs). Secondly, it seeks to measure their level of satisfaction with four specific AICs after a 1-month intervention. Lastly, it aims to evaluate their perspectives on the potential advantages and drawbacks of AICs in language learning as future educators.

App-Integrated Chatbots (AICs) in language learning

When it comes to education-related applications of AI, the media have paid the most attention to applications like students getting chatbots to compose their essays and term papers. Concerning the educational setting, Spanish participants interacted more frequently with all four AICs compared to Czech students. The SD values show a similar level of variation in the weekly interaction hours across all four AICs for both Spanish and Czech participants, suggesting a comparable spread of interaction frequencies within each group. Look for features such as natural language processing, integration capabilities with school databases, scalability, and the ability to handle a wide range of queries.

We have extensive information on chatbot-related topics, such as how to automate contact information collection and how to maximize customer service potential. By understanding and leveraging these advantages, businesses can enhance their interactions with customers, fostering stronger relationships and driving growth. This proactive engagement can lead to higher enrollment rates and improved student satisfaction.

• Based on my initial explorations of the current capabilities and limitations of both types of chatbots, I opted for scripted chatbots.
• Administrators can take up other complex, time-consuming tasks that need human attention.
• Only two studies used chatbots as teachable agents, and two studies used them as motivational agents.

The results of the evaluation studies (Table 12) point to various findings such as increased motivation, learning, task completeness, and high subjective satisfaction and engagement. One of them presented in (D’mello & Graesser, 2013) asks the students a question, then waits for the student to write an answer. Then the motivational agent reacts to the answer with varying emotions, including empathy and approval, to motivate students. Similarly, the chatbot in (Schouten et al., 2017) shows various reactionary emotions and motivates students with encouraging phrases such as “you have already achieved a lot today”. In general, most desktop-based chatbots were built in or before 2013, probably because desktop-based systems are cumbersome to modern users as they must be downloaded and installed, need frequent updates, and are dependent on operating systems.

Other chatbots used experiential learning (13.88%), social dialog (11.11%), collaborative learning (11.11%), affective learning (5.55%), learning by teaching (5.55%), and scaffolding (2.77%). In terms of the interaction style, the vast majority of the chatbots used a chatbot-driven style, with about half of the chatbots using a flow-based with a predetermined specific learning path, and 36.11% of the chatbots using an intent-based approach. Only four chatbots (11.11%) used a user-driven style where the user was in control of the conversation. A user-driven interaction was mainly utilized for chatbots teaching a foreign language. A notable example of a study using questionnaires is ‘Rexy,’ a configurable educational chatbot discussed in (Benedetto & Cremonesi, 2019).

These FAQ-type chatbots are commonly used for automating customer service processes like booking a car service appointment or receiving help from a phone service provider. Alternatively, ChatGPT is powered by the large language models (LLMs), GPT-3.5, and GPT-4 (OpenAI, 2023b). LLMs are AI models trained using large quantities of text, generating comprehensive human-like text, unlike previous chatbot iterations (Birhane et al., 2023). Renowned brands such as Duolingo and Mondly are employing these AI bots creatively, enhancing learner engagement and facilitating faster comprehension of concepts. These educational chatbots play a significant role in revolutionizing the learning experience and communication within the education sector. In the mentoring role (Mentoring), chatbot actions deal with the student’s personal development.

Providing timely, personalized, and effective support through chatbots can enhance an online school’s reputation, leading to positive word-of-mouth and increased enrollment. Chatbots helps identify and address potential issues before they escalate, leading to increased student retention. According to a study by EducationDIVE, 81% of students who leave an online course do so because they feel unsupported. This leads to improved customer satisfaction, as users can access help whenever they need it. Chatbots can initiate conversations with website visitors, increasing user engagement and retention rates. In the images below you can see two sections of the flowchart of one of my chatbots.

Answer to Research Questions

In terms of the evaluation methods used to establish the validity of the articles, two related studies (Pérez et al., 2020; Smutny & Schreiberova, 2020) discussed the evaluation methods in some detail. However, this study contributes more comprehensive evaluation details such as the number of participants, statistical values, findings, etc. Qualitative data, obtained from in-class discussions and assessment reports submitted through the Moodle platform, were systematically coded and categorized using QDA Miner. The goal was to analyse and identify the main benefits and drawbacks of each AIC as perceived by teacher candidates.

After coding a larger set of publications, it became clear that the code for service-oriented chatbots needed to be further distinguished. This was because it summarized e.g. automation activities with activities related to self-regulated learning and thus could not be distinguished sharply enough from the learning role. After refining the code set in the next iteration into a learning role, an assistance role, and a mentoring role, it was then possible to ensure the separation of the individual codes. Research in this area has recently focused on chatbot technology, a subtype of dialog systems, as several technological platforms have matured and led to applications in various domains. Chatbots incorporate generic language models extracted from large parts of the Internet and enable feedback by limiting themselves to text or voice interfaces. For this reason, they have also been proposed and researched for a variety of applications in education (Winkler and Soellner, 2018).

In other studies, the teaching agent emulates a teacher conducting a formative assessment by evaluating students’ knowledge with multiple-choice questions (Rodrigo et al., 2012; Griol et al., 2014; Mellado-Silva et al., 2020; Wambsganss et al., 2020). Six (16.66%) articles presented educational chatbots that exclusively operate on a mobile platform (e.g., phone, tablet). Examples include Rexy (Benedetto & Cremonesi, 2019), which helps students enroll in courses, shows exam results, and gives feedback.

Subsequently, the assessment of specific topics is presented where the user is expected to fill out values, and the chatbot responds with feedback. The level of the assessment becomes more challenging as the student makes progress. A slightly different interaction is explained in (Winkler et al., 2020), where the chatbot challenges the students with a question. If they answer incorrectly, they are explained why the answer is incorrect and then get asked a scaffolding question. Expanding on the necessity for improved customization in AICs, the integration of different features can be proposed to enhance chatbot-human personalization (Belda-Medina et al., 2022).

Chatbots have been utilized in education as conversational pedagogical agents since the early 1970s (Laurillard, 2013). Pedagogical agents, also known as intelligent tutoring systems, are virtual characters that guide users in learning environments (Seel, 2011). They are characterized by engaging learners in a dialog-based conversation using AI (Gulz et al., 2011).

The landscape of mobile-application language learning (MALL) has been significantly reshaped in recent years with the incorporation of AICs (Pham et al., 2018). This innovative approach to mobile learning has been positively received by both students and teachers. For example, Chen et al. (2020) highlighted the effectiveness of AICs for Chinese vocabulary learning by comparing chatbot-based tutoring with traditional classroom settings.

When you think of advancements in technology, edtech might not be the first thing that pops into your head. But during the COVID-19 pandemic, edtech became a true lifeline for education by making it accessible and easy to use despite there being numerous physical restrictions. You can foun additiona information about ai customer service and artificial intelligence and NLP. Today, technologies like conversational AI and natural language processing (NLP) continue to help educators and students world over teach and learn better.

For (Goal 5), we want to extend the work of (Winkler and Soellner, 2018) and (Pérez et al., 2020) regarding Application Clusters (AC) and map applications by further investigating specific learning domains in which chatbots have been studied. Addressing these gaps in the existing literature would significantly benefit the field of education. Firstly, further research on the impacts of integrating chatbots can shed light on their Chat PG long-term sustainability and how their advantages persist over time. This knowledge is crucial for educators and policymakers to make informed decisions about the continued integration of chatbots into educational systems. Secondly, understanding how different student characteristics interact with chatbot technology can help tailor educational interventions to individual needs, potentially optimizing the learning experience.

Believe it or not, the education sector is now among the top users of chatbots and other smart AI tools like ChatGPT. Like all of us, teachers are bound by time and space — but can educational technology offer new ways to make a teacher’s presence and knowledge available to learners? Stanford d.school’s Leticia Britos Cavagnaro is pioneering efforts to extend interactive resources beyond the classroom. She recently has developed the “d.bot,” which takes a software feature that many of us know through our experiences as customers — the chatbot — and deploys it instead as a tool for teaching and learning. Jenny Robinson, a member of the Stanford Digital Education team, discussed with Britos Cavagnaro what led to her innovation, how it’s working and what she sees as its future.

• In our study, the primary focus was on evaluating language teacher candidates’ perceptions of AICs in language learning, rather than assessing language learning outcomes.
• Chatbots emerge as crucial tools for efficiently managing inquiries and standing out in the competitive field”, he added.
• The value was determined by looking at the search results in detail using several queries to exclude as few relevant works as possible.
• To improve the clarity of the discussion section, we employed Large Language Model (LLM) for stylistic suggestions.

Similar feedback functions are incorporated on a smaller scale into software applications such as Grammarly, Microsoft Word, and Google Docs. Utilizing chatbots, students can make their statements more clear and concise (Cunningham-Nelson et al., 2019) and receive assistance solving difficult problems (Kaur et al., 2021). In one study, students used chatbots to provide continuous feedback on their argumentative essays to assist with writing (Guo et al., 2022). Typically, this feedback is received after peer review or first draft submissions rather than concurrently within the writing process.

Chatterjee and Bhattacharjee (2020), Merelo et al. (2022), and Kim and Kim (2022) noted that teachers will be more likely to adopt chatbots if there is continued support and professional development provided by their organizations for chatbot use. Funding for technology support should be taken into consideration by the administration when deciding whether to adopt chatbot technologies in education. Teachers and students should be provided initial training to increase PEU (Chatterjee & Bhattacharjee, 2020) and have continued support if they require assistance when using chatbots. Historically, educators viewed their interactions with chatbots negatively, citing that the responses from chatbots were rigid and unoriginal (Kim & Kim, 2022), only capable of answering simple questions (Cunningham-Nelson et al., 2019). This is likely because these traditional chatbots, or frequently asked questions (FAQ)-type chatbots (Cunningham-Nelson et al., 2019; Merelo et al., 2022), do not utilize AI and are trained to respond using predetermined criteria (Smutny & Schreiberova, 2020).

In contrast, NLP chatbots, which use Artificial Intelligence, make sense of what the person writes and respond accordingly (NLP stands for Natural Language Processing). Based on my initial explorations of the current capabilities and limitations of both types of chatbots, I opted for scripted chatbots. Studies that used questionnaires as a form of evaluation assessed subjective satisfaction, perceived usefulness, and perceived usability, apart from one study that assessed perceived learning (Table 11). Assessing students’ perception of learning and usability is expected as questionnaires ultimately assess participants’ subjective opinions, and thus, they don’t objectively measure metrics such as students’ learning.

For example, the authors in (Fryer et al., 2017) used Cleverbot, a chatbot designed to learn from its past conversations with humans. User-driven chatbots fit language learning as students may benefit from an unguided conversation. The authors in (Ruan et al., 2021) used a similar approach where students freely speak a foreign language. The chatbot assesses the quality of the transcribed text and provides constructive feedback.

This approach ensured higher participation and meaningful interaction with the chatbots, contributing to the study’s insights into the effectiveness of AICs in language education. Chatbot use in education can provide benefits to both the student and the teacher. Chatbots have been shown to be capable of providing students with immediate feedback, quick access to information, increasing engagement and interest, and creating course material individualized to the learner. The release of Chat Generative Pre-Trained Transformer (ChatGPT) (OpenAI, 2023a) in November 2022 sparked the rise of the rapid development of chatbots utilizing artificial intelligence (AI). Chatbots are software applications with the ability to respond to human prompting (Cunningham-Nelson et al., 2019).

The study reported positive user feedback on the chatbot’s ease of use, usefulness, and enjoyment, as measured by the Technology Acceptance Model (TAM). Similarly, Yang (2022) underscored the favourable views of AICs in English language education, with teachers valuing the chatbot’s capacity to manage routine tasks, thereby allowing them to concentrate on more substantial classroom duties. In this study, students appreciated the supplemental use of chatbots for their ability to provide immediate feedback on unfamiliar words or concepts, thereby enriching their English textbook learning. The third area explores how AICs’ design can positively affect language learning outcomes. Modern AICs usually include an interface with multimedia content, real-time feedback, and social media integration (Haristiani & Rifa’I, 2020). They also employ advanced speech technologies to ensure accessible and humanlike dialogues (Petrović & Jovanović, 2021).

They should avoid sharing sensitive personal information and refrain from using the model to extract or manipulate personal data without proper consent. Chatbots’ expertise is based on the training data it has received (although they do have the ability to “learn” with exposure to new information), and they may not possess the depth of knowledge in specialized or niche areas. In such cases, subject matter experts should be consulted for accurate and comprehensive information.

As an example of an evaluation study, the researchers in (Ruan et al., 2019) assessed students’ reactions and behavior while using ‘BookBuddy,’ a chatbot that helps students read books. The researchers recorded the facial expressions of the participants using webcams. It turned out that the students were engaged more than half of the time while using BookBuddy.

All rights are reserved, including those for text and data mining, AI training, and similar technologies. If you are ready to explore chatbots’ potential in the education sector, consider trying respond.io, a platform that revolutionizes customer communication. Education businesses like E4CC, Qobolak and CUHK have already seen success with respond.io. It is a superfast virtual agent that can accurately reply to customer inquiries. To ensure this, you only need to make sure you train it with your knowledge sources, such as course catalogs and syllabi, policies and procedures. Admitting hundreds of students with varied fee structures, course details, and specializations can be a task for administrators.

While chatbots serve as valuable educational tools, they cannot replace teachers entirely. Instead, they complement educators by automating administrative tasks, providing instant support, and offering https://chat.openai.com/ personalized learning experiences. Teachers’ expertise and human touch are indispensable for fostering critical thinking, emotional intelligence, and meaningful connections with students.

For example, while Buddy.ai is oriented towards developing oral skills in children at a lower level, John Bot and Andy are designed for vocabulary and grammar building through role-playing interactions at more intermediate levels. Natural Conversational Interaction (#7NCI) pertains to the chatbot’s ability to emulate the natural flow and dynamics of human conversation. It involves several key elements, such as maintaining a contextually relevant conversation, understanding and responding appropriately to user inputs, demonstrating empathy, and adapting the language style and tone to suit the learner’s preferences. The goal is to create a conversation that not only provides informative and accurate responses but also engages users in a manner that simulates a human-to-human interaction. None of the AICs reached the desired level of conversational naturalness, as participants found their responses predictable and lacking the adaptability seen in human tutors. The proliferation of smartphones in the late 2000s led to the integration of educational chatbots into mobile applications.

Their ability to communicate in various languages fosters inclusivity, ensuring that all students can learn and engage effectively, irrespective of their native language. Through this multilingual support, chatbots promote a more interconnected and enriching educational experience for a globally diverse student body. These educational chatbots are like magical helpers transforming the way schools interact with students. Now we can easily explore all kinds of activities related to our studies, thanks to these friendly AI companions by our side. Relations graph of pedagogical roles and objectives for implementing chatbots.

AI chatbots can be attentive to – and train on – students’ learning habits and areas of difficulty. It has been scientifically proven that not everyone understands and learns in the same way. To cater to the needs of every student in terms of complex topics or subjects, chatbots can customize the learning plan and make sure that students gain maximum knowledge – in the classroom and even outside.

In terms of the educational role, slightly more than half of the studies used teaching agents, while 13 studies (36.11%) used peer agents. Only two studies presented a teachable agent, and another two studies presented a motivational agent. Teaching agents gave students tutorials or asked them to watch videos with follow-up discussions.

Yellow.ai is an excellent conversational AI platform vendor that can help you automate your business processes and deliver a world-class customer experience. With our AI chatbots in education, schools can engage with prospective students right from the point of admission to making learning fun for them.If your educational institution is looking for an AI chatbot, schedule a demo and have a conversation with our experts. They can guide you through the process of deploying an educational chatbot and using it to its full potential. Education chatbots are interactive artificial intelligence (AI) applications utilized by EdTech companies, universities, schools, and other educational institutions. They serve as virtual assistants, aiding in student instruction, paper assessments, data retrieval for both students and alumni, curriculum updates, and coordinating admission processes.

In 2023, AI chatbots are transforming the education industry with their versatile applications. Among the numerous use cases of chatbots, there are several industry-specific applications of AI chatbots in education. Institutions seeking support in any of these areas can implement chatbots and anticipate remarkable outcomes.

It should be noted that pedagogical roles were not identified for all the publications examined. Future studies should explore chatbot localization, where a chatbot is customized based on the culture and context it is used in. Moreover, researchers should explore devising frameworks for designing and developing educational chatbots to guide educators to build usable and effective chatbots. Finally, researchers should explore EUD tools that allow non-programmer educators to design and develop educational chatbots to facilitate the development of educational chatbots.

AI chatbots provide time-saving assistance by handling routine administrative tasks such as scheduling, grading, and providing information to students, allowing educators to focus more on instructional planning and student engagement. Educators can improve their pedagogy by leveraging AI chatbots to augment their instruction and offer personalized support to students. By customizing educational content and generating prompts for open-ended questions aligned with specific learning objectives, teachers can cater to individual student needs and enhance the learning experience. Additionally, educators can use AI chatbots to create tailored learning materials and activities to accommodate students’ unique interests and learning styles. Only four (11.11%) articles used chatbots that engage in user-driven conversations where the user controls the conversation and the chatbot does not have a premade response.

Students and teachers should be educated on the accuracy of the text produced by chatbots and always fact-check the information produced by them. Conversational AI is revolutionizing the way businesses communicate with their customers and everyone is loving this new way. Businesses are adopting artificial intelligence and investing more and more in it for automating different business processes like customer support, marketing, sales, customer engagement and overall customer experience. From teachers to syllabus, admissions to hygiene, schools can collect information on all the aspects and become champions in their sector.

Both Google Bard and ChatGPT are sizable language model chatbots that undergo training on extensive datasets of text and code. They possess the ability to generate text, create diverse creative content, and provide informative answers to questions, although their accuracy may not always be perfect. The key difference is that Google Bard is trained on a dataset that includes text from the internet, while ChatGPT is trained on a dataset that includes text from books and articles.

The design of CPAs must consider social, emotional, cognitive, and pedagogical aspects (Gulz et al., 2011; King, 2002). Chatbots, also known as conversational agents, enable the interaction of humans with computers through natural language, by applying the technology of natural language processing (NLP) (Bradeško & Mladenić, 2012). In fact, the size of the chatbot market worldwide is expected to be 1.23 billion dollars in 2025 (Kaczorowska-Spychalska, 2019). In the US alone, the chatbot industry was valued at 113 million US dollars and is expected to reach 994.5 million US dollars in 2024 Footnote 1. The Chatbot-Human Interaction Satisfaction Model (CHISM) is a tool previously designed and used to measure participants’ satisfaction with intelligent conversational agents in language learning (Belda-Medina et al., 2022). This model was specifically adapted for this study to be implemented with AICs.

While the benefits of chatbots in education are significant, there are challenges to consider. Regular testing with real users and incorporating their feedback is critical to the success of your chatbot. Each iteration should aim to improve the user experience and streamline communication further.

Finally, the chatbot discussed by (Verleger & Pembridge, 2018) was built upon a Q&A database related to a programming course. Nevertheless, because the tool did not produce answers to some questions, some students decided to abandon it and instead use standard search engines to find answers. A conversational agent can hold a discussion with students in a variety of ways, ranging from spoken (Wik & Hjalmarsson, 2009) to text-based (Chaudhuri et al., 2009) to nonverbal (Wik & Hjalmarsson, 2009; Ruttkay & Pelachaud, 2006). Similarly, the agent’s visual appearance can be human-like or cartoonish, static or animated, two-dimensional or three-dimensional (Dehn & Van Mulken, 2000). Conversational agents have been developed over the last decade to serve a variety of pedagogical roles, such as tutors, coaches, and learning companions (Haake & Gulz, 2009). The CHISM model offers a comprehensive approach to evaluating AICs, encompassing not only linguistic capabilities but also design and user experience aspects.

This work was supported by the Ministry of Higher Education, Scientific Research and Innovation, the Digital Development Agency (DDA), and the CNRST of Morocco (Al-Khawarizmi program, Project 22). Authors are thankful to all the teaching staff from the Regional Center for Education and Training Professions of Souss Massa (CRMEF-SM) for their help in the evaluation, and all of the participants who took part in this study. Users should provide feedback to OpenAI, Google, and other relevant creators and stakeholders regarding any concerns or issues they encounter while using chatbots.