Language Teaching

Chatting About and Chatting With ChatGPT: Evaluating a Chatbot as a Language Learning Tool

Gwangju News

By David Shaffer

Google’s Bard, Amazon’s Lex, Microsoft’s Bot Framework, and of course, OpenAI’s ChatGPT – these are all chatbot platforms that possess natural language processing capabilities. They are not to be confused with personal assistants such as Apple’s Siri and Amazon’s Alexa. Though all use artificial intelligence (AI), personal assistants are programmed to directly control digital devices and applications to make daily life more convenient for the user.

Chatbots recently shot into the popular dialogue, propelled mainly by the release of ChatGPT a few months ago. It has been lauded is some circles as a godsend, while others denounce it as bedeviling, especially those in the field of education. Here we will discuss what ChatGPT can do, what it does well and not so well, and the impacts that it might have in the fields of education and second language learning for both teachers and students. (By the way, the “GPT” in “ChatGPT” stands for “generative pre-trained transformer,” a technology that analyzes input text and predicts the output by training itself on a large dataset to produce human-like written language.)

The fields of design, manufacturing, and marketing have already espoused AI to optimize products, adjust design, target advertising, and improve customer service. Using the same technology that powers ChatGPT, CarMax is summarizing its many customer reviews, and the groceries delivery and pick-up company Instacart is integrating ChatGPT into its customer service to respond to customer questions.1 AI chatbots, of which ChatGPT is presently the most advanced chatbot resource available, apply natural language processing technology (as opposed to computer language) to large language models to create their database of knowledge to answer written questions and perform certain tasks such as writing essays and creating poetry. (You may remember that the mystery story in last month’s issue of the Gwangju News was written by ChatGPT.)

While there is the potential for AI to be used for nefarious purposes, it appears that ChatGPT does not provide information on how to accomplish activities that are illegal and unethical. In a recent New York Times article, Noam Chomsky states that humans can still outthink AI. He argues that ChatGPT and similar chatbots are inadequate for producing meaningful human language, as they simply mimic language usage rather than understanding its meaning.2 And if you ask ChatGPT if it agrees with Chomsky, it will say that it agrees completely!

Svetlana Kandybovich echoes Chomsky’s sentiment: “GPT is powered by a large language model, which is ultimately a statistical tool which is asked to produce statistically plausible answers. It’s used to predict language without understanding” (37’ 03”).3 Kandybovich goes on to say, “So all it does is spit out what it expects is going to come next. This means that GPT may generate incorrect answers by unintentionally putting the wrong bits of information together. It never generates anything too unexpected, so it’s like the average of all the contents on the internet now. GPT cannot be trusted as it does not have any inbuilt notion of what is true and what is not” (37’ 23”). An anecdote to add here is that when I asked ChatGPT in Korean whether it knows Korean (한국말을 아십니까?), it replied that it knows Korean very, very well (네 한국말을 아주 잘 아십니다), using an honorific form in reference to itself, something that is not done in Korean. However, Kandybovich goes on to say that although ChatGPT was not designed for second language learning, it has potential for use in both language teaching and learning.

Language Learning Benefits

Let’s see what ChatGPT has to say about its potential for language learning. I asked ChatGPT to tell me in about 300 words how it can be beneficial in learning a second language. It told me that it is a potentially valuable tool in allowing users to interact with a virtual conversation partner in the language that they are learning to build conversational fluency. It can do this by answering the user’s questions – something that ChatGPT is quite good at. And it is always there for you for real-time interaction, while your dialogue partner on text messengers may not be. ChatGPT provides immediate feedback and error correction. If you ask it whether something is grammatically correct, it will often provide you with the correct form without even being asked to.

As ChatGPT is software rather than “humanware,” it removes the fear that so many learners in Korea have of making mistakes. This fosters the building of learner confidence and motivates further learning. ChatGPT can also help learners with understanding the nuances of the target language if the learner asks ChatGPT about items that they have encountered elsewhere, such as idioms (“He kicked the bucket”), culture-related elements (“Why do English speakers say ‘thank you’ so often?”), and things like jokes and sarcasm.

ChatGPT will repeatedly tell you that it can provide the language learner practice with “human-like conversation.” Let’s take a minute to consider this, though. Conversation is oral dialogue, but ChatGPT has no capability to produce oral output. Its output is written only. You could copy-and-paste the text it produces into a text-to-speech device as you can do the same with any digital text. This still does not make its written output highly conversational because its output is in a more-formal written style than in spoken style. Spoken language also contains pauses ( . . . ), fillers (ah, um, you know), false starts, intonation, accents, gestures, and a wide range of contractions (she’ll, gonna, y’know) that ChatGPT does not produce. Additionally, its replies to questions are stilted rather than conversational in that the question is often repeated in its entirety, albeit restructured, in the answers it produces. For example, if you ask ChatGPT “Does Texas border four states, one country, and five bodies of water?” it will reply “Yes, Texas borders four states, one country, and five bodies of water” rather than the idiomatic “Yes, it does.”

In addition to ChatGPT not producing oral output in its interaction with its user, the user does not speak either. The user’s output in also written, so there is no oral speaking practice taking place. The chatbot–user interaction is more akin to sign language than it is to speaking practice. As you can see, although ChatGPT does have capabilities that can be helpful to a language learner in improving their conversational skills, its ability to produce “human-like conversation” is limited in multiple ways.

Advantages over the Classroom

Ask ChatGPT what the advantages are of using it for second language learning over learning in the classroom, and it will tell you some advantages that are hard to argue with. An engaging and interactive environment – yes, it has engaged me for hours with its written input/output interactivity. Immediate feedback – one does not need to wait their turn to speak as they would have to in a classroom with two dozen classmates. No fear factor – one does not have to worry with ChatGPT of 50 piercing human eyes staring at you as when you are taking your turn to speak in the classroom. Practice at your own pace – this is easy to do with ChatGPT but very difficult in a classroom of 20-plus students, even if the teacher uses differentiated learning techniques. The fear of being the last one to complete the activity is gone with ChatGPT.

ChatGPT vs. the Human Teacher

Does this mean that ChatGPT and similar chatbots will be replacing the classroom teacher? Don’t get too concerned just yet. Even ChatGPT can tell you that the human teacher can be of value in ways that ChatGPT cannot. A teacher can act as a mentor, giving direction and feedback focusing more directly on a student’s needs. A teacher can provide personalized guidance in language learning skills (e.g., pronunciation for speaking, phonics for reading, and grammar correction). A teacher can provide a more holistic approach to language learning, incorporating the four macroskills into a single activity, as well as incorporating microskills and even cultural and custom aspects associated with the language. A teacher can also provide an array of interesting and interactive activities to engage students, as well as a multitude of multimedia materials: videos, audio materials, songs, PowerPoint presentations, language learning apps, etc. And very importantly, a teacher can provide a model of oral speech and idiomatic conversation, and also very importantly, the teacher can have students interact with their peers in practicing their oral skills.

Harmonious Habitation

Presented above are strong points of both ChatGPT and the classroom teacher. Rather than trying to choose which may be better, the two can and should complement each other. While the teacher is well suited for the classroom environment, ChatGPT can be an excellent supplement for out-of-class language learning. The teacher can assign activities for students to do with ChatGPT as homework. Especially useful would be activities that foster a sense of language learning autonomy in students. Once students realize how to use ChatGPT and the variety of language learning activities that they can do efficiently and interestingly with ChatGPT, they will be on their way to much more autonomous language learning.

Doing Your Own Work

Much of the uproar that occurred when ChatGPT came out, among educators in general and among language educators specifically, was that ChatGPT and similar chatbots can be used to do homework assignments. This chatbots can do well for a variety of assignments. However, in checking this out for essay writing, I asked ChatGPT to produce an essay of about 500 words using the five-paragraph model. What I got back, to my surprise, was a 250-word composition of three short paragraphs. Not a passing grade in my book, though the grammar was perfect.

What teachers need to do is familiarize themselves with chatbots, i.e., with what they can do and the type of output they produce. Then, give homework assignments that would be difficult for a chatbot to do well. Remember when the internet first came into widespread use? Everyone was so worried about copy-and-paste plagiarism. But various ways have been found to detect it, and it is not longer considered such a big issue. I imagine that something similar will occur involving academic integrity in relation to chatbots. As one educator put it, “To overcome these challenges, language instructors need a good understanding of the tool and must closely monitor and supervise students’ learning experience with it.”4

As with most any good thing, it can also be put to a bad use. Teachers must, of course, be vigilant in guarding against such misuse. However, as ChatGPT can be such a powerful language learning tool, we should embrace it in our teaching, in our lesson preparation, and in our students’ learning.

Sources

1 O’Brien, M., & Hadero, H. (2023, March 8). From marketing to design, brands adopt AI tools despite risk. Chicago Tribune.

2 Chomsky, N. (2023, March 3). Noam Chomsky and AI: Humans can still outthink machines. The New York Times.

3 Thornbury, S. (2023, February 18). OpenAI and the future of language teaching [Webinar]. YouTube. https://www.youtube.com/watch?v=zS0cd7e_hh4

4 Cai, W. (2023, February 24). ChatGPT can be powerful tool for language learning. University Affairs. https://www.universityaffairs.ca/career-advice/career-advice-article/chatgpt-can-be-powerful-tool-for-language-learning/

Gwangju-Jeonnam KOTESOL Upcoming Events

Gwangju-Jeonnam Chapter Workshop

  • Workshop Topic: What Does Artificial Intelligence Mean for Language Teachers?
  • SwapShop Topic: Top Tech Tips, According to Real Teachers
  • Date: April 8, 2023
  • Venue: Gwangju National University of Education

Check the Chapter’s webpages and Facebook group periodically for updates on chapter events and other online and in-person KOTESOL activities.

For full event details:

  • Website: http://koreatesol.org/gwangju
  • Facebook: Gwangju-Jeonnam KOTESOL

The Author

David Shaffer has been involved in TEFL, teacher training, writing, and research in Gwangju for many years, almost totally as a professor at Chosun University. As vice-president of the Gwangju-Jeonnam Chapter of KOTESOL, he invites you to participate in the chapter’s teacher development workshops and events (online and in person) and in KOTESOL activities in general. Dr. Shaffer is a past president of KOTESOL and is the editor-in-chief of the Gwangju News.