How ChatGPT Could Help Educators Teach Physics

Since the release of OpenAI’s chatbot ChatGPT in November 2022, Anthropic’s chatbot Claude in March 2023, and Google’s chatbot Gemini in February 2024, millions of people have used generative AI to write emails, plan travel, and create art. Proponents say the technology is poised to upend entire industries, from marketing to scientific research.

Generative AI has cropped up in education, too. While many early headlines were cautionary, highlighting students who used the technology to cheat, researchers are also studying how educators could embrace the technology in the classroom.

“Instead of viewing AI as an enemy, what we really should be doing is to help people develop skills to properly use AI, and understand its abilities and limitations,” says Zhongzhou Chen of the University of Central Florida, who presented his physics education research on ChatGPT at this year’s meeting of the APS Division of Atomic, Molecular and Optical Physics in Fort Worth, Texas.

Yasemin Kalender, a physics education researcher at the University of Liverpool, suggests that educators view generative AI as another teaching tool. She also notes that banning large language models — the AI technology that powers chatbots — simply isn’t realistic. “This tool is out there,” she says. “Instead of trying to police students, we can actually have an honest conversation.”

Large language models come with caveats, including their energy consumption, says Kalender. Over the last five years, Google’s emissions rose 48 percent, much of which is attributed to AI. The rise comes despite the company’s goal in 2021 to achieve net zero emissions by 2030.

The models’ propensity to “hallucinate” — generate false or irrelevant information — is also well-known. In June 2023, a U.S. federal judge fined two lawyers for citing fictional legal cases that ChatGPT invented. This February, Google apologized when Gemini generated images depicting Nazi-era German soldiers as people of color.

Chen and his colleagues have turned ChatGPT’s fallibility on its head to teach students about misinformation. In a recent project, they first prompted the tool to generate misinformation about climate change that “sounded” valid — for example, that cool summers proved climate change was a hoax. A colleague then used that text to teach students about logical fallacies in a course on the physics of climate science. Chen plans to analyze how those generated texts affected the students’ comprehension.

Meanwhile, Kalender has explored how to incorporate ChatGPT in introductory physics labs. For example, she and a colleague had ChatGPT simulate a common lab experiment using a mass on a spring and then analyze the data to measure the spring constant. A plug-in to ChatGPT called CodeInterpreter generated “perfect” Python code for the analysis, she says. This may help students focus on learning concepts during the lab, instead of spending time debugging code. “It was taking them a little bit away from the physics,” says Kalender.

Kalender has also used ChatGPT to brainstorm lab activities. After prompting ChatGPT to come up with a two-week lab exercise on projectile motion, the chatbot suggested a timeline for a catapult-building lab. Upon further prompting to make the lab “equitable,” the chatbot recommended a format for dividing the labor in which students rotated roles between weeks. These pedagogical cues could be helpful to instructors, says Kalender.

However, ChatGPT made mistakes in the lab, including on error propagation, a skill that students typically struggle at. While ChatGPT could correct its mistakes after prompting, it may pose problems for students.

Chen is investigating generative AI’s ability to write exams, too. He has used GPT-3.5, a large language model that ChatGPT used, to generate variations of problems that test the same concepts. He started by writing the “seed,” which describes the problem (like a block sliding down a ramp), the known variables (like the ramp’s angle), and what the student was to solve for. Then, he would brainstorm variations, like swapping the block for a cart. Using the seeds as input, ChatGPT fleshed out the problems in full sentences and chose numbers for variables. In this way, he created “banks” of 30 to 50 problems that tested the same concepts.

For an exam in a physics class in spring 2023, Chen assigned a problem from the bank, as well as another that was not in the bank but tested the same concept. The students’ performance on both questions were correlated, implying that most students did not merely memorize the problem in the bank.

ChatGPT-generated problem banks could help instructors offer different versions of the same exam, potentially giving students more flexibility. Many nontraditional students, such as parents and working students, run into logistical challenges during final exams, says Chen. That flexibility could also help students whose coursework is disrupted by natural disasters, such as hurricanes in Chen’s home state of Florida.

“The traditional mode of everybody marching at the same pace is facing increasing difficulty because of the diversity of the student population, and even also climate change,” he says.

However, Kalender is wary that students will rely on ChatGPT too readily. “In learning theories, there is this concept of ‘productive struggle,’ where in order to learn new information, you have to struggle — not too much, but a little bit,” says Kalender. She is concerned that ChatGPT “will take away from that productive struggle.”

Chen envisions a future in which students are allowed to use any tools, including AI, to learn physics. Come exam time, students solve problems in a proctored room with no access to AI. This way, students have less motivation to use AI without actually learning the material, says Chen, “because they know that their end goal is to perform without using AI.” He also thinks that educators should design tests to assess students’ ability to use AI.

But it’s not only students facing the trials of new technologies, notes Kalender. “It’s forcing us as educators to come out of our comfort zone.”