The Truth About ChatGPT as Your Coding Mentor: A Reality Check

The concept of using AI for educational purposes is not new, but its practical applications are still in the experimental phase. One such application is the use of ChatGPT as an automated mentor in the programming world. While the idea sounds promising, the reality may not be as rosy. This article delves into the various roles that ChatGPT can play in the educational process and evaluates its effectiveness, particularly in the realm of coding.

The Seven Roles of ChatGPT in Education

According to a paper by Ethan and Lilach Mollick, ChatGPT can serve in seven different roles in the educational process: Mentor, Tutor, Coach, Student, Teammate, Student, Simulator, and Tool. Each role comes with its own set of guidelines, risks, and instructions for both teachers and students. However, the focus here is primarily on the Mentor role, which aims to guide individuals in acquiring new technical skills.

The Mentor Role: A Closer Look

The Mentor role is crucial for learning any skill, including programming. It's not just about understanding the syntax and semantics of a programming language; it's about problem-solving. Tim O'Reilly has always emphasized that educational materials should act like "someone wise and experienced looking over your shoulder." But does ChatGPT live up to this expectation?

The Ruby Experiment

The author tested ChatGPT's mentoring capabilities by running a Ruby program for generating prime numbers. ChatGPT offered basic advice on following Ruby naming conventions and avoiding inline comments. It also pointed out a security issue related to reading command-line arguments. However, ChatGPT went ahead and rewrote the program, which is not what a mentor should do. The learner should make the changes themselves.

The Python Experiment

In another test with a Python program that used the Pandas library, ChatGPT suggested stylistic changes and some exception handling. However, it failed to provide any insightful advice. When the author pushed ChatGPT for more, it only agreed with the suggestions made by the author but did not offer any of its own. Again, ChatGPT rewrote the code, introducing subtle changes that could lead to errors.

The Limitations

ChatGPT is good at giving basic advice but falls short when it comes to providing insightful suggestions. It can recognize good suggestions from the user but fails to make those suggestions itself. Moreover, ChatGPT's tendency to rewrite code can introduce errors, emphasizing the need for rigorous testing.

Final Words

While ChatGPT shows promise as an automated mentor, it's not there yet. It lacks the ability to provide truly insightful advice and tends to make subtle changes that could lead to errors. For now, if you're looking for a mentor, you might be better off seeking human guidance. However, as language models like ChatGPT continue to evolve, their potential as mentors should not be underestimated.