Building a vibe coding instructor

I’ve spent many years teaching programming, and I’ve always tried to adapt my approach to fit the current technology and anticipate my learners’ needs. AI is having a huge impact on programming: Novices can now simply talk to an LLM and get it to output working code. On the whole, this is a good thing. Programming can be fun and rewarding, and vibe coding opens up possibilities for a new cohort of hobbyists and tinkerers who, otherwise, might have considered it beyond their capabilities.

But where does that leave the field of coding education, especially at the novice level? I believe that we have to adapt our approach to fit with the current tools and with the habits and aspirations of learners. But we also need to give them a scaffold that they can use to build a solid foundation, if they wish to. A base from which to explore programming and computer science in a more structured way.

Use, Modify, Create

I’m a fan of the Use, Modify, Create framework, which happens to be a great fit for AI-assisted coding. Someone who’s vibe coding in the purest sense is relying entirely on an LLM to write code for them, iterating on the work in a chat format. They’re squarely in the Use phase: They have no understanding of the code. So their investment is limited by the capabilities of the agent and the complexity of the project.

To get to the next step, modifying the AI-generated code, learners have to begin to read and understand the output. One way to do that is to make small changes to the code themselves. With the right learning approach, this is a great way to gradually build confidence and investment in the project.

As learners gain confidence, they’ll begin attempting to write code on their own. This is a critical step in learning, and one they can’t skip. The further they progress down that path, the more they’ll be able to accomplish with an AI assistant.

A goal

I have aspirations to build an app that delivers a learn-to-code experience for novices, with AI assistance built in, based on the use-modify-create framework.

My current choice of environment is P5.js. Like Scratch, it provides instant feedback when people change their code. It’s a constrained environment, which helps to keep complexity manageable, and it’s geared towards creative expression, which I think is a great vehicle for learning.

The seed of my idea comes from a Playlab app I built to help programming teachers construct learning paths for novice coders. In building the app, I spent time reflecting on what a good P5.js learning path looks like. I built a structured representation of some core P5.js and JavaScript concepts to formalize the relationships between them so I could determine the prerequisites necessary to learning a given topic.

A prototype

I’m approaching things incrementally; there are many unknowns, and I prefer to prototype and iterate on ideas, knowing that things will evolve as I learn more and as LLMs change.

Right now, I’m tinkering with a custom GPT. It’s clunky: You have to use the ChatGPT window to interact with the AI and generate code, then copy it from the chat and paste it into a P5.js editor. But it’s a proof of concept that I can experiment with, and a good first step in the design process.

My biggest challenge is forcing the LLM to stay on the learning path. I found early on that it wasn’t enough to hand it the JSON file defining the dependencies, and then rely on it to generate appropriate code—for example, if a learner is just starting out, don’t put an array or for loop in your code.

I spent some time modifying the JSON from the Playlab app (you can view it here), imposing a cleaner structure and adding regular expressions to check for the presence of code pertaining to any given learning objective. Then I built a little Python script that reads the JSON, scans the code, and given the scope of a learner’s current objectives, validates its appropriateness.

Limitations

Unfortunately, I haven’t found a way to force a custom GPT to run the script every time it generates code. I was encouraged by its performance at first, but I’ve found that it will still create code that skips many steps of my formalized learning path.

I’m fairly certain that I’ll have to build an app on my own, which is unfortunate: a custom app relies on API calls, which will cost me money. I’m going to do more investigation to see if there’s a way to do what I want while relying on learners to use their own AI service accounts. In the meantime, you can check out the current version of the GPT and let me know what you think.