AI for Coding: Good, Bad, or Ugly?

Generative AI and Coding 🔗

Generative AI (genAI) and more specifically, LLMs (large language models) are upending a lot of fields right now and none more so than computer programming. My personal experience has been that while AI can be useful for generating somewhat bland, occasionally inaccurate written content for human consumption in other fields, it really does shine at writing code. That doesn’t mean that it’s perfect though. There are a ton of issues with using AI for coding and I want to explore both sides of my experience with it here. To be upfront, while I do a lot of casual coding as a hobby and incidentally for my job, I have no formal training in software engineering, and would not at all consider myself a professional software developer.

I’ve read a few blogs from those that seem to have more experience than me in the field with wildly different conclusions. Unix Sheikh’s recent post basically concluded that AI is snake oil for most developers, whereas this post from Simon Willison claims LLMs are powerful programming tools if you know how to use them. Even Linus Torvalds, the creator of the Linux kernel has stated that he used genAI for one of his projects. So what’s my experience? While I do think that the way LLM technology is being rolled out as a whole is a net negative for human society, I can’t help but admit that it is a net positive productivity tool for anyone doing any sort of coding. The tools are just too powerful to be ignored in my opinion. I’ll go into a bit of detail below on the strengths as well as the risks of using LLMs as a coding tool.

Vibe Coding to Artisan Coding: A Spectrum 🔗

As with any tool, there are multiple ways genAI can be used to code. On the one extreme, we have “vibe coding”, which essentially is where you use the AI to generate code and run it without examining the code at all. If there’s an error, just copy and paste the error message into the AI chat and have it change the code accordingly. This method has obvious limitations, and it is unlikely that you will be able to build anything too complex with this method. Near the other extreme, you can use genAI for very limited things, like asking it to format some regex to use within your code. This saves you the trouble of remembering the regex syntax while also being easily verifiable. And of course, there are many levels in between. Have the genAI build a function or class for you and carefully review it, making manual changes as you go. Or write the initial code yourself and have the genAI model review it and offer suggestions for improvements and changes based on your goals. Even if you don’t want to vibe code, there are many ways to integrate genAI into your coding workflow without blindly trusting its output. And of course, you could always go to the other far extreme and eschew using AI altogether, what Unix Sheikh dubbed “artisan” coding in his blog post.

Raise the Coding Skill Floor 🔗

Even the best genAI models currently available will not replace an experienced developer. That said, it does have the ability to turn someone with little to no technical knowledge into a developer of modest skill. Sure, trying to purely vibe code anything too complex will lead to a broken mess, but for simple scripts and even basic tools, genAI can write something that works well enough for a user with virtually no knowledge of the language its being written in.

Build a Development Team 🔗

For an experienced developer, the genAI’s output will almost certainly be inferior to what the developer could write themselves. So why use them? The main advantages are code volume and speed. A senior developer could take weeks or months to write all the code for a large project on their own. But with genAI at their disposal, they can just outline the project, have the genAI write it, and then review the code output. The ability to quickly read and understand another’s code is an important skill here, but with it, the senior developer is simply reading code and providing feedback and making changes rather than writing it all themselves. This is already how many software development teams work even without AI, with senior developers guiding the project and reviewing the junior developer’s code. If you see AI as a team of extremely fast but somewhat error-prone junior developers, it can turn a single experienced programmer into an entire software development team. More advanced AI tools even allow you to give the AI models PR permissions to your git repositories and merge or reject PRs as desired.

The Ultimate Rubber Duck 🔗

Personally, I find genAI models are great for any kind of design or debugging process. Rather than using the traditional rubber duck, I can simply write out my assessment of the situation in an AI chat. The best part is the AI actually responds and will often notice things that I did not! Especially if a small syntax error is ruining your project, genAI can very quickly review the problematic code and identify the problem with none of the tedium required by a human to do the same. A real, intelligent human to debug with you would often be preferable in many scenarios, but real humans may not have the energy or time to devote to helping you with your project. GenAI has infinite patience and will work on your schedule.

Private Coding Tutor 🔗

GenAI can also teach you as you use it, assuming you use it correctly. Read the code it outputs. Don’t understand something? Ask the AI. Verify its explanations, don’t trust it explicitly. If the explanations don’t make sense, double-check it against the documentation or run a code snippet and see that it behaves as expected according to the explanation. Once you understand what the code is doing, you can challenge its methods and propose alternative solutions. Ask the AI to make a table comparing and contrasting the different approaches. Using this method, you will learn a lot as you go and you will end up with better code as well, even if it requires more time than vibe coding.

Dangers & Risks 🔗

Using genAI has a lot of security risks, but I want to focus here on the productivity risks. GenAI is another tool you need to add to your workflow. It will take time to get it working and integrate it in a way that works for you. GenAI is also evolving fast with literal trillions of dollars being poured into the field. By the time you learn the limitations and what it can and can’t do, chances are those limitations will have changed. GenAI also promotes extreme laziness. Sure, you should read the code, but why bother, it’s probably good enough. When you have code that seems to be working, it can be very tempting to declare it “good enough” even if you have no idea how or why the code works. Of course, genAI is also a master of generating things that make sense at first glance, but fall apart upon further inspection. You may even review the code, think it looks good and then realize it has all these edge cases and bugs that would never have made it past you if you had coded the logic yourself. There is also the risk of outright hallucinations like making up code libraries of development tools that don’t exist. If you rely too heavily on the AI model, you may find your project filled with references to non-existent dependencies.

Tips for Better Outputs 🔗

I’ve mentioned some of these tips along the way, but let me make them explicit here.

1. Outline a plan with the AI before you start coding. Even if you’re not writing the code yourself, you shouldn’t offload all of the cognitive work. Ask the AI to make a plan for the code it will write in high-level pseudo-code. This is the best stage to determine the logic and make changes, even before you or the AI has written a single line of code.

2. Challenge the AI and ask it for improvements. Don’t assume that the AI’s output is the best it can do. Ask it why it’s using a specific library, ask it to make the code more concise or better commented. When the code is finished, ask it to review its work and recommend improvements. Ask it to audit the code for security. This is nowhere near as effective having a security professional review it, but AI will often generate insecure code that it can correct if you specifically ask it to look for vulnerabilities.

3. Verify, verify, verify. Verify that what the AI outputs makes sense. Verify that what the code actually does matches what the AI says it should do. Verify that the code holds up in edge cases and other scenarios that don’t exactly match the original prompt.

Hopefully some of these tips or the even just the general discussion will be helpful to people. There are a lot of points of discussion around generative AI beyond how it helps people code, and my attitude is a lot less positive toward those other aspects. But this is one area where I think the potential benefits really are obvious and if people can take advantage of it, I hope they do so!