A bunch of these should be enforce with linting, that way people who still hand-craft code get the same kind of feedback, e.g. Always use {}, even on a one-line "if" statement. & Keep function names short. Less than 30 characters.
Then this one really is a pattern that creates a lot of churn:
- Add a small, to the point, comment to explain what the block does and why. Use examples when possible. Propose ASCII drawings to explain complete systems.
I forbid my agents from adding any comments. I review the code and add comments manually. If I can't understand something despite having the context then I throw away the code instead of having an LLM generate comments to explain what it did. This way the code stays readable/debuggable by humans.
If you don’t trust the code to write a decent comment, why trust it write good code?
Of course, ensuring compilation or other checks can verify some code, which it can’t do for comments. But comments still serve the same purpose as human comments.
If I understand correctly, it’s not that the LLM can’t write a good comment, it’s that you want to be able to interpret and understand the generated code without comments - and in that process end up writing comments yourself.
I don’t get that argument. Most of the time by the end of the session the comments from the agent encode tricky details that I told the agent to write down so it stops making “simplifying” assumptions. Thus comments at the end of a couple days of agent-only coding, when I start to actually read and edit the prose, contain the details which aren’t possible to know from reading the local code. It may help that my last couple sessions before I start reading the code myself are variations on telling the agent to self-review and improve the comments in specific ways, so the comments left are only those the agent thought remain meaningful at clarifying unexpected interactions between the local code and other code that needs to be referenced to understand it.
The actual code output has improved a lot over the past year. I’ve found it matches existing patterns better, and the code is succinct so I can easily tweak it if I don’t like the way the agent wrote it.
The problem with comments is that LLMs tend to copy their verbose chat output format and insert session/prompt specific details. It makes me think that LLMs aren’t constrained in their comment output the same way they are with their code output
Add "Don't add any code comments anywhere" to your system prompt.
If the model doesn't follow this, you want to start using a better model ASAP, because SOTA models for the last year or so, been able to following this without an issue.
Alright, what you'd put instead of "Don't add any code comments anywhere"?
I agree with the general guidance, but it's a general one and not applicable for everything. Some things cannot be expressed in a "do" way rather than "don't".
I don't think changing "don't" to "do not" is what the paper authors had in mind
I use things like "Your code should be self-documenting, so as to require as few comments as possible. Add comments to explain "why" or give important context not apparent from the code itself, but keep them to necessary comments only"
But that's a much laxer rule. I don't think you can truly express "no comments, ever" without a "don't" rule.
Your proposal is still a "Prohibition-type constraint" that the paper you linked earlier say "isn't good".
Some of these constraints we want simply aren't possible without adding "do not" somewhere in the line, even if you prefix/suffix it with other stuff, as you noticed yourself :)
I'm not convinced, yet, that eliminating LLM-generated comments is the right path for me. I do review everything written by an LLM and some comments are actually pretty good, but sometimes I'm just too tired to try to figure out how to reword an oddly worded one.
I just added Sanglard's rules to my ~/.claude/CLAUDE.md file, did another code review, and found some LLM-generated comments were really hard to understand. I think they're due to invented metaphors and flowery language instead of using standard terms, so I've added this:
- Comments must be literal. Don't invent figurative language for what a plain technical term already says — write "rows still reference it," not "rows still wear it."
Improving LLM code generation is an iterative process. I'm glad people share their efforts to improve it.
> This way the code stays readable/debuggable by humans.
Please take the following as expressed with genuine curiosity: Do you not use an editor with syntax highlighting and collapsible comments?
At least on JetBrains you can configure the editor to collapse all comments on open and to have the comments displayed in a low-contrast color. This way, LLMs add a bunch of comments, but it doesn't affect your actual experience in trying to read the code. If you encounter code that seems inexplicable, then and only then would you expand the comment to see if that helps you understand.
Comments should be written only when there is (hidden) complexity or external context strictly required. Otherwise it is just easier to read the code. Comments then signal one of two things: a) the following code is really complex and I need to tread carefully, or b) this code is complicated, and could benefit from a refactor.
In regards to agentic coding, all these comments are extra contents, driving down quality while increasing cost. Agents also tend to be inconsistent about updating comments, I've had cases repeatedly where a comment did not match the code, at which point it is just a documentation liability.
Also, the language model might not fully understand the code then add a comment, then the next iteration will treat assumptions in the comment as the truth.
Sometimes I try to add comments in a new session and the agent just don't have enough context for it to give a comprehensive sentence with full context on the why, then the agent will just describe what it does.
Human comment is in another level to answer the questions mainly like "why do it like this" for the later collaborators or the forget-ed self, so the important blocks live when it is needed and can be eliminated when it does not.
> If the human can read/understand it well enough to comment it, then it is readable by humans.
No, because the one who is writing the comment has context later reader dont. The writer knows what the requirements are, what he was trying to achieve and what he struggled to comprehend. Writer also presumably spent more time trying to understand it then the person coming later should.
It is not perfect, but the delta between 'cannot understand what agent wrote' and 'it makes sense to me right now' is already an improvement. That it may not be sufficient, doesn't mean it isn't a necessary condition.
Besides, it has always been like this. I sometimes can't even understand some of the things I wrote myself a couple of months ago, because I forgot the context. Good comments and documentation will help you re-acquire the context you need, not completely eliminate it.
I added to the memory, system prompts, and the prompt itself and every soa model still litters code with the most inane useless crap. I will then get code to review from a coworker using fable/opus. It has more lines of comments then code.
Maybe I am some god tier code reader (i am not) but i dont think i have ever found a comment in code to be useful in my day job. That isnt true, i once came across
// submit to the dark lord
Above the function that sent a payment to PayPal for processing. It made me laugh so I let it be.
Hmm, I guess everyone here is using Claude? I find that Sol is much more restrained, to the point where I have a prompt to tell it to add short comments for things that are not obvious. Really, I find the verbosity problem to be worse in tests. I regularly prompt my review agent to remove tests used only for scaffolding to write the code in the first place. The agent is in a way following strict TDD, which reminds me why I don’t like TDD, even though some of the generated tests can be useful.
Sol adds comments when I use it, but it's not nearly as verbose as Opus 5/Fable.
For the latter I'll often include an example of a comment it wrote, along with my own rephrasing, and tell it that "future readers will understand code context; good naming is the best documentation". This works alright if I include in the actual prompt, but annoyingly it often doesn't in CLAUDE.md or memory.
We trained agents on millions of pages of documentation telling them to write good comments and good code and then we tell them never to write any comments.
It’s almost the “we built a robot who loves to play Sonatas and gave it no hands” type of thing.
Being overly trained on comments in documentation could be one of the reasons why Claude models write frustrating comments (which often manifest as written justifications of how the code satisfies the prompt). A comment in a tutorial is going to be geared to explaining how the code relates back to the tutorial task or restating what the code does, rather than documenting the "why" of surprising code.
What models are you using? I've had "Never add any new code comments" in my system prompts for like a year at this point, seems every model above 14B picks this up just fine. Are you using llama 7b or similar for testing this?
Right. I've really struggling to get AI to stop explaining the what. It seems to add it to the commits, PRs, code, wherever it feels like. I've put in multiple places to not write the "what", but the "why", and in multiple ways, but it still does it in one or other place.
The best way I've found to solve this is using LLM as CI - use a small cheap model to inspect the diff and look for those kinds of comments. Prompt left to the observer but using `claude -p` / `codex exec` gets you a lot cleaner output usually, and makes robots fight robots instead of you constantly having to reprompt and it ignoring you.
I would rather bet that people don’t know that their problem has been solved for ages. Either they don’t know about the tools or can’t make the leap to think of using something like awk or sed to quickly script out their use cases. Or even quickly draft up a quick function/plugin in something like vim, emacs, sublime,…
In “The Pragmatic Programmer”, the power of unix tools and editor fluency is well argued. There are plenty of other books like “Unix Power Tools”, “Small, Sharp Software Tools”,…
I'm not always forced for time. And the time is slightly less when you know how to use man (with apropos and whatis, a few unix conventions (null vs newline when piping text) and the quirks/feature of your shell.
Yeah, I've actually found in my own testing and usage of LLMs that this is where I get a lot of benefit. I already have fd, ripgrep, etc. installed and know how to use them, but it's not hard to tell the LLM to do it and it often finds things just as well. Or even better.
It's especially handy on modern style code where things get broken up across a multitude of files based on convention.
Incidentally, I'm from the opposite school and consider every “if” followed by a braced block a smell.
If a conditional body needs a block, it's doing enough to deserve a name, so I promote it to a single named call, à la "Extract till you drop".
Another phrase for this is "functional decomposition", which usually is a good thing.
Better yet is to identify conditional execution paths as early as possible in order to obviate conditionals in the call tree. For example, identifying a "create a new something" verses an "update an existing something" based on the workflow initially invoked greatly simplifies service and/or persistent store logic.
Labs have now long understood that ASCII drawing is a core skill needed for coding agents. However, I would not trust them understanding what an existing drawing means, unless it has generated itself.
The cost of custom linters has like any other code dropped through the floor. I'm sprinkling all kinds of linters over my latest projects. It seems some people are still sleeping on this, expecting great code from the agents.
They're fast and deterministic and I run them in git pre-commit.
Yeah, I'm not sure what people are thinking. I keep reading stuff from folks like "I wish models had more common sense" and "I wish they wrote better code", not realizing this is 100% in your own hands, always been. There is no such thing as "clean code" that every programmer agrees on, you have your own subjective opinion and "good" taste about the code, instruct the models to follow it! And automate it while you're on it.
Even the why sometimes shouldn’t be a comment, unless it’s very immediate to the code itself. What’s often more necessary is a high level overview of the design of the solution, because that’s what drives the design of the code and link disparate section. Especially the glossary , which you let you understand the name of the symbols (variables, struct. functions,…) used in the code.
It’s like learning the culture associated to a foreign language instead of trying to translate each single word with a dictionary.
A comment is just a summary of the code in an abstraction that's easier to follow. Let's say you have a simple function which would produce an almost as large comment, yes obviously useless. If the function is large enough, yeah summarizing it as a comment is a good idea.
Now you might say, don't write huge functions. Sure I agree, but most codebase or teams are not super disciplined enough. So comments are a compromise.
>> A comment is just a summary of the code in an abstraction that's easier to follow.
I disagree. The code already tells you what it does. A summary has low value.
Comments should be for explaining the why: the reason the function uses a particular algorithm even if it's a bit slower, or why the return format is an unconventional shape or contains redundant bits. This is so that someone coming in later (either a human or agent) doesn't get confused or think that the function needs refactoring.
Summary comments have ended up as the bane of my existence everywhere I've worked, for one simple reason: they go stale and there's no way to prevent it from happening.
That's your experience and it's valid. For me documentation AND comments have been useful on occasion.
The core thing that I agree with is that comments (or docs) can get stale and not follow what is actually being executed. The way I work with it is by being structured, consistent and follow standards. Unfortunately for me, not all developers follow the same guidelines.
Comments lie. What's worse, AI trusts the comments and apparent logic (inferred from identifiers and whatnot) more than the actual logic. You can quickly get into a mess of stale comments. What's worse, the LLM can sometimes just spit out garbage that poisons the context of the next agent. I disallow comments from my LLM for that last reason.
```auto a = 65535 + 1``` will give you a different answer depending on the architecture. You can then decide.. oh don't use auto or don't use c++ or whatever and have standards... like the standard you set for yourself "disallow comments from my LLM".
At my work we have basically banned "what" comments for blocks of code. JSDoc comments can still document what a function does at a high level but it should not go into implementation details unless they are important to anyone using the function. And any comments inside the function should always be for explaining why, not what. The "what" of a piece of code should be self explanatory by just reading the code. If it's not, then you likely should rewrite it to be more clear (sometimes hard-to-read code is necessary and then a "what" comment would be appropriate, but this is rare).
"What" comments almost always end up falling out of date or even sometimes being slightly incorrect from day one. Incorrect comments lead to confusion and bugs. If a comment says some code does X but the code actually does Y, then you don't know whether the comment is just out of date or if Y is actually a bug. But if a comment explains the intention of the code and the code contradicts that intention, then you know it's likely a bug.
> Then this one really is a pattern that creates a lot of churn:
> - Add a small, to the point, comment...
As if you even need to tell Claude to add comments. Over the past few weeks I've noticed Claude over-commenting everything. Massive PRs where you realise that fully half or more of the lines that have changed are comments.
It's no good at all: it just pollutes the context, causes token churn, ablates quality, and makes getting to a high quality outcome considerably slower and more expensive.
I get that sometimes knowing why a thing is the way it is can be useful and valuable, but this is what commit comments are for in my mind.
I've had to tell Claude to stop commenting code because the behaviour has become so problematic.
> The what _is_ the code.
Exactly.
If I don't know what the code does because it's arcane and not commented I can simply ask the LLM to explain it to me. I don't need an essay in comment form.
Linters and static analysis -> setup as hooks in your harness. Don’t rely on CLAUDE.md because it’ll ignore it a lot.
> ASCII drawings in code
Please don’t this is super obnoxious. Make proper diagrams and kee them in knowledge base. Link out to them if you need to and let the agent fetch them via MCP or API or whatever if it wants them.
This! I also saw lot of potential work that could be done by linting tools. Remember to always prefer mechanical guards than agent instructions, as they cannot ignore them.
We've been using agents heavily for all code for a long time now in my company - everyone has comprehensive & opinionated AGENTS.md customisations & they're widely shared & discussed. Almost everything in this post seems incredibly naive day-one LLM user mistakes - especially everything related to coding style at line level granularity.
We've had non-LLM tooling for these kinds of standards for many years now & the great thing about agents is they're already versed in said tooling. If you haven't got a decent lint setup, ask your agent to set one up. It'll give you much better guarantees than this slop which is just going to drift from model to model & is completely unverifiable.
One thing I don’t get with a lot of these agents.md and other skills are… why not throw as much mechanical checks and other stuff at the repo to constrain as you want instead of asking a non-deterministic agent (squishy or non-squishy) to maintain it.
With the mechanical routes, we get checks, failures, and so much more. A bit wild to me.
Make an agent operate within defined constraints and yell at it when it doesn’t.
Then this one really is a pattern that creates a lot of churn:
- Add a small, to the point, comment to explain what the block does and why. Use examples when possible. Propose ASCII drawings to explain complete systems.
The what _is_ the code.