Documentation hell

Documentation hell

I have been doing a LOT of vibe coding lately.

The Claude mobile app has much better (still not perfect) support for Claude Code now, so I can keep multiple agents going from my phone all day long. It is properly addictive. Work projects, personal projects, throwaway games for my kids that get played for an afternoon and forgotten about...

My wife now reckons she's a Claude widow. 🤣

The small stuff is great. The big stuff though... Once a project gets to around 50k lines, Claude seems to over document the project into a state that I've started calling documentation hell. So I wrote a skill to get out of it, compact-docs.

I find documentation grows way too fast with a vibe coded project. Every agent that touches the repo leaves a bit more prose behind than it found, but hardly ever deletes any of it:

This is all empirical (I have no evals) but under documentation hell I notice:

In a time before decent AI coding tools, I wrote (yet another) Game Boy emulator in Rust: gb. I was happy with the documentation as it was hand rolled. I then (mostly) vibe coded a text interface over the top of it, so that an LLM could play Pokémon Red.

When I measured it, there were 67k lines of Rust and 33k lines of comments. One comment line for every two lines of code in a project where the core of it was hand rolled!

The longest single comment block was 205 lines (twice as long as this article), around 800 comments cited a plan and the markdown had 279 ⚠️ and 70 ⭐ in it.

I find that the more cluttered with documentation a large project gets, the more weird language creeps in. I don't know how many times I've read something like "a [thing] nobody asked for". I call it the nobody tic. Here's some examples taken from vibe coded commit messages, documentation and code comments in gb:

My throwaway kids game make-believe has the same "nobody" tic EVERYWHERE I LOOK:

Other (non-nobody) funnies I found:

The problem is that the more of this garbage you have in your code base, the more the LLM will vomit it back at you in response to a prompt.

I've been asking Claude to clean up documentation with bespoke prompts for a while now but the last time I did it in the gb repo, it turned out that a lot of it shouldn't exist. Here's the before and after from that first pass, which only looked at the Rust:

MeasureBeforeAfter
Comment lines33,11911,060
Comment / code ratio0.490.17
Comment blocks >= 20 lines1930
Longest comment block20512
Comments citing a plan~8000
Markdown lines6,2591,083

I thought this worked well so I asked the agent to distil what we'd done into a global skill. It's a SKILL.md and a measure.py with no dependencies. The script counts code, comment and markdown lines in every language in the repo, only across files git doesn't ignore so node_modules and target never count, and reports the worst files, the longest blocks and the noise, i.e. plan ids, dates, "used to" and emoji markers. I checked the measurement script against eight of my other projects covering stacks I use for work and personal projects (Python, React, Vue, .NET, Next.js, node).

The skill then runs a loop:

  1. Measure and save the baseline.
  2. Prune code and files that should not exist.
  3. Rewrite every comment and document to one of two outcomes:
    • It's junk, delete it.
    • It's too verbose, replace it with a plain statement.
  4. Re-measure against the baseline.
  5. Repeat 3 and 4 until the targets hold.

The targets are pass/fail, e.g. comments at most 0.15 of code, no comment block over 12 lines, markdown at most 0.05 of code and zero plan references.

The skill finishes by leaving a short set of ground rules in CLAUDE.md so the next agent doesn't undo it all. One of my favourites, and the one with the most immediate effect, is that plans live in docs/PLAN-*.md, are excluded from git and are deleted when they're done, and nothing outside a plan may cite it.

I ran the finished skill over the gb repository and it found a load more. The first pass had been Rust focussed, but the skill measures every language, so the k8s config, CI and the web UI had barely been touched, e.g. the k8s configmap went from 180 comment lines to 20. Over the whole repo the ratio went from 0.18 (the 0.17 above was Rust only) to 0.11 and another 262 plan ids, dates, history and ⚠️ came out of the comments.

The pass also found about 40 doc comments attached to the wrong item or describing code that no longer existed, and some actual bugs. Five string literals had lost their backslash line continuation, so a run of indentation had become part of the text, one of them in a tool description sent to the LLM on every turn.

The skill says not to fix logic in a comment pass, so these went into a separate plan and another agent fixed them.

I then ran it over make-believe:

MeasureBeforeAfterTarget
Comment lines / code lines0.34 (5,730 lines)0.07 (1,184)≤ 0.15
Worst single file6.570.29≤ 0.30
Comment blocks over 12 lines4800
Markdown / code0.05 (896 lines)0.01 (213)≤ 0.05
Milestone/play-test refs, history words, glyphsmany00

Vibe coding in these clean projects seems more responsive, faster and more successful, but again I have no evals, I'm just going on the vibes.

The skill is on GitHub at axle-h/skills, copy it into ~/.claude/skills/ and ask your agent to compact your docs. If you give it a go I'd love to know what ratio you started on 😀.