9 Rules for Coding with LLMs in Production
After 2.5+ years of shipping commercial software with LLMs as pair programmers and autonomous agent harnesses (building 50+ production systems), we have codified 9 fundamental rules for engineering leaders.
Typed JSON schemas, Pydantic models, and strict TypeScript types prevent 95% of downstream hallucinations.
Never ask an LLM to plan, code, and verify in a single prompt. Split tasks into distinct Observe-Orient-Decide-Act loops with validation gates.
Store every model invocation, prompt template version, and tool input in an append-only ledger for post-incident debugging.
Use deterministic AST parsers, linters, and compilers to verify code before burning API tokens on LLM-based code review.
Agents should only have read access until explicit cryptographic approval is granted for mutating production databases or deploying code.
Context older than 60 days must be explicitly re-verified against the active architecture graph before being fed to agent reasoning loops.
Chat transcripts are lossy. Operational facts must compile into discrete database nodes with explicit causal links.
Run daily regression benchmark evaluations on your agent prompts using DSPy to detect model drift across upstream provider updates.
The goal of software engineering with AI is not to generate 10x more code, but to ship 10x more value with zero cognitive noise.