Design Claude Code

You're asked to design a coding agent. It takes an instruction like "add rate limiting to the API and make the tests pass," then reads the codebase, edits files, runs commands, reads the output, and repeats until it's done.

This agent runs arbitrary commands against a filesystem it can modify. Let's focus on the control layer between model decisions and machine state: tools, permissions, context management, resumability, and verification.

Clarify the requirements

• Where does the code live and where do commands run? Ask first whether execution is local, in a cloud sandbox, or both.
• How large are the repositories? Millions of lines, which means the codebase can never fit in context and retrieval is mandatory rather than optional.
• How autonomous is it? Fully autonomous versus approval-gated changes the safety design completely. Ask where the default sits.
• How long can a task run? Minutes to hours. Long-running sessions need resumability, which a chat product doesn't.
• Is it one agent or several? Start with one well-instrumented loop and treat parallel sub-agents as an optimization, since coordination overhead is real.

Assume: both local and cloud execution, very large repositories, approval gates on destructive operations, long-running resumable tasks.

Back-of-envelope numbers

• Repository: $1\text{M lines} \times 40\text{ B} \approx 40\text{ MB}$ of source, roughly $10\text{M tokens}$, which is orders of magnitude beyond any context window
• A task: $50 \text{ model calls}$ across the loop, each with $30\text{k}$ input tokens after retrieval and truncation
• Token cost per task: $50 \times 30\text{k} = 1.5\text{M input tokens}$, which at typical rates is dollars rather than cents
• Tool calls: $150\text{ per task}$, across reads, searches, edits, and command runs
• Session duration: $5 \text{ to } 60 \text{ minutes}$, long enough that a crash mid-task must not lose the work

Ten million repository tokens won't fit in our context window. The agent must find relevant code rather than receive the whole repository, so let's design tools around navigation and selective reads.

High-level architecture

Components

1. Orchestrator. Owns the loop, the budget, and the session state.
2. Model gateway. Routes to a model tier with retries and usage accounting.
3. Permission layer. Deterministic code deciding which tool calls execute, and which need approval.
4. Execution environment. The developer's machine or a cloud sandbox, with the same tool interface either way.
5. Repository. The working tree the agent reads and modifies.
6. Context builder. Assembles the prompt from the task, the session so far, and retrieved code.
7. Code index. Structural and semantic index over the repository.
8. Session log. Every model call, tool call, and result, kept for resume, review, and audit.

Deep dive 1: designing the tool surface

The tools you give the agent determine what it can do and how efficiently it does it, and the instinct to provide one powerful tool is wrong here.

The set that works is small and composable:

• Search. Find files by name pattern and content by regex. This is the most-used tool by a wide margin, because it's how the agent locates relevant code without reading the repository.
• Read. A file, or a range of lines within it. Range reads matter: a four-thousand-line file shouldn't cost four thousand lines of context when the agent needs one function.
• Edit. Replace an exact string, rather than rewriting the file. This is a deliberate design choice covered below.
• Run. Execute a command and return its output, truncated.

Edit-by-exact-match rather than write-whole-file is worth arguing for explicitly. Rewriting a file means the model must reproduce every line it isn't changing, which costs output tokens proportional to file size and risks silently dropping code it didn't attend to. An exact-match replacement costs tokens proportional to the change, and it fails loudly when the old string isn't found, which is exactly what you want, because a failed match usually means the model's view of the file is stale.

Truncate tool output aggressively. A test suite can emit tens of thousands of lines, and feeding all of it back consumes the context the agent needs for reasoning. Return the head and tail with a marker, and let the agent search the full output if it needs more. Uncontrolled tool output is one of the most common ways an agent loop degrades.

Deep dive 2: working in a repository that doesn't fit

The repository exceeds the context window, so the agent must navigate rather than read everything.

Give it a map of the repository. A directory tree, build configuration, and project documentation cost a few thousand tokens and help the agent form hypotheses. We can combine that map with search, much as a developer approaches an unfamiliar codebase.

Search is retrieval. Regex search over a repository is precise, cheap, and requires no index maintenance, and for code it often beats semantic search, because identifiers are exact tokens. A semantic index adds value for "where is authentication handled," where the answer contains no obvious search term. Offering both, and preferring search for anything with a known symbol, is the practical answer.

Manage the loop's own context growth. Fifty model calls each returning tool output would overflow the window long before the task finishes. The techniques are the same as any long-running agent: summarize completed sub-tasks and drop their intermediate output, keep the original instruction pinned since it's the task definition, and hold the plan and the list of modified files in the orchestrator's own state rather than re-deriving them from the transcript.

Structural indexing helps where text search fails. A symbol index of definitions, references, and the call graph answers "what calls this function" far better than grep, and it's what makes a refactor across many files tractable. It's also expensive to maintain on a large repository, so it's a reasonable thing to name as a scaling improvement rather than a starting point.

Deep dive 3: local versus cloud execution

Support both modes through one tool interface, with different execution properties behind it.

Running on the developer's machine puts execution where the code already is. That means the real environment, with its installed dependencies, credentials, running services, and local database, so commands behave the way the developer expects. The costs are real: every tool call is a round trip from the orchestrator to the machine, the environment is uncontrolled and unreproducible, and the agent has whatever access the developer's user account has, which is usually everything.

Running in a cloud sandbox inverts each of those. The environment is defined and reproducible, tool calls are local to the executor and therefore fast, and the blast radius is a container that can be discarded. The costs are that the repository has to get there, the environment may not match the developer's, and network access has to be granted deliberately, since a container with unrestricted egress plus a model that can be prompt-injected by repository content is a data exfiltration path.

Two design points make the pairing work:

• Identical tool interface. read, search, edit, and run mean the same thing in both, so the agent's behavior doesn't depend on where it runs. Only the transport differs.
• Chattiness decides the boundary. Because the agent makes on the order of a hundred and fifty tool calls per task, keeping the tool executor adjacent to the filesystem matters enormously. Locally that means an agent process on the developer's machine rather than each tool call crossing the network individually; in the cloud it means the executor lives in the sandbox.

Cloud execution enables parallelism that local can't. Several tasks can run in isolated sandboxes simultaneously, each on its own branch, without interfering. That's a genuine capability difference rather than a deployment preference, and it's the strongest argument for supporting both.

Deep dive 4: permissions and not destroying anything

The agent can run arbitrary commands, so the difference between a useful tool and a dangerous one is entirely in the control layer.

Classify operations by reversibility, and put the classification in code rather than in the prompt:

• Read-only. Searching, reading, listing. Always allowed.
• Reversible writes. Editing tracked files in a git repository, where the change is recoverable from version control. Allowed with a visible diff.
• Consequential. Deleting files, git push, installing packages, modifying anything outside the working tree, any network call. Approved explicitly, or restricted to an allow-list.
• Prohibited. Reading credential files, writing outside the project root, disabling the safety layer itself.

Version control is the safety net, and the design should lean on it. Working on a branch means every change is inspectable and revertable, which converts most mistakes from incidents into diffs. An agent operating on a dirty working tree with uncommitted changes is the genuinely dangerous case, and checking for that before starting is a small, high-value guard.

Prompt injection through repository content is the threat that's specific to this system. The agent reads files, and a file can contain text addressed to the agent, whether in a comment, a README, a test fixture, or a dependency's source. Repository content must be treated as untrusted data rather than instructions, and the structural defenses are the ones that don't depend on the model's judgment: permissions enforced outside the model, network egress restricted by default, and approval gates on anything consequential.

Put a hard limit on the loop. Cap the number of steps, the tokens spent, and the wall-clock time, and decide what happens when a limit is hit. For a coding agent, stop and report what was changed and where it got stuck, leaving the branch intact for a human to pick up.

Deep dive 5: verification, and knowing when it's done

An agent that writes code without running it is a code generator. The loop that makes it useful is that it checks its own work.

Run the tests and read the output. After each meaningful change, the agent runs the relevant tests and feeds the result back into the loop. A failing test is information, and iterating against it is the mechanism by which the agent converges instead of merely producing plausible code.

Prefer narrow verification. Running the full suite after every edit is slow and buries the signal. Running the tests for the module just changed, and the full suite once at the end, is both faster and easier for the agent to interpret.

Completion needs a definition, not a feeling. "The tests pass and the change matches the request" is checkable; "the model thinks it's finished" is not. Where an objective check exists, whether tests, a type checker, a linter, or a build, the agent should be required to satisfy it before reporting success, and should report honestly when it can't.

Persist the session as it goes. Every model call, tool call, and result appended to a log means a crash resumes rather than restarts, a developer can review exactly what happened, and a task that went wrong can be diagnosed from evidence rather than reconstructed. For a system that runs for an hour and modifies real files, that record is what makes it trustworthy.

Common pitfalls

• Loading the repository into context. It's orders of magnitude too large; the agent must search.
• Whole-file rewrites instead of targeted edits. Cost scales with file size and untouched code gets silently dropped.
• Permissions expressed in the prompt. Prompt instructions are suggestions; the permission layer is the enforcement.
• Treating repository content as trusted. Files can contain instructions aimed at the agent.
• No verification step. Without running anything, the agent produces plausible code rather than working code.

Leveling signals

Related lessons

Design ChatGPTHard

Serve a multi-turn conversational product backed by a large language model.

Design an AI-Powered Customer Support SystemMedium

Answer customer questions from company data and take actions on their accounts.