IIT Gandhinagar · Agentic Engineering

Build Your Own Coding Harness

From one API call to a safe multi-tool coding agent

Arnav Gupta · Lecture Deck

What is a coding harness?

Core idea

A coding harness is the runtime around the model.

keeps conversation and repo state
exposes tools, shell access, and file operations
applies policy, permissions, and budgets
verifies results against the environment

The model decides; the harness does.

Three boxes to remember

Model Harness Environment

flowchart TD U[User Goal] --> H[Harness] H <--> M[LLM] H <--> E[Files · Shell · APIs]

Model: plans the next step
Harness: runs actions and enforces rules
Environment: provides reality and feedback

Synthesis grounded in OpenAI Responses/function-calling docs and Anthropic's harness/tool engineering posts.

Build path for today's harness

Step 1

One call

Prompt in, text out.

Step 2

Chat loop

Add state and repetition.

Step 3

One tool

Image question answering.

Step 4

Many tools

ls, read, edit, bash.

Step 5

Design choice

MCP vs codemode.

Repeated pattern

show the architecture
trace one turn with a timing diagram
highlight the code we added

Practical constraint

We keep the harness intentionally small enough that you could build it in a weekend and still understand every moving piece.

The point is not vendor lock-in. The point is the reusable control-loop pattern.

Step 1 — one OpenAI call

Architecture

flowchart TD U[Prompt] --> A[Small App] A --> API[Responses API] API --> A A --> OUT[Print output]

Timing

sequenceDiagram autonumber actor User participant App participant API as OpenAI API User->>App: "Write a haiku about debugging" App->>API: responses.create(model, input) API-->>App: output_text App-->>User: render text

What exists so far?

Only a request-response wrapper. No memory. No tools. No environment. No verification.

Source: OpenAI Responses API overview.

Step 1 — minimal pseudocode


from openai import OpenAI

client = OpenAI()

response = client.responses.create(
    model="gpt-5",
    input="Write a haiku about debugging"
)

print(response.output_text)

Walkthrough mode

Advance through the slide: the full code stays visible, while the active lines and matching explanation come into focus.

Create the client

The harness first creates a client object that knows how to talk to the model API.

Send one request

This is the entire one-shot harness: choose a model, send one input, and wait for one response.

Render the answer

The app prints the model's output text. At this stage, there is still no memory, no tools, and no verification.

Treat this as the irreducible base case for the whole lecture.

Step 2 — turn it into a chat looper

Architecture

flowchart LR U[User input] --> H[Chat history] H --> API[Responses API] API --> R[Assistant reply] R --> H H --> U

Timing

sequenceDiagram autonumber actor User participant Chat as Chat loop participant API as OpenAI API loop each turn User->>Chat: next message Chat->>Chat: append to history Chat->>API: responses.create(history) API-->>Chat: assistant text Chat->>Chat: append reply Chat-->>User: print reply end

New idea

A chatbot is not a different kind of model call. It is just a loop plus saved state.

Source: OpenAI Responses API conversation/state mental model.

Step 2 — code delta: add history


history = [
    {"role": "system", "content": "You are a helpful assistant."}
]

while True:
    user_text = input("> ")
    history.append({"role": "user", "content": user_text})

    response = client.responses.create(
        model="gpt-5",
        input=history,
    )

    print(response.output_text)
    history.append({"role": "assistant", "content": response.output_text})

Walkthrough mode

The chat version adds only one big idea: preserve state and keep looping.

Start with history

The harness owns the transcript. The system message seeds the assistant's behavior.

Add a turn loop

Each user turn is appended to history before the next model call.

Send the whole context

The model only gets memory because the application resends the accumulated history.

Store the reply

After rendering the answer, save it too, so the next turn can refer back to it.

A chat harness is just state plus repetition.

Step 3 — add one tool: inspect an image

Architecture

flowchart LR U[Question about screenshot.png] --> H[Harness] H --> M[LLM + tool schema] M -->|inspect_image call| H H --> V[Vision helper] V --> IMG[(Image file)] V -->|tool result| H H --> M M --> A[Final answer]

Timing

sequenceDiagram autonumber actor User participant H as Harness participant M as Model participant T as inspect_image tool User->>H: "What error is visible in screenshot.png?" H->>M: input + tool schema M-->>H: tool call inspect_image(...) H->>T: run tool with path and question T-->>H: observed error text H->>M: tool result M-->>H: final answer H-->>User: print answer

Why this is a great first tool

It has the exact same loop shape as later tools like read_file, run_tests, or bash, but with fewer destructive side effects.

Sources: OpenAI function-calling guide; OpenAI images and vision guide.

Step 3 — code delta: add tool calling


TOOLS = [{
    "type": "function",
    "name": "inspect_image",
    "parameters": { ... }
}]

response = client.responses.create(
    model="gpt-5",
    input=items,
    tools=TOOLS,
)

for call in response.tool_calls:
    result = inspect_image(
        call.arguments["path"],
        call.arguments["question"],
    )
    items.extend([call, tool_result(call, result)])

Walkthrough mode

This is the key transition from chat to agency: the model can now ask the harness to do structured work.

Declare the tool

The schema tells the model what the tool is called and what arguments it expects.

Expose it to the model

Passing tools=TOOLS means the next response may include a tool call instead of only plain text.

Execute the requested action

The harness reads the arguments, runs the tool, and turns the outside world into an observation.

Feed the result back

The loop continues only after the tool result is appended and returned to the model.

Tool use is structured I/O plus a control loop.

Step 4 — expand to a real coding surface

Architecture

flowchart LR U[Bug report or coding task] --> H[Harness loop] H --> M[LLM planner] M -->|ls / read_file / edit_file / bash| H H --> G{policy gate} G --> L[ls] G --> R[read_file] G --> E[edit_file] G --> B[bash] B --> ENV[(Repo · Tests · Shell)] E --> ENV R --> ENV L --> ENV ENV -->|results| H H --> M

Timing

sequenceDiagram autonumber actor User participant H as Harness participant M as Model participant Env as Tools + Repo User->>H: "Fix the failing test" H->>M: task + tool schemas M-->>H: ls(".") H->>Env: ls Env-->>H: files H->>M: tool result M-->>H: read_file("app.py") H->>Env: read_file Env-->>H: source code H->>M: tool result M-->>H: bash("pytest -q") H->>Env: run tests Env-->>H: failing trace H->>M: tool result M-->>H: edit_file(...) H->>Env: apply edit Env-->>H: diff preview

This is the smallest tool surface that still feels like a practical coding assistant.

Step 4 — code delta: dispatch multiple tools


def run_tool(name, args):
    if name == "ls":
        return ls(args["path"])
    if name == "read_file":
        return read_file(args["path"])
    if name == "edit_file":
        return edit_file(args)
    if name == "bash":
        return run_bash(args["command"])
    return {"ok": False, "error": "unknown tool"}

def guarded_tool_call(call):
    check_policy(call.name, call.arguments)
    result = run_tool(call.name, call.arguments)
    return normalize_result(result)

Walkthrough mode

This slide turns one tool into a small coding platform: dispatch, gate, execute, then normalize the observation.

Start with a dispatcher

run_tool is the switchboard that maps tool names to actual host-side executors.

Keep the tool surface explicit

Each branch is a named capability. This keeps the action surface small enough to inspect and reason about.

Check policy before execution

The harness should decide whether the call is allowed before any tool actually touches the environment.

Normalize the result

Return a predictable shape so the model reasons over structured observations instead of executor noise.

Anthropic's tool-design guidance strongly supports small, clear tool interfaces and predictable result formats.

Minimal tool contract for a coding harness

Structured tools

ls(path)
read_file(path)
edit_file(path, old_text, new_text)

Great for common actions that should stay narrow, typed, and easy to audit.

General tool

bash(command)

Powerful escape hatch for tests, package managers, formatters, scripts, and git.

Field	Why the harness should return it
`ok`	Uniform success/failure signal across every tool
`stdout` / `stderr`	Lets the model reason over actual observations
`exitCode`	Separates command failure from command output
`diffPreview`	Gives edits a compact, inspectable summary

A small, well-shaped tool contract is better than a huge uncontrolled action surface.

MCP vs codemode: two philosophies of tool access

Question	MCP-style tools	Codemode / sandboxed execution
What can the model do?	Call a finite set of declared tools	Write arbitrary code or commands within the sandbox
Main trust boundary	Tool/server boundary	Runtime / VM / container boundary
Typical safety controls	schemas, auth scopes, validation, consent, logging	filesystem/network isolation, approvals, mounts, guardrails
Best use case	external systems like GitHub, Slack, cloud APIs	local repo work, scripts, tests, open-ended procedures

Teaching metaphor for MCP

A control panel of labeled buttons.

Teaching metaphor for codemode

A workshop inside a locked room.

Sources: MCP specification/docs; OpenAI Codex security, sandbox, and approval docs. Comparative phrasing is synthesis.

Where does safety live?

MCP-style boundary

flowchart LR M[Model] --> H[Host] H --> S[MCP server] S --> X[External API] N[Safety focus] --> H N --> S

server exposes the tool menu
server validates inputs and auth
host decides whether to allow the call

Codemode boundary

flowchart LR M2[Model] --> R[Sandbox runtime] N2[Safety focus] --> R R --> FS[Workspace files] R --> SH[Shell or code] R --> NET[Network]

the model invents procedures on the fly
the runtime decides what the process can touch
approvals and isolation replace a narrow tool menu

Practical recommendation

Use codemode to think and work locally. Use MCP-style tools to touch real external systems through narrower authenticated interfaces.

Real harnesses often combine both models rather than choosing exactly one.

If you were building this for real


mini-harness/
├── main.py              # event loop
├── llm.py               # model calls
├── tools.py             # schemas + dispatch
├── policy.py            # approvals, limits
├── state.py             # transcript/history
├── exec.py              # bash, file ops, image tool
└── verify.py            # tests, lint, assertions

Keep responsibilities separate

llm.py: only speaks model API
tools.py: declares and dispatches tools
policy.py: decides what is allowed
exec.py: touches the environment

Course-friendly scope

This is still small enough for students to read end-to-end and test with a few local tasks.

The educational goal is transparency, not industrial complexity.

Build checklist: what makes it a harness?

Must-have pieces

clear system prompt
small tool surface
standardized tool results
state/history management
policy checks before execution
verification after mutation

Common failure modes

letting transcripts grow without compaction
mixing tool execution with policy logic
returning messy tool output with no structure
granting shell power without sandbox or approvals
stopping after action without verification

call → observe → act → verify → repeat

Verification is not a luxury feature. It is part of the architecture.

Takeaway

You can build the harness incrementally

Start with one call. Add state. Add one tool. Add many tools. Then decide where the trust boundary belongs.

Next step: turn this into a lab scaffold