The control loop
- Build a model request from the user task, instructions, relevant state, and the allowed tool schemas.
- Receive either a final answer or one or more proposed tool calls.
- Validate each call: known name, valid arguments, permitted action, remaining budget.
- Run the tool with a timeout and capture its structured result or safe error.
- Append the assistant call and matching tool result to history; ask the model for its next turn.
- Stop on final answer, cancellation, policy denial, error budget, time limit, or turn limit.
Minimum boundaries
- Allowlist tools; never execute a name merely because the model requested it.
- Validate tool inputs in code and enforce user permissions outside the prompt.
- Require confirmation for sensitive, irreversible, or externally visible actions.
- Set a maximum number of turns, a wall-clock deadline, and a resource budget.
- Treat tool outputs and retrieved documents as untrusted data; redact secrets from logs.
- Retry only when the operation is safe to repeat or carries an idempotency key.
Evaluation record
| Field | Example |
|---|---|
| Task and environment | Prompt, fixture, repository snapshot, permissions |
| Configuration | Model, prompt version, tools, MCP servers, limits |
| Outcome | Validator result, task success, partial completion |
| Process | Tool calls, errors, retries, final artifact |
| Cost and time | Tokens, tool calls, wall-clock latency, model cost |
See Lesson 4: Evaluate the whole harness and Lesson 9: Test and improve your harness.