Why start with a task?
“Build an agent” is too broad to make useful design decisions. “Given a repository and a failing test, identify a likely cause, make a minimal change, and run that test in a sandbox” is specific enough to ask what context, tools, permissions, and success checks are needed.
Do not assume every task needs an autonomous loop. If the steps are fixed and predictable, a normal workflow may be cheaper and easier to debug. Anthropic’s guide recommends starting with the simplest architecture that meets the task’s needs and adding agentic behavior when flexibility is valuable.
Write a task contract
Task: Fix one failing unit test in the provided repository.
Inputs: Repository snapshot, test output, user request.
Allowed actions: Read project files, edit files in the sandbox, run the selected test.
Forbidden actions: Network access, secret access, changes outside the workspace.
Success: Target test passes; diff is limited and inspectable.
Stop and ask: The change requires deleting data, changing credentials, or expanding scope.
This contract does two jobs. It tells you what capabilities the harness needs, and it gives you the basis of a test: restore the same repository snapshot, run the same request, and check the same observable outcome.
Choose an observable outcome
Prefer “the selected test command exits successfully” over “the agent gives a good answer.” Prefer “the created record has these fields” over “the agent appears to have saved it.” Some tasks need judgment, but make the objective portion mechanically checkable first. Record subjective review separately.
Now make a simple capability map:
| Need | Capability | Limit |
|---|---|---|
| Inspect a failing test | Read files and run tests | Workspace sandbox; selected paths |
| Change the project | Write patch | No network; diff review before merge |
Ask your teacher to review the scope of a task you want your own harness to handle.