Assignment
Create read_ticket and propose_update tools over an in-memory sandbox, with a durable operation store simulated by SQLite.
Build sequence
- Validate names and arguments before execution.
- Represent error categories explicitly and limit retries for transient failures.
- Simulate a timeout after a mutation commits.
- Replay the same operation ID and verify its result is reused.
Acceptance checks
- Unknown tools and unauthorized actors never reach business logic.
- A post-commit timeout does not create duplicate mutations.
- Traces identify validation, permission, transport, and domain failures separately.
Evidence to keep
Submit the working artifact or decision document described above, the inputs used, and the observed results for every acceptance check. Include one failed attempt and the change you made after investigating it. Label fixture-based outcomes separately from live-system measurements. These artifacts become part of your final portfolio review.
Guided workshop: build a bounded loop without a framework
A model can propose a tool call but must not dispatch arbitrary code. Use an allowlist, argument validation, a trusted principal, and an explicit step budget. This deterministic example uses scripted decisions to test the runtime independently of a model.
# academy-check
def run(decisions, principal, max_steps=3):
trace = []
for step, decision in enumerate(decisions):
if step >= max_steps:
return "budget_exhausted", trace
if not isinstance(decision, dict):
return "invalid_decision", trace
name = decision.get("tool")
if name == "finish":
return "completed", trace
if name != "search_docs":
return "unknown_tool", trace
args = decision.get("args")
if not isinstance(args, dict) or set(args) != {"query"}:
return "invalid_arguments", trace
if not isinstance(args["query"], str) or not 1 <= len(args["query"]) <= 100:
return "invalid_arguments", trace
if principal.get("can_search") is not True:
return "forbidden", trace
trace.append({"tool": name, "status": "ok", "ids": ["guide-1"]})
return "incomplete", trace
search = {"tool": "search_docs", "args": {"query": "reset"}}
assert run([search, {"tool": "finish"}], {"can_search": True})[0] == "completed"
assert run([search] * 4, {"can_search": True})[0] == "budget_exhausted"
assert run([search], {"can_search": False})[1] == []
assert run([{"tool": "shell"}], {"can_search": True})[0] == "unknown_tool"
The trace records normalized observations, not secrets or a hidden reasoning transcript. completed means the runtime accepted a finish decision; a separate evaluator must still determine whether the task was solved correctly.
Exercise: replace the fixture search with your authorized retrieval function. Introduce fake timeouts and invalid arguments, then prove those calls do not bypass the guard. Add a total deadline and a maximum observation size so tool output cannot consume unbounded context.
Worked extension: maintain remaining steps and remaining cost in trusted state. Pass a summary to the model, but never accept a model-provided larger budget. Parallel read calls may be useful when independent; writes require their own conflict and approval semantics.
Checkpoint: show unknown-tool rejection, permission rejection, malformed arguments, exhausted budget, and explicit completion. Connect a real model only through an adapter that returns the same decision contract. Keep deterministic runtime checks and live decision-quality evaluations separate.