Assignment
Prepare twenty tasks covering normal completion, invalid arguments, missing evidence, rejected approval, injected instructions, and interrupted execution.
Build sequence
- Record structured events for each state transition and tool call.
- Write deterministic checks for unauthorized actions and budget overruns.
- Replay known failures with fixed observations, then rerun model-dependent cases separately.
- Compare a change against the previous version using case-level outcomes.
Acceptance checks
- Critical boundary failures block release regardless of average score.
- Replayed and live results are labeled separately.
- Every discovered failure has a regression case and an identified owning component.
Evidence to keep
Submit the working artifact or decision document described above, the inputs used, and the observed results for every acceptance check. Include one failed attempt and the change you made after investigating it. Label fixture-based outcomes separately from live-system measurements. These artifacts become part of your final portfolio review.