Assignment
Package your knowledge assistant or a fake-backed version in a reproducible local environment. Use synthetic documents and a staging configuration.
Build sequence
- Write startup checks, a health endpoint, and a version manifest.
- Create a pipeline that runs contract tests and the evaluation scorecard.
- Release a deliberately incompatible retrieval configuration in staging and detect the failure.
- Restore the previous compatible release and write a short incident review.
Acceptance checks
- A clean setup can run one successful request from the README.
- Rollback restores both application behavior and retrieval compatibility.
- The incident review identifies detection, impact, recovery, and a prevention change.
Evidence to keep
Submit the working artifact or decision document described above, the inputs used, and the observed results for every acceptance check. Include one failed attempt and the change you made after investigating it. Label fixture-based outcomes separately from live-system measurements. These artifacts become part of your final portfolio review.
Guided workshop: rehearse a release with a known bad candidate
Treat a release as an immutable set of application, model, prompt, retrieval, and schema versions. A source commit alone does not capture a changed prompt stored in configuration or a newly rebuilt document index.
Create two local release directories, release-a and release-b, each with a manifest and the same test command. Release A should pass. In B, deliberately break unsupported-question handling. Your release pipeline should run deterministic tests, evaluate the held-out cases, compare gates, and refuse to advance the active version.
# academy-check
releases = {
"a": {"schema": 1, "access_failures": 0, "unsupported_failures": 0},
"b": {"schema": 1, "access_failures": 0, "unsupported_failures": 2},
}
def promote(current, candidate):
report = releases[candidate]
if report["access_failures"] or report["unsupported_failures"]:
return current
return candidate
assert promote("a", "b") == "a"
The example illustrates a release decision, not a deployment system. In your actual project, store candidate artifacts immutably and update the routing or deployment configuration only after checks pass. Restrict who can promote, and preserve the previous version for recovery.
Operational rehearsal: define /health for process health and a readiness check for serving requests under the service's documented dependency policy. Kill the model connection, disable generation, and confirm that search-only or manual support remains available if that is your chosen fallback. Do not report readiness merely because the process exists when it cannot serve its advertised contract.
Migration exercise: add a nullable metadata field while keeping A able to read records written by B. Deploy the compatible schema before switching writers. Roll B back to A and verify reads still work. A destructive schema migration may make application rollback insufficient; require a tested data recovery or forward-fix plan.
Runbook: include release identifiers, configuration names, secret locations without values, health checks, alert ownership, disable switch, rollback command, database backup and restore procedure, and verification requests. Have another learner follow it without hints.
Acceptance: demonstrate one successful release, one blocked candidate, one rollback, one dependency outage, and one restored backup in an isolated environment. Save the evidence with timestamps. Only label a deployment verified when you actually ran those checks in the target environment; local exercises establish the workflow, not cloud readiness.