Assignment
Use a local feature flag and three synthetic user cohorts. Simulate normal traffic, a connector outage, and a drop in draft acceptance.
Build sequence
- Define entry, expansion, pause, and rollback criteria.
- Enable the smallest cohort and record technical and task metrics.
- Trigger a dependency outage and disable affected functionality.
- Draft a concise user update explaining impact and the available fallback.
Acceptance checks
- Disabling AI leaves the core manual workflow available.
- The cohort cannot expand while hard gates fail.
- The pilot review uses observed data and states its sample size.
Evidence to keep
Submit the working artifact or decision document described above, the inputs used, and the observed results for every acceptance check. Include one failed attempt and the change you made after investigating it. Label fixture-based outcomes separately from live-system measurements. These artifacts become part of your final portfolio review.
Guided workshop: expand only after observing the previous stage
Roll Northstar out in three exercise stages: offline evaluation, shadow mode, and assisted drafting for two operators. Shadow mode computes candidate drafts without showing or sending them; its data use still requires authorization. Assisted mode shows drafts for human review and preserves manual support.
Define entry and exit gates for each stage. A stage needs a named owner, eligible traffic, observation window, success measures, stop conditions, and recovery action. “Enable for everyone after the demo” skips the evidence the pilot is meant to produce.
Worked rollout ledger: stage 1 passes all access tests and meets the held-out grounding target. Stage 2 shows that 30% of tickets lack current policy evidence. Narrow stage 3 to the supported ticket categories and assign a data-remediation owner rather than hiding those unsupported cases from the report.
Exercise: implement a server-side feature flag by tenant and operator in the sandbox. Test the disabled path, one enabled operator, another operator in the same tenant, and a different tenant. The flag controls availability; it does not replace authorization. A user cannot enable themselves by changing client state.
Stop drill: inject a citation regression after enabling two operators. Disable drafting, preserve access to search or the manual workflow, and confirm new requests do not reach generation. Inspect in-flight requests separately: disabling admission may not cancel already-started work.
Rollback review: reverting code is one recovery mechanism. Disabling a feature, changing traffic routing, restoring an index version, and reconciling pending jobs are different actions. Match the response to the failure and preserve evidence for diagnosis.
Assessment: submit a rollout plan and an executed drill showing the flag state, request outcomes, operator message, and recovery verification. Do not claim a simulated two-person pilot establishes general adoption or production reliability. State exactly which population and workload were tested and what evidence is still needed before expansion.