Simulate a customer engagement for a small support team. Its staff spend time locating policy documents and drafting ticket responses. Your task is to discover whether AI helps, build a bounded pilot, and hand it over. Use synthetic tickets and clearly label the engagement and business results as simulated.
Discovery brief
Interview a peer acting as the support lead, or write explicit assumptions when no participant is available. Record users, current steps, frequency, pain points, data owners, permissions, and excluded actions. Measure a small manual baseline with five example tickets. Separate measured timings from estimates.
Agree on pilot criteria before building: grounded response quality, successful permission checks, human review requirements, acceptable latency, and a budget. The initial pilot drafts answers; a person remains responsible for sending them.
Architecture and integration
Draw the flow from authenticated user through the application, ticket API, document retrieval, and model provider. Describe where data is stored and which identity checks each boundary enforces. Compare a search-only alternative with retrieval-assisted drafting. Explain why the proposed complexity is justified.
Use two synthetic customer accounts and distinct document collections. Ensure the requesting identity limits retrieval and ticket access before information reaches the model. Do not rely on a prompt to enforce tenant isolation.
Pilot delivery
- Implement a sandbox connector with explicit timeout and retry behavior.
- Create representative acceptance cases with the simulated customer, including unsupported questions and inaccessible documents.
- Run a supervised pilot, collect feedback, and record failed cases rather than removing them from the report.
- Test disabling model calls while keeping manual support usable.
- Rehearse a handoff with someone who did not build the system.
Acceptance checks
- An account cannot retrieve another account’s documents or read its tickets, including when IDs are guessed.
- Provider outages return a usable fallback and a diagnosable trace without sensitive data.
- The pilot report compares the agreed criteria with results and clearly identifies the sample size.
- Rollback instructions can be followed by the receiving operator.
- The handoff recipient can configure the sandbox, run a sample request, identify a failed integration, and disable the AI feature using the runbook.
Deliverables
Submit a discovery brief, architecture decision record, integrated prototype, acceptance matrix, pilot review, and operational runbook. Include ownership, monitoring, escalation, rollback, and user training. Present a short recommendation: expand the pilot, revise it, or stop it, with supporting evidence.
This exercise demonstrates the delivery process. It does not establish real customer ROI or certify production readiness.
Guided workshop: complete the Northstar delivery simulation
Carry the same Northstar Support case through discovery, architecture, prototype, security review, demo, deployment planning, incident response, and executive readout. Reuse the technical foundation from the shared knowledge service, then adapt it to the customer's explicitly agreed scope.
Starting brief: six operators handle two synthetic tenants, Alder and Birch. Policies are partly outdated, the ticket adapter can time out after a write, and the sponsor asks for automatic resolution. The security owner permits draft-only assistance with human review. Your first deliverable is a reconciled problem statement, not an automatic agent.
Prepare a delivery dossier in this order:
- Discovery notes, baseline plan, assumption register, and exclusions.
- Data inventory, current-policy decisions, and permission owners.
- Two-option architecture decision and integration contracts.
- A working bounded prototype using synthetic data.
- Threat model, access matrix, and negative-test evidence.
- Customer demo, operator instructions, and unresolved objections.
- Frozen UAT cases, outcomes, and remediation ownership.
- Rollout gates, disable procedure, operational owner, and handoff.
- Incident timeline, recovery verification, and corrective work.
- Executive recommendation with measured, estimated, and simulated results clearly distinguished.
Change inject: after the prototype, request CRM writes and automatic refunds. Explain the change in authority, risk, cost, and evaluation scope. Produce a revised plan that either defers those actions or defines a separate approved work package. Do not quietly expand the original pilot.
Incident inject: a partially completed index update serves an outdated policy. Pause the affected behavior, preserve manual support, communicate known impact, restore a verified version, and retest. Record how you prevent queued work or stale caches from reintroducing the failed behavior.
Worked final decision: if supported-ticket handling improves but data freshness and adoption remain weak, recommend a narrower continued pilot with named remediation owners. If the intervention adds review effort without improving useful outcomes, recommend stopping or returning to search-only support. Delivery success includes an evidence-based decision against unnecessary AI complexity.
Scoring: award 0–2 each for discovery, architecture, integration, security, evaluation, and handoff. Require at least 10 of 12 and full marks for security and handoff. A fluent presentation cannot compensate for an operator who cannot disable or recover the system. Have a peer challenge the decision and operate the sandbox using only the dossier. When working alone, label the role-play and self-review rather than claiming external customer signoff.