The project
Build a push-to-talk field assistant with visible session state, interruption support and one read-only lookup tool.
Prerequisites
Complete the six teaching modules and their exercises. Keep the design notes and failure cases: they form the starting evidence for this project.
Build in increments
1. What a Live API session is
Understand the difference between a persistent realtime session and a single generation request. Apply the module decision to this project. Your checkpoint: Your design separates connection state from microphone and playback state.
2. Microphone capture and audio formats
Prepare input that the service can actually interpret. Apply the module decision to this project. Your checkpoint: You can state the exact format of every audio buffer at the transport boundary.
3. Playback and turn completion
Produce coherent output without racing the receive loop. Apply the module decision to this project. Your checkpoint: Playback resources are released on stop, cancellation and connection failure.
4. Interruptions and voice activity
Make the assistant stop speaking when the user takes the floor. Apply the module decision to this project. Your checkpoint: An interrupted response cannot resume due to a late network event.
5. Live tools and session recovery
Keep realtime convenience from creating duplicate or unauthorized actions. Apply the module decision to this project. Your checkpoint: Recovery does not silently duplicate an externally visible action.
6. Camera context and realtime evaluation
Add media only when it improves the actual task. Apply the module decision to this project. Your checkpoint: You can demonstrate media consent, freshness and a measurable benefit from the camera input.
Acceptance criteria
- Measure first-audio latency and interruption stop time
- Stop microphone and playback on session exit
- Recover from a dropped connection without repeating a side effect
Evidence to submit
- A repository or reproducible design artifact with setup instructions and tested tool versions.
- A diagram showing components, data flow, authority and failure boundaries.
- An evaluation table with inputs, expected results, actual results and explanations of failures.
- A demonstration of the normal path and at least two relevant failure paths.
- A short decision record covering alternatives, known limitations and the next improvement.
Review rubric
| Dimension | Ready to demonstrate | Revisit when |
|---|---|---|
| Understanding | Explain each boundary and why it exists | You can only repeat framework terminology |
| Implementation | Reproduce the core behavior from a clean setup | Hidden local state is required |
| Reliability | Show bounded failures and useful recovery | The demo only works on the happy path |
| Evidence | Present observed outcomes and limitations | Success is inferred from a generated answer |
| Ownership | Modify a requirement and explain the consequences | You cannot adapt the implementation independently |
Do not treat a completed reading checklist as proof of project readiness. Revisit any module whose checkpoint you cannot demonstrate.