Benchmark the actual deployment configuration.
1. CPU and accelerators
Available backends differ by runtime, model and hardware. A GPU or NPU label does not guarantee faster execution for every workload. Compare supported backends on representative devices.
2. Context growth
Longer inputs and conversation state increase work and memory. Bound input size and measure latency as context grows, not only for a one-word prompt.
3. Thermal and battery limits
Sustained inference can throttle the device and affect battery. Test repeated runs and foreground interaction together; an isolated benchmark may conceal user-visible jank.
Worked scenario
A backend wins a short warm benchmark but slows under sustained use. The app selects a stable configuration based on the real interaction pattern.
Apply it
Benchmark short, medium and maximum accepted inputs, including repeated requests and UI responsiveness.
Check your understanding
Your performance claims include device, backend, model and workload. Explain the decision and show evidence from your implementation or design. If you cannot demonstrate it yet, revisit the relevant section before continuing.