Collect enough evidence to operate the feature without over-collecting user data.
1. Correlation and redaction
Use request and operation IDs to connect mobile, backend and tool events. Log timings and outcome categories while redacting raw private content and credentials.
2. Cost per outcome
Track total work including retries, retrieval and failed requests. Cost per successful task is often more useful than cost per model call.
3. Retention and deletion
Define retention for prompts, responses, traces and cached documents. User deletion should account for derived storage, not only the primary application record.
Worked scenario
A cheap model produces many repair attempts. Its cost per valid extraction exceeds a more capable model used once.
Apply it
Create a telemetry schema and review each field for operational need and privacy impact.
Check your understanding
You can diagnose a failed task without routinely storing its full sensitive payload. Explain the decision and show evidence from your implementation or design. If you cannot demonstrate it yet, revisit the relevant section before continuing.