Make prompt changes reviewable and production behavior bounded.
1. Model compatibility
Check supported capabilities before switching models. Run the evaluation suite for each candidate model and keep configuration rollback available. Similar model names do not guarantee identical output behavior.
2. Quotas and retries
Retry only recoverable errors with bounded exponential backoff and jitter. Respect server retry guidance. Do not blindly replay a write after an ambiguous timeout.
3. Quality and cost
Track success per task, invalid outputs, refusal outcomes and cost per successful result. A cheaper request that requires several retries may cost more overall.
Worked scenario
A model upgrade improves prose but doubles invalid itinerary responses. The release gate should catch the schema regression before rollout.
Apply it
Compare two prompt configurations on held-out travel requests and write a rollback decision from the results.
Check your understanding
You can defend the selected configuration with task-level evidence. Explain the decision and show evidence from your implementation or design. If you cannot demonstrate it yet, revisit the relevant section before continuing.