An AI service still receives ordinary software inputs: strings, JSON objects, status codes, and exceptions. Validate those inputs before sending anything to a model. Keep the transport layer separate from the logic so you can exercise failure cases without paying for model calls.
A small validation boundary
def read_question(payload):
if not isinstance(payload, dict):
raise ValueError("Expected an object")
question = payload.get("question")
if not isinstance(question, str) or not question.strip():
raise ValueError("A question is required")
if len(question) > 2000:
raise ValueError("Question is too long")
return question.strip()
assert read_question({"question": " What is retrieval? "}) == "What is retrieval?"
The length limit is a product choice, not a token limit. Token counts depend on the model. Configuration and credentials belong on the server, outside source control and client applications.
Handle the request lifecycle
Give outbound requests a deadline. Distinguish invalid input from unavailable infrastructure. Retry transient failures only within a bounded time budget; authentication errors need a configuration fix. Avoid blindly retrying requests that create side effects.
Exercise
Write cases for a valid question, missing field, whitespace, a number, and an oversized string. Replace the outbound client with a fake that returns success or raises a timeout.
Check: invalid input never reaches the client; a timeout returns a controlled error; logs contain a request ID rather than credentials. Explain how you would cancel the outbound work when the caller disconnects.