Build an HTTP API that answers questions from a small collection of public documents. A web or Android client is optional. Use documents you can redistribute and keep model credentials on the server.
Scope and architecture
Start with 10–20 documents about one product. Store a stable document ID, source URL, and text for each document. Split text into passages, retrieve relevant passages for a question, and ask the model to answer using only that evidence. Return an answer, citations, and an explicit unsupported-answer state.
Request: {"question":"How do I reset my workspace?"}. Response: {"answer":"…","sources":[{"id":"guide-3","url":"…"}],"supported":true}. Validate this structure before returning it. A schema-valid response can still be factually wrong; evaluate the answer separately.
Build sequence
- Write a basic retrieval baseline and inspect the top passages for five questions.
- Add answer generation and preserve source identifiers end to end. Never accept invented citation IDs.
- Create 30 evaluation questions: 15 supported, 10 unsupported, and 5 ambiguous. Record expected evidence before tuning prompts.
- Add authentication, request limits, timeouts, and safe error responses. Avoid logging private question text by default.
- Deploy an endpoint and record model/version, retrieval settings, latency, and approximate usage cost.
Acceptance checks
- Every returned source exists in the supplied evidence and actually supports its associated claim.
- Unsupported questions trigger an explicit fallback rather than a fabricated answer.
- Empty input, a model timeout, and an unavailable retrieval service return controlled responses.
- Retrieved text containing instructions to ignore the user cannot change the application’s permissions or reveal credentials.
- Report supported-answer accuracy and unsupported-question handling separately. Choose a target before the final run, record all failures, and explain tradeoffs against the retrieval-only baseline.
Deliverables
Submit a repository with setup instructions, document provenance, a sample request, an evaluation table, and a short deployment runbook. The runbook should describe configuration, secret management, troubleshooting, and how to disable the endpoint. Explain one change that improved quality and one that increased cost without enough benefit.
This is an independent project specification, not a supplied implementation or a production certification.