Assignment
Create a local collection containing two contradictory policy versions, a copied summary, and an unrelated but keyword-similar document.
Build sequence
- Represent a question as two or three claims needing support.
- Record evidence per claim and preserve source/version identifiers.
- Allow at most three search refinements with a reason for each.
- Produce a final answer with supported claims and unresolved gaps separated.
Acceptance checks
- Copied summaries are not counted as independent confirmation.
- Every citation supports the associated claim.
- The workflow stops within its budget even when a contradiction remains unresolved.
Evidence to keep
Submit the working artifact or decision document described above, the inputs used, and the observed results for every acceptance check. Include one failed attempt and the change you made after investigating it. Label fixture-based outcomes separately from live-system measurements. These artifacts become part of your final portfolio review.
Guided workshop: stop research for an explicit reason
A research workflow should track claims, evidence, unresolved contradictions, and remaining search budget. Five sources repeating one press release are not five independent confirmations. Preserve the origin of a claim rather than counting URLs alone.
Use this synthetic ledger:
| Claim | Source | Origin | Supports? |
|---|---|---|---|
| Feature supports export | Product guide v2 | Product documentation | Yes |
| Feature supports export | Blog summary | Product documentation | Yes |
| Export includes attachments | Product guide v2 | Product documentation | No statement |
| Export includes attachments | Release note v1 | Release notes | No |
Worked answer: the export claim has one direct source plus a derivative summary. Attachment export is unresolved or unsupported under the supplied evidence; it must not be inferred from the general export claim. Version and publication context matter when sources disagree.
Build steps: give each claim an ID, record source URL or fixture ID, observed date, document version, relevant excerpt, and relation (supports, contradicts, irrelevant). After every search, update the ledger and the budget. Stop when the required claims meet your stated evidence criterion, the budget expires, or an unresolved contradiction needs human review.
Exercise: implement a maximum of three queries and at most two retrieved passages per query. Use a fake search service so you can return duplicates, contradictions, empty results, and a timeout. A timeout still consumes an attempt under this lab's policy. Return a final result with supported claims and explicit unresolved items.
Solution policy: a new query is useful when it targets a specific evidence gap. Rephrasing an already-supported claim repeatedly wastes budget. Deduplicate by source origin where possible, but retain contradictory documents rather than collapsing them merely because they share a domain.
Assessment: compare a fixed single-search baseline with the adaptive workflow on six questions. Report evidence coverage, incorrect claims, calls, and latency. A longer report with more citations is not necessarily better. The final narrative must link claims to the source passages actually retrieved and distinguish inference from direct support.