Assignment
Create 60 synthetic tickets, including 12 urgent examples and 10 near-duplicate pairs. Keep a separate group identifier for related records.
Build sequence
- Write a keyword classifier and a majority-class baseline.
- Split by group and document why a naive random split leaks information.
- Compute the confusion matrix, precision, and recall without using the held-out cases to tune keywords.
- Inspect five mistakes and propose a new feature or data collection change.
Acceptance checks
- No group appears in both development and test data.
- Metric calculations match hand-counted true positives, false positives, and false negatives.
- The report states the sample size, baseline comparison, and cost of missed urgent tickets.
Evidence to keep
Submit the working artifact or decision document described above, the inputs used, and the observed results for every acceptance check. Include one failed attempt and the change you made after investigating it. Label fixture-based outcomes separately from live-system measurements. These artifacts become part of your final portfolio review.