What you will learn
Vectors, Probability, Train/validation/test, Precision and recall, Data leakage.
Engineering the capability
A baseline answers whether a complex system earns its cost. For ticket routing, begin with a majority-class classifier or keyword rules, then compare a learned model. Accuracy alone can hide poor behavior when almost every ticket belongs to one category. Precision asks how many predicted positives were correct; recall asks how many real positives were found.
Embeddings represent inputs as vectors whose geometry supports comparisons. Cosine similarity compares direction, not truth or permission. A near neighbor can be wrong for the task even if the embedding captures related language.
Split data before fitting learned preprocessing or tuning a model. Put related records into the same split: two near-duplicate tickets in training and test can exaggerate quality. Use a time-based split when the deployment will predict future cases from past data. Reserve a final test set for a decision after development.
Compute classification metrics
def precision_recall(tp, fp, fn):
if min(tp, fp, fn) < 0:
raise ValueError("Counts must be nonnegative")
precision = tp / (tp + fp) if tp + fp else 0.0
recall = tp / (tp + fn) if tp + fn else 0.0
return precision, recall
assert precision_recall(12, 3, 8) == (0.8, 0.6)
assert precision_recall(0, 0, 5) == (0.0, 0.0)
This uses zero when a denominator is zero; document that convention in your report. It computes one class at a time and does not substitute for a full confusion matrix.
Worked case
Of 20 truly urgent tickets, a classifier flags 12. It also flags 3 non-urgent tickets. Precision is 12/15 = 0.80; recall is 12/20 = 0.60. Improving precision by flagging fewer tickets may miss more urgent work. Choose the tradeoff with the team that bears the cost of each error.
Put it into practice
Continue with the next lab: audit a ticket-routing baseline. Build the artifact, record the failure cases, and explain the tradeoff before moving on.