Learn the concept
AI projects often use table and array libraries. Learn the underlying operations first: select columns, filter rows, group records, aggregate values, and join datasets. Pandas and NumPy provide optimized tools for these tasks, but their convenience does not remove the need to understand shapes and missing values.
An array has a shape and a data type. Elementwise multiplication differs from a dot product. Broadcasting can align compatible shapes but can also produce a result with an unintended shape. Inspect dimensions before interpreting a numerical result.
A table transformation should define what happens to missing values and duplicate keys. Converting all missing entries to zero changes meaning when missing means unknown. The standard-library example below makes the operation explicit; later projects can implement the same contract with a numerical library in their own environment.
Run and inspect
rows = [{"team": "a", "minutes": 2}, {"team": "a", "minutes": 4}, {"team": "b", "minutes": None}]
groups = {}
for row in rows:
if row["minutes"] is not None:
groups.setdefault(row["team"], []).append(row["minutes"])
means = {key: sum(values) / len(values) for key, values in groups.items()}
assert means == {"a": 3.0}
Your exercise
Calculate per-team averages while retaining a separate missing-value count. Draw the shapes of two vectors and their dot-product result.
Check your understanding
Unknown measurements are not counted as zero, and you can explain the difference between a scalar, vector, and table.
Guided workshop: bound concurrent document work
Async programming overlaps waiting; it does not make CPU-heavy Python calculations automatically run in parallel. Bound the number of active operations so a large import does not overwhelm storage or a downstream API. Also bound queued work for genuinely large streams.
# academy-check
import asyncio
async def process_all(values):
limit = asyncio.Semaphore(2)
active = 0
peak = 0
async def process(value):
nonlocal active, peak
async with limit:
active += 1
peak = max(peak, active)
try:
await asyncio.sleep(0)
return value * 2
finally:
active -= 1
results = await asyncio.gather(*(process(v) for v in values))
return results, peak
results, peak = asyncio.run(process_all([1, 2, 3, 4]))
assert results == [2, 4, 6, 8]
assert peak <= 2
gather returns results in input order even when tasks finish in another order. The semaphore bounds active work, but this example still creates a coroutine for every input. For a million records, use a bounded queue and a fixed worker pool to control waiting work too.
Exercise: add a deliberately failing item and decide whether one item should fail the batch or produce a per-item rejection. Preserve cancellations: a user cancelling the import is not an ordinary bad record. Use finally for cleanup so counters and resources stay correct on failure.
Worked data task: for rows with token counts [100, 200, 300], the arithmetic mean is 200. If one row represents ten requests and the others represent one each, the per-request mean needs weights. Tabular libraries align values by columns or labels; array libraries operate by dimensions. Check shapes, missing values, and units before computing a model feature.
Checkpoint: explain when you would use async I/O, a thread for a blocking I/O library, or a process for CPU-heavy work. Record throughput and peak concurrency separately. Increasing concurrency beyond a provider's capacity can increase timeouts and total cost rather than improve useful throughput.