Learn the concept
Strings are immutable sequences of Unicode code points. Indexing selects a position and slicing selects a range; the end of a slice is excluded. Methods such as strip and lower return new strings rather than modifying the original. Use f-strings to format values clearly.
A text character visible to a user can contain multiple code points. The length of a Python string is therefore not necessarily the number of visual characters, bytes, or model tokens. Encoding converts text to bytes for storage or transport; decoding reverses that conversion using a specified encoding.
Normalize only when the domain permits it. Lowercasing a case-insensitive search field is different from changing an identifier or password. Preserve original text when you need to show evidence or reproduce an input.
Run and inspect
raw = " Ticket A-17 "
clean = raw.strip()
assert clean == "Ticket A-17"
assert raw.startswith(" ")
assert clean[:6] == "Ticket"
print(f"Loaded: {clean}")
assert "café".encode("utf-8").decode("utf-8") == "café"
Your exercise
Normalize a search query while preserving its original value. Include accented text, whitespace, and an empty string. Print the character and UTF-8 byte counts.
Check your understanding
Your normalization does not overwrite the original, and you do not use byte or character counts as model token counts.