Humans: "The café is open." / "THE CAFE IS OPEN!!!" / "the café is open" — same meaning. Computer? Totally different strings.
What normalization does
Cleans text: lowercase everything, remove extra punctuation, standardize accents.
Before
THE CAFE IS OPEN!!!
After
the cafe is open
⚖ Tradeoff
| More cleaning | Less cleaning |
|---|---|
| Faster training | More nuance |
| Smaller models | Richer context |