What Training Data Does
The model knows exactly what was in its training corpus — and nothing else. Coverage depth determines knowledge depth. The data mix encodes bias. The cutoff date creates an invisible wall. And as AI-generated text fills the internet, a slow degradation problem called model collapse looms.
A stale answer
It sounds sure. But a newer tournament may have been played since, and it can’t tell.
A model only knows the text it was trained on.
A frozen snapshot
Training text is collected up to a date, then frozen. Nothing after it exists to the model.
And it has no clock. Unless you tell it today’s date, it doesn’t know time has passed.
Read it or not
Could a model have learned this from its training text?
Tap a box for each one.
How much it read
It also matters how much it read about a topic.
The mix leans
Whatever the text over-represents, the model does too.
Most of it is in English, from a handful of countries. Ask about “a typical wedding” and answers lean that way.
Copies of copies
More and more web text is now written by AI. That text feeds the next models.
Train on enough of it and rare ideas get rarer each round. Researchers call this model collapse. It’s still an open problem.
Shaped by its reading
A model knows what its data covered, as well as it was covered, up to its cutoff.
The text it learned from is its training data. The date that text stops is its knowledge cutoff.