World model vs LLM: what each predicts, and why games need both

“World model” and “large language model” are often used as if one replaces the other. They do not. They are trained to predict different things, and that difference decides what each is good at when you build an interactive world.

This guide explains the world model vs LLM question in plain terms, with the research behind it, and ends with how the two fit together in a game.

What does an LLM predict?

A large language model is trained on text to predict what comes next. The model behind GPT-3, described in the paper Language Models are Few-Shot Learners, is an autoregressive language model with 175 billion parameters. It reads a sequence of tokens (word fragments) and produces the next one, then the next.

That simple task produces a lot. To continue a sentence well, the model has to pick up grammar, facts, tone, and the way characters and plots tend to behave. It can write dialogue, summarise a rulebook, invent a quest and play a convincing innkeeper.

What it does not do is hold a scene in a form it can check. The world it describes exists only as words. If you ask where the key is, the answer comes from the text so far, and nothing stops it contradicting an earlier line.