World models
L3G-World: a compact 3D memory for real-time AI worlds
KAIST researchers have published L3G-World, a world model that keeps a small 3D memory of the places it has generated, so a scene stays consistent when you walk back to it. Its distilled variant runs at about 9 frames per second on one H100.
A team at KAIST has released L3G-World, an interactive world model built around one idea: remember the world as a compact 3D scene, not as an ever-growing pile of past video frames. It is a research paper with a project page and a code repository, not a product.
What happened
On 29 October the group posted the paper (arXiv 2610.40614) and a project page. L3G-World starts from a single image and a camera path, then generates video of an explorable place in real time. With a few-step distilled version of the generator, the authors report 9.02 frames per second on one H100 GPU.
The problem it targets is consistency. If you walk away from a building and come back, many world models draw a different building. Fixing that needs memory, and memory is expensive.
How it works
The paper compares two existing approaches. Temporal-context models keep earlier frames or attention caches, so cost grows with every second generated. In the authors’ 400-frame test, one such model (WorldKV) ran out of memory, and another (Yume 1.5) used 66.7 GiB of GPU memory. Explicit 3D memories stay anchored to the scene, but most are either slow to build or store far more points than they need.
L3G-World uses a latent 3D Gaussian memory. Two modules take turns:
- L3G-Mem adds new 3D Gaussians only where the camera has seen something new, and nudges existing ones to fix errors, using the video chunk just generated.
- L3G-Gen, a video diffusion transformer built on NVIDIA’s Cosmos-Predict2.5, generates the next chunk from the memory.
Each Gaussian also stores a feature in the video model’s own latent space. The memory can therefore be rendered straight into that space, skipping a slow decode-and-re-encode step. The memory grows with how much of the world you have explored, not with how long you have been playing.
The numbers
All figures are the authors’ own, on the DL3DV and RealEstate10K datasets, with 481-frame trajectories that go out and retrace the same path back.
- Against NeoVerse, another Gaussian-based method, L3G-World stores 266 to 417 times fewer primitives (33,000 against 8.8 million on DL3DV) and uses 42 to 67 times less memory.
- Peak GPU memory in the 400-frame test is 7.4 GiB, against 11.7 to 66.7 GiB for the other methods that finished.
- On DL3DV it scores best on loop-closure PSNR and LPIPS, the measures of whether the return trip matches the outward one.
- That distilled variant, L3G-World Fast, trades some consistency for speed.
Limits
The memory is designed for static scenes. It does not model objects that move on their own, which rules out crowds, traffic and characters. The authors also note that the memory can still grow in very large worlds, and that errors made during long generation can carry into later updates. The model was trained on real-estate and scene-walkthrough footage, so it is a long way from a game with rules.
Our take
For anyone building interactive worlds, this is a useful split. A place can be remembered cheaply as geometry, while the things that change in it, such as people, objects and plot, need a different kind of memory. That second kind is what we describe in memory in AI games. The first kind is closer to the problem in how AI world models work: a model that forgets the room behind you cannot hold a story together. Expect the next steps to combine a stable static scene with a state layer for characters, which is where the design in world model vs game engine points.