World models
TouchéWorld: reaching into an AI world model with your hands
A new open research model adds live hand tracking to a camera-controlled video world model, so you can walk up to a tree and touch it while the scene keeps generating. Adapter weights and code are out; the full distilled model is not.
Most interactive world models let you move the camera and nothing else. TouchéWorld, a tech report posted to arXiv on 4 November 2026 by researchers at Université Sorbonne Paris Nord, Université Paris Cité, East China Normal University and Dalian Jiaotong University, adds a second control channel: your hand.
The idea is easy to picture. You start from one image and a scene description. You steer through the generated scene with camera controls, walk up to an osmanthus tree, reach into its branches and watch the foliage move, then pull back and carry on. The model keeps generating as you do it.
How it works
TouchéWorld is built on the LTX-2.5 image-to-video backbone from Lightricks, with the SolarWM camera adapter that handles movement through the scene. The authors freeze that base and train a separate hand LoRA on top.
Hand input arrives in two forms: rendered skeleton videos, which tell the model where the hand is in the frame, and numerical features (wrist translation, joint geometry and a validity mask), which carry the motion over time.
The training data is the hard part. The team built joint camera-and-hand clips from ordinary egocentric videos by estimating missing camera and hand annotations, rejecting clips where the estimates disagreed, and aligning everything in time. They plan to release that dataset.
For live use, they adapt the model to generate causally, in chunks. Each new chunk sees the controls you have given so far and the chunks before it, never the future. A few-step distillation then speeds this up; the paper reports a four-step model.
What was released
- Paper: arXiv 2611.06061, CC BY 4.0.
- Weights: adapters only, Teacher-27 and Causal-27, on Hugging Face. You need the official LTX-2.5 base model and the SolarWM camera adapter as well. The licence is the LTX-2.x community licence.
- Distilled (DMD) weights: listed as “coming soon”.
- Code: training and inference source on GitHub, committed on 3 November.
- TouchéBench: a new benchmark covering hand-motion following, camera-hand coordination, online interaction and object response, using 200 held-out HOI4D clips plus purpose-recorded scenarios.
Limits
This is a research release. The paper compares the model against two other hand-conditioned video models (JCV and Hand2World) and against SolarWM, and reports stronger joint camera-and-hand control for its own model, but with trade-offs: hand following and camera–hand coordination scores drop after causal adaptation and few-step distillation. The few-step distilled model improves PSNR and SSIM over the multi-step causal one, with mixed results on other quality measures.
The training footage is first-person video, so expect the model to behave best in that kind of view. We have not seen a published frame rate or latency figure, so “live” here means chunk-by-chunk generation, not a proven real-time rate.
Our take
Hands are a more natural input than a keyboard for anything that feels like touching a world: opening, pushing, picking up. If a world model is going to be a game, it needs to respond to what you do, and to remember it afterwards. TouchéWorld tests the first half of that. The second half, keeping the consequences of a touch, is the memory problem we cover in memory in AI games. For how these systems generate frames in the first place, start with how AI world models work.