Language modeling progress is largely evidenced by steadily rising scores
on benchmarks of increasing apparent difficulty. From this, the field infers
increasingly general capabilities, many of which presuppose robust world
modeling. Interpreting a score, however, requires understanding both the
task’s computational requirements and how the test-taker generalizes from
them. Unlike humans, who demonstrably generalize well, large language models
(LLMs) often do not: they instead learn heuristics fit to the minimal sufficient
computational requirements of a task, which may be far simpler than the
task appears to demand. This thesis extends this analysis to embodiment,
where multimodal LLMs (MLLMs) serve as the perception and reasoning
core of embodied agents, whose reliable deployment depends on benchmarked
capability including a robust world model of the agent’s environment.

We evaluate world model robustness by independently manipulating object
count, duplication, trajectory length, and viewpoint change in a controlled
synthetic benchmark of multi-frame egocentric trajectories, and find that
frontier MLLMs degrade sharply as difficulty increases, while humans remain
near ceiling. We then characterize embodied question answering (EQA) task
demands through three nested paradigms: selecting relevant observations, clus-
tering nearby observations into local spatial-semantic units, and propagating
semantic state across experience. Stratifying questions by the weakest sufficient
paradigm yields a difficulty progression. Prominent EQA benchmarks pre-
dominantly test only the weakest paradigm, single-frame selection, and so we
introduce Campus-Bench, three multi-hour, campus-scale episode histories with
questions stratified into selection and propagation regimes, directly comparing
model performance on the two over the same episodes. We additionally develop
a diagnostic method in which an MLLM incrementally constructs and traverses
a hierarchical spatial-semantic memory, executing propagation explicitly. A
frontier long-context MLLM performs strongly on selection but collapses on
propagation; the same model leveraging our method dramatically improves
it while remaining competitive on selection, suggesting models struggle to
maintain state internally.

From this, we argue that frontier MLLMs do not yet maintain the robust
world models their benchmark scores suggest, and that the field needs to make
progress in formalizing tasks’ computational requirements and evaluating along
their progressions of difficulty.