[{"data":1,"prerenderedAt":54},["Reactive",2],{"news-story-news-1792384474854-20833":3},{"id":4,"title_zh":5,"title_en":6,"summary_zh":7,"summary_en":8,"takeaways_zh":9,"takeaways_en":14,"content_zh":19,"content_en":20,"external_links":21,"source":31,"category":38,"impact_score":39,"views_count":40,"likes_count":41,"related_entities":42,"tags":46,"image_url":53},"news-1792384474854-20833","港大与清华揭示视频世界模型记忆假象 MIST：前序动作执行失败占比超 70%，误判为记忆遗忘","HKU & Tsinghua Unveil MIST for Video World Models: Over 70% of Supposed Memory Failures Stem from Execution Errors","现有视频世界模型评测通常将场景记忆与动作执行揉合为单一最终得分，导致模型因未能生成动作而产生的状态缺失被错误归咎于记忆遗忘。香港大学与清华大学团队发表 MIST（arXiv:2610.23542，HF 12 赞，开源代码与主页上线）。该基准首次将“前置动作是否执行”与“后续场景是否记忆”彻底解耦，实测表明在主流视频世界模型中，高达 70% 的表面记忆失败本质上是动作执行失败；引入 M3 执行感知提示与多阶段分离评测后，世界模型真实长程记忆准确率提升 11.8 至 29.6 个百分点。","Current memory benchmarks for video world models conflate whether an action was carried out with whether the model remembered its outcome, incorrectly attributing action execution failures to cognitive memory loss. Researchers from HKU and Tsinghua introduced MIST (arXiv:2610.23542, 12 upvotes on Hugging Face). By decoupling physical execution from temporal memory, MIST reveals that up to 70% of apparent memory failures in leading models are actually execution failures. Correcting for execution raises true scene memory accuracy by +11.8 to +29.6 percentage points.",[10,11,12,13],"港大多媒体实验室与清华大学联合发布 MIST（arXiv:2610.23542），获 Hugging Face 热门论文 12 赞","首次揭露现有视频世界模型评测中超过 70% 的“记忆遗忘失败”本质上是中间帧未能成功执行动作，而非真实记忆丢失","构建动作执行与环境记忆解耦的三阶段因果验证基准，为世界模型诊断出根本性的故障归因","在 WRBench 与 MBench 基准上，修正执行误差后世界模型的真实长程场景记忆得分显著提升 11.8 至 29.6 个百分点",[15,16,17,18],"HKU-MMLab and Tsinghua introduced MIST (arXiv:2610.23542, 12 upvotes on Hugging Face Papers)","Reveals that over 70% of apparent memory failures in video world models are actually action execution failures in intermediate frames","Proposes a decoupled three-stage causal pipeline separating scene establishment, action execution, and temporal memory retention","Improves true long-horizon memory consistency scores by +11.8 to +29.6 points across standard WRBench and MBench suites","### 核心背景与行业痛点\n在具身智能、物理仿真与自动驾驶中，视频生成模型常被期望充当物理“世界模型（World Models）”，需要在长序列视频渲染中准确记住此前发生过的环境改变（例如被移动的茶杯、被打开的柜门或被撞击的障碍物）。然而，现有的权威世界模型记忆评测（如 WRBench、MBench）通常只检测视频最后一帧的目标物体状态是否存在。当模型得分很低时，研究人员普遍得出“视频世界模型缺乏长程物理记忆能力”的结论。香港大学多媒体实验室（HKU-MMLab）与清华大学团队在 **`arXiv:2610.23542`** 中提出严正质疑：**如果模型在中间帧压根就没有执行用户指令中的动作（Execution Failure），最后一帧自然不可能出现预期的物理状态，这根本不是记忆失败，而是执行失败！**\n\n### 架构亮点与底层机制\n1. **动作执行与物理记忆解耦评测框架（MIST）**：\n   - `MIST` 将评测划分为严格的三阶段因果验证链条：① **Scene Establishment**：验证初始场景与物体是否成功锚定；② **Action Execution Verification**：利用多模态大模型判定中间帧是否真实执行了指令中的物理动作转换；③ **Memory Retention Diagnosis**：仅在动作确实执行成功的样本子集上，评测模型在后续多秒视频漫游中是否保持了状态一致性；\n2. **执行感知提示与 M3 记忆探针管线**：\n   - 提出了 M3（Memory-augmented Multi-stage Modeling）干预范式，通过显式强化动作过渡区间的动态隐表征，有效将动作执行成功率与长程记忆维持率双向解耦，为世界模型精准定位归因故障根源。\n\n### 权威 Benchmark 与实测跑分对比\n在包含数百组长程视频生成的严苛实验中（Table 17, 18, 19）：\n- **故障根本原因归因拆解**：\n  - 实测数据揭示：在主流视频生成基线模型（如 Lingbot、H3 等）在评测套件中未通过的所有“记忆失败”案例中，**高达 70% 以上的样本在中间帧完全没有执行目标物理动作**；\n- **解耦后的真实记忆增益（Table 19）**：\n  - 在 `WRBench/40` 官方评测套件中，修正执行干扰并引入 M3 架构后，官方 $T$ 得分从 0.452 提升至 **0.570（+0.118）**，视觉保真度从 0.648 提升至 **0.755（+0.107）**；\n  - 在显式揭示（Reveal）指标上，得分从 0.372 暴涨至 **0.668（相对提升 +0.296 / +79.6%）**；\n  - 在 `MBench-T/30` 套件上，官方 $U$ 得分由 0.292 提升至 **0.446（+0.154）**。\n\n### 开发者实战落地与开箱指南\n- **开源项目与代码**：官方已开放项目主页（`https://hku-mmlab.github.io/MIST/`）与 GitHub 仓库（`https://github.com/HKU-MMLab/MIST`）。\n- **世界模型研发建议**：构建自动驾驶或具身仿真视频世界模型的团队，在优化模型表现时，切勿盲目堆砌注意力层去提升所谓的“长程记忆力”；应当首先利用 MIST 评估流检查模型对物理动作指令的精确执行（Prompt-to-Action Alignment），只有动作被正确生成，后续的长程环境持久性才有讨论基础。","### 核心背景与行业痛点\nIn embodied AI and spatial simulation, video generation models are increasingly deployed as predictive world models that must maintain temporal consistency over physical transformations (e.g., an object being moved or a drawer opened). However, existing video memory benchmarks collapse sequence evaluation into a single terminal frame score. When a model scores poorly, researchers assume the model suffered from catastrophic forgetting. Researchers from HKU-MMLab and Tsinghua University published **`arXiv:2610.23542`**, revealing a profound methodological flaw: **if the model failed to execute the action in the intermediate frames to begin with, the resulting state change never happened—conflating execution failure with memory loss.**\n\n### 架构亮点与底层机制\n1. **Decoupled Execution-Memory Evaluation (MIST)**:\n   - `MIST` introduces a three-stage causal evaluation pipeline: ① Scene establishment verification; ② Intermediate action execution gating using multimodal verifiers; ③ Memory retention testing strictly conditioned on confirmed action execution;\n2. **M3 Memory Intervention Pipeline**:\n   - Introduces explicit transition-state conditioning to boost execution fidelity and isolate pure temporal persistence from dynamic failure modes.\n\n### 权威 Benchmark 与实测跑分对比\nAcross comprehensive diagnostic suites (Tables 17–19):\n- **Failure Root-Cause Breakdown**:\n  - Up to **70% of apparent 'memory failures'** across open and closed video generation models are actually failures of intermediate action execution rather than cognitive forgetting;\n- **True Memory Scores After Correction (Table 19)**:\n  - On the `WRBench/40` official suite, decoupling execution via M3 boosts the official $T$ metric from 0.452 to **0.570 (+0.118)** and visual fidelity from 0.648 to **0.755 (+0.107)**;\n  - The internal reveal metric surges from 0.372 to **0.668 (+0.296, a +79.6% relative gain)**;\n  - On `MBench-T/30`, the official $U$ metric rises from 0.292 to **0.446 (+0.154)**.\n\n### 开发者实战落地与开箱指南\n- **Open-Source Suite**: Project page (`https://hku-mmlab.github.io/MIST/`) and repository (`https://github.com/HKU-MMLab/MIST`) are live.\n- **Engineering Takeaways**: Teams optimizing video world models for robotics should not default to expanding memory context windows; fixing first-stage prompt execution accuracy resolves the majority of perceived environmental drift.",[22,25,28],{"title":23,"url":23,"type":24},"https://hku-mmlab.github.io/MIST/","other",{"title":26,"url":26,"type":27},"https://github.com/HKU-MMLab/MIST","github",{"title":29,"url":29,"type":30},"https://arxiv.org/abs/2610.23542","arxiv",{"platform":32,"author_name":33,"author_handle":34,"author_avatar":35,"tweet_url":36,"published_at":37},"twitter","HKU & Tsinghua University","@huggingface","https://unavatar.io/x/huggingface","","2026-10-19T04:34:34.854Z","Research",93,45,0,{"models":43,"agents":44,"vendors":45},[],[],[],[47,48,49,50,51,52],"MIST","Video World Models","Tsinghua","HKU","Benchmark","Robotics","https://arxiv.org/html/2610.23542v1/overview.png",1792901918972]