Self-Evolving Embodied Agents Group

About 关于我们

MMLab@SIGS

We are a dynamic research collective at the forefront of artificial intelligence, dedicated to solving the fundamental challenges of embodied intelligence. Our vision is to create self-evolving agents capable of perceiving, reasoning, and acting in complex, real-world environments—agents that continuously learn and adapt to seamlessly bridge the digital and physical worlds.

Our mission is to establish a complete pipeline for these agents, forming a closed loop of efficient data preparation, multimodal perception, spatiotemporal decision-making, and continuous learning. By integrating cutting-edge research in 3D vision, world models, and large language models, we are building the foundation for the next generation of intelligent systems.

We are always looking for motivated PhD students, postdocs, and research assistants who share our vision. Check out the Join Us section and follow us on Wechat. 我们是一个走在人工智能前沿的充满活力的研究集体，致力于解决具身智能的根本挑战。我们的愿景是创造能够在复杂的现实世界环境中感知、推理和行动的自我进化智能体——能够持续学习和适应，无缝连接数字与物理世界的智能体。

我们的使命是为这些智能体建立一个完整的创建流程，形成高效数据制备、多模态感知、时空决策和持续学习的闭环。通过整合3D视觉、世界模型和大型语言模型领域的最前沿研究，我们正在为下一代智能系统奠定基础。

我们随时欢迎有共同愿景的博士生、博士后和研究助理加入我们。请查看我们的加入我们栏目并关注我们的微信公众号。

Research Directions研究方向

Efficient Embodied Data Preparation 高效具身数据制备

We develop innovative methods for efficient collection, annotation, and preprocessing of embodied AI data. We focus on creating high-quality datasets that enable robust robot learning in real-world environments. 我们开发创新的方法以前沿、高效的方式收集、标注和处理具身AI数据。我们专注于创建高质量数据集，以在现实环境中实现鲁棒的机器人学习。

Automated Pipelines Synthetic-to-Real Active Learning

Multimodal Perception & World Models 多模态感知与世界模型

We build comprehensive world models through multimodal sensory fusion and understanding. Our research enables agents to perceive and reason about complex 3D environments using multiple modalities. 我们通过多模态感知融合建立全面的世界模型。我们的研究使智能体能够使用多模态感知和推理复杂的3D环境。

3D Vision Sensor Fusion World Modeling

Spatiotemporal Decision Making 时空决策与机器人策略学习

We train robot policies that can make intelligent decisions in complex spatiotemporal environments. Our work bridges the gap between simulation and real-world deployment through robust policy learning. 我们致力于训练能够在复杂时空环境中做出智能决策的机器人策略模型。我们的研究通过鲁棒策略学习弥合了仿真与现实部署的差距。

Imitation Learning Reinforcement Learning Sim-to-Real

Agent Continuous Learning 智能体持续学习

We enable agents to continuously learn and adapt to new environments and tasks without catastrophic forgetting. Our research addresses the fundamental challenges of lifelong learning in embodied AI. 我们让智能体能够不断学习并适应新环境和新任务。我们的研究应对了具身AI的持续学习的基础挑战，从而避免灾难性遗忘。

Test-time Adaptation Continual Learning LLM Adaptation

Selected Publications代表性成果

EIHMR: Collaborative Human-Camera Estimation for Global Human Mesh Recovery

NeurIPS 2026 (CCF-A)

[NeurIPS] [OpenReview]

Human Mesh Recovery Camera Motion Estimation 3D Reconstruction

EIHMR 面向移动相机单目视频中的全局三维人体运动恢复，协同估计人体与相机运动。在固定关键帧视角下，通过深度与接触约束细化人体轨迹，再将人体重渲染为静态网格，为运动感知 SLAM 提供几何匹配线索；在 EMDB 与 3DPW 上评估人体姿态，并在 EMDB2 上评估相机及全局人体轨迹。

Treat Domain-Specific Languages as Design Variables in LLM Agents

NeurIPS 2026 · Position Paper Track (CCF-A)

[NeurIPS] [OpenReview]

LLM Agents Domain-Specific Languages Executable Interfaces

本文主张将领域专用语言及其执行验证框架视为 LLM 智能体的设计变量，在语义推理与外部规则系统之间建立可检查的接口。通过高层抽象与显式约束缩短意图到动作的路径、提前排除无效动作，并从生命周期、应用领域及编码、SQL、图表生成案例讨论接口设计的影响。

What Drives Test-Time Adaptation for CLIP? A Controlled Empirical Study from an Update Perspective

NeurIPS 2026 · Evaluations and Datasets Track (CCF-A)

[arXiv] [Code]

Test-Time Adaptation Vision-Language Models Distribution Shift Benchmark

本文构建 TTABC，统一评测 20 余种 CLIP 测试时自适应方法，按参数、状态和推理更新分为三类。受控实验表明，适应收益主要来自测试时证据与可靠代理，重度优化未必必要；不同分布偏移下没有普遍最优的适应范式。

RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Robotics

ECCV 2026

[Project] [arXiv] [Code]

RoboStream 提出面向机器人操作的时空推理与记忆增强视觉语言模型框架，引入时空融合 Token 实现持久目标定位，构建因果时空图追踪状态转移，无需训练即可在 RLBench 长时域任务上达到 90.5% 成功率，在真实世界方块操作任务中达到 44.4% 成功率。

Neural Collapse in Test-Time Adaptation

CVPR 2026 (CCF-A)

[Paper] [arXiv]

研究测试时自适应（TTA）中的神经坍塌现象，分析模型在分布变化环境下特征表示与分类边界的变化规律。通过理论与实验分析，揭示神经坍塌对模型稳定性和泛化能力的影响，并探讨其在测试阶段持续适应过程中的作用机制，为提升模型在动态环境中的鲁棒性提供参考。

IGen: Scalable Data Generation for Robot Learning from Open-World Images

CVPR 2026 (CCF-A)

[Project] [arXiv] [Code]

本文提出 IGen，一种面向机器人学习的可扩展数据生成框架，可从开放世界图像中自动生成逼真的视觉观测与可执行动作，从而在无需人工遥操作数据的情况下训练有效的操作策略。

MoETTA: Test-Time Adaptation Under Mixed Distribution Shifts with MoE-LayerNorm

AAAI 2026 (CCF-A)

[arXiv] [Code]

针对真实场景中复杂的混合分布偏移，提出 MoETTA 框架，引入专家混合(MoE)架构为不同域偏移提供结构解耦的多专家系统。在全新构建的 Potpourri 基准上全面超越现有基线。

SizeGS: Size-aware Compression of 3D Gaussian Splatting via Mixed Integer Programming

ACM MM 2025 (CCF-A) Best Paper Candidate最佳论文候选

[Project] [arXiv] [Code]

基于混合整数规划的尺寸感知压缩框架，旨在通过快速搜索超参数将 3DGS 压缩至预定大小。能在一分钟内搜索到满足尺寸约束的最佳参数，实现 SOTA 级别的离线压缩性能。

Team团队

Principal Investigator指导老师

Ph.D. Students博士研究生

Master Students硕士研究生

Join Us加入我们

We are always looking for passionate and talented students to join our team. If you are interested in shaping the future of embodied AI, we encourage you to apply!

我们一直在寻找有热情和才华的学生加入我们的团队。如果您对塑造具身人工智能的未来感兴趣，我们鼓励您申请。

Self-Evolving Embodied Agents Group 自我进化具身智能体小组

MMLab@SIGS

Research Directions研究方向

Efficient Embodied Data Preparation 高效具身数据制备

Multimodal Perception & World Models 多模态感知与世界模型

Spatiotemporal Decision Making 时空决策与机器人策略学习

Agent Continuous Learning 智能体持续学习

Selected Publications代表性成果

Team团队

Principal Investigator指导老师

Zhi Wang (王智)

Jingyan Jiang (姜婧妍)

Ph.D. Students博士研究生

Master Students硕士研究生

Join Us加入我们

🎓 Undergraduates 🎓 本科生

🔬 Masters & PhDs 🔬 硕博研究生

🌍 Interns & Visiting 🌍 实习 / 访问学者

🤝 Corporate Collaboration 🤝 企业合作