Lune

CVPR2024Top-tier venue

Model Adaptation for Time Constrained Embodied Control

Jaehyun Song, Minjong Yoo, Honguk Woo

Abstract

When adopting a deep learning model for embodied agents, it is required that the model structure be optimized for specific tasks and operational conditions. Such optimization can be static such as model compression or dynamic such as adaptive inference. Yet, these techniques have not been fully investigated for embodied control systems subject to time constraints, which necessitate sequential decision-making for multiple tasks, each with distinct inference latency limitations. In this paper, we present MoDeC, a time constraint-aware embodied control framework using the modular model adaptation. We formulate model adaptation to varying operational conditions on resource and time restrictions as dynamic routing on a modular network, incorporating these conditions as part of multi-task objectives. Our evaluation across several vision-based embodied environments demonstrates the robustness of MoDeC, showing that it outperforms other model adaptation methods in both performance and adherence to time constraints in robotic manipulation and autonomous driving applications.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on12

  1. Object Goal Navigation using Goal-Oriented Semantic ExplorationDevendra Singh Chaplot, Dhiraj Gandhi, Abhinav Gupta, Ruslan SalakhutdinovNeurIPS 2020 · 857 citations
  2. Multi-Task Reinforcement Learning with Soft ModularizationRuihan Yang, Huazhe Xu, Yi Wu, Xiaolong WangNeurIPS 2020 · 247 citations
  3. Less is More: Task-aware Layer-wise Distillation for Language Model CompressionChen Liang, Simiao Zuo, Qingru Zhang, Pengcheng He et al.ICML 2023 · 119 citations
  4. Vision-Language Navigation with Random Environmental MixupChong Liu, Fengda Zhu, Xiaojun Chang, Xiaodan Liang et al.ICCV 2021 · 113 citations
  5. SplitNet: Sim2Sim and Task2Task Transfer for Embodied Visual NavigationDaniel Gordon, Abhishek Kadian, Devi Parikh, Judy Hoffman et al.ICCV 2019 · 80 citations

Related papers

  1. NeSyPr: Neurosymbolic Proceduralization For Efficient Embodied ReasoningWonje Choi, Jooyoung Kim, Honguk WooNeurIPS 2025 · 4 citations
  2. Towards Efficient Embodied Reasoning: Mixture-of-Depth Compute Allocation for Vision-Language-Action ModelWeiying Xie, Qingchen Zeng, Zihan Meng, Jiayun Tian et al.KDD 2026
  3. Fast-ThinkAct: Efficient Vision-Language-Action Reasoning via Verbalizable Latent PlanningChi-Pin Huang, Yunze Man, Zhiding Yu, Min-Hung Chen et al.CVPR 2026 · 24 citations
  4. ModularAgent: A Task-Aware Modular Framework for Joint Optimization of Multimodal Large Language Models and World ModelsYu-Wei Zhan, Xin Wang, Pengzhe Mao, Tongtong Feng et al.CVPR 2026
  5. Smartadapt: Multi-branch Object Detection Framework for Videos on MobilesRan Xu, Fangzhou Mu, Jayoung Lee, Preeti Mukherjee et al.CVPR 2022 · 14 citations
Dusk over the sea between two cliffs drawn in fine vertical lines