UniHOI: Unified Human-Object Interaction Understanding via Unified Token Space
Panqi Yang, Haodong Jing, Nanning Zheng, Yongqiang Ma
Abstract
In the field of human-object interaction (HOI), detection and generation are two dual tasks that have traditionally been addressed separately, hindering the development of comprehensive interaction understanding. To address this, we propose UniHOI, which jointly models HOI detection and generation via a unified token space, thereby effectively promoting knowledge sharing and enhancing generalization. Specifically, we introduce a symmetric interaction-aware attention module and a unified semi-supervised learning paradigm, enabling effective bidirectional mapping between images and interaction semantics even under limited annotations. Extensive experiments demonstrate that UniHOI achieves state-of-the-art performance in both HOI detection and generation. Specifically, UniHOI improves accuracy by 4.9% on long-tailed HOI detection and boosts interaction metrics by 42.0% on open-vocabulary generation tasks.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 37ee8c73-6307-4b56-b296-10f1dfa36ca3Cited by top-tier papers7
- Leveraging Machine Unlearning for Cost-Efficient Preference AlignmentXiaoHua Feng, Yuyuan Li, HuWei Ji, Li Zhang et al.ICML 2026 · 4 citations
- MUSE: Resolving Manifold Misalignment in Visual Tokenization via Topological OrthogonalityPanqi Yang, Haodong Jing, Jiahao Chao, Tingyan Xiang et al.ICML 2026 · 3 citations
- SCoRe: Salience-Coverage Reduction for Vision Token Pruning in Vision-Language ModelsTong Xu, Hailong Shi, Xingyu GaoCVPR 2026
- LLM-as-Scheduler: Agentic Workflow Dynamic SchedulingDawei Xiang, Kexin Chu, Wenyan Xu, Wenhui Zhang et al.ACL 2026
- Test-Time Debiasing with Probabilistic Prompts via Wasserstein Distance in Vision-Language ModelsChengye Wang, Yuyuan Li, XiaoHua Feng, Xiaolin Zheng et al.ICML 2026
Builds on24
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Photorealistic Text-to-Image Diffusion Models with Deep Language UnderstandingChitwan Saharia, William Chan, Saurabh Saxena, Lala Li et al.NeurIPS 2022 · 8,965 citations
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 7,873 citations
- GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion ModelsAlexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam et al.ICML 2022 · 4,691 citations
- SDXL: Improving Latent Diffusion Models for High-Resolution Image SynthesisDustin Podell, Zion English, Kyle Lacey, Andreas Blattmann et al.ICLR 2024 · 4,569 citations
Related papers
- Improving Human-Object Interaction Detection via Phrase Learning and Label CompositionZhimin Li, Cheng Zou, Yu Zhao, Boxun Li et al.AAAI 2022 · 43 citations
- Bilateral Collaboration with Large Vision-Language Models for Open Vocabulary Human-Object Interaction DetectionYupeng Hu, Changxing Ding, Chang Sun, Shaoli Huang et al.ICCV 2025 · 1 citation
- Detecting Any Human-Object Interaction Relationship: Universal HOI Detector with Spatial Prompt Learning on Foundation ModelsYichao Cao, Qingfei Tang, Xiu Su, Song Chen et al.NeurIPS 2023 · 64 citations
- LINK: Learning Instance-level Knowledge from Vision-Language Models for Human-Object Interaction DetectionEastman Z. Y. Wu, Yali Li, Yuan Wang, Shengjin WangICLR 2026
- Learning Transferable Human-Object Interaction Detector with Natural Language SupervisionSuchen Wang, Yueqi Duan, Henghui Ding, Yap-Peng Tan et al.CVPR 2022 · 66 citations
