Lune

CVPR2024Top-tier venue

On the Robustness of Language Guidance for Low-Level Vision Tasks: Findings from Depth Estimation

Agneet Chatterjee, Tejas Gokhale, Chitta Baral, Yezhou Yang

Abstract

Recent advances in monocular depth estimation have been made by incorporating natural language as additional guidance. Although yielding impressive results, the impact of the language prior, particularly in terms of generalization and robustness, remains unexplored. In this paper, we address this gap by quantifying the impact of this prior and introduce methods to benchmark its effectiveness across various settings. We generate “low-level” sentences that convey object-centric, three-dimensional spatial relationships, incorporate them as additional language priors and evaluate their downstream impact on depth estimation. Our key finding is that current language-guided depth estimators perform optimally only with scene-level descriptions and counter-intuitively fare worse with low level descriptions. Despite leveraging additional data, these methods are not robust to directed adversarial attacks and decline in performance with an increase in distribution shift. Finally, to provide a foundation for future research, we identify points of failures and offer insights to better understand these shortcomings. With an increasing number of methods using language for depth estimation, our findings highlight the opportunities and pitfalls that require careful consideration for effective deployment in real-world settings.11Code/Data: https://github.com/agneet42/lang_depth

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Scalable Autoregressive Monocular Depth EstimationJinhong Wang, Jian Liu, Dongqi Tang, Weiqiang Wang et al.CVPR 2025
  2. Latent Radiance Fields with 3D-aware 2D RepresentationsChaoyi Zhou, Xi Liu, Feng Luo, Siyu HuangICLR 2025

Related papers

  1. WorDepth: Variational Language Prior for Monocular Depth EstimationZiyao Zeng, Daniel Wang, Fengyu Yang, Hyoungseob Park et al.CVPR 2024 · 20 citations
  2. Vision-Language Embodiment for Monocular Depth EstimationJinchang Zhang, Guoyu LuCVPR 2025
  3. Exploiting Multi-Object Relationships for Detecting Adversarial Attacks in Complex ScenesMingjun Yin, Shasha Li, Zikui Cai, Chengyu Song et al.ICCV 2021 · 25 citations
  4. Evaluating Robustness of Monocular Depth Estimation with Procedural Scene PerturbationsJack Nugent, Siyang Wu, Zeyu Ma, Beining Han et al.NeurIPS 2025 · 6 citations
  5. RoSAMDepth: Robust Self-supervised Depth Estimation Leveraging Segment Anything ModelXuanang Gao, Zhiwei Ning, Gengming Zhang, Jiaxi Cao et al.CVPR 2026
Dusk over the sea between two cliffs drawn in fine vertical lines