Lune

ICLR2025Top-tier venue

Tracking objects that change in appearance with phase synchrony

Sabine Muzellec, Drew Linsley, Alekh Karkada Ashok, Ennio Mingolla, Girik Malik, Rufin VanRullen, Thomas Serre

Abstract

Objects we encounter often change appearance as we interact with them. Changes in illumination (shadows), object pose, or the movement of non-rigid objects can drastically alter available image features. How do biological visual systems track objects as they change? One plausible mechanism involves attentional mechanisms for reasoning about the locations of objects independently of their appearancesa capability that prominent neuroscience theories have associated with computing through neural synchrony. Here, we describe a novel deep learning circuit that can learn to precisely control attention to features separately from their location in the world through neural synchrony: the complex-valued recurrent neural network (CV-RNN). Next, we compare object tracking in humans, the CV-RNN, and other deep neural networks (DNNs), using FeatureTracker: a large-scale challenge that asks observers to track objects as their locations and appearances change in precisely controlled ways. While humans effortlessly solved FeatureTracker, state-of-the-art DNNs did not. In contrast, our CV-RNN behaved similarly to humans on the challenge, providing a computational proof-of-concept for the role of phase synchronization as a neural substrate for tracking appearance-morphing objects as they move about.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Emerging Properties in Self-Supervised Vision Transformers cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • Efficiently Modeling Long Sequences with Structured State Spaces cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • Multiscale Vision Transformers cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • Multiscale Vision Transformers cites Is Space-Time Attention All You Need for Video Understanding?
  • Is Space-Time Attention All You Need for Video Understanding? cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • OrthoRF: Exploring Orthogonality in Object-Centric Representations cites Emerging Properties in Self-Supervised Vision Transformers
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. OrthoRF: Exploring Orthogonality in Object-Centric RepresentationsDespoina Touska, Bastiaan Onne Fagginger Auer, Alexandru Onose, Tejaswi Kasarla et al.ICLR 2026

Builds on14

  1. An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
  2. Emerging Properties in Self-Supervised Vision TransformersMathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou et al.ICCV 2021 · 8,921 citations
  3. Efficiently Modeling Long Sequences with Structured State SpacesAlbert Gu, Karan Goel, Christopher RéICLR 2022 · 3,482 citations
  4. Is Space-Time Attention All You Need for Video Understanding?Gedas Bertasius, Heng Wang, Lorenzo TorresaniICML 2021 · 2,927 citations
  5. Multiscale Vision TransformersHaoqi Fan, Bo Xiong, Karttikeya Mangalam, Yanghao Li et al.ICCV 2021 · 1,611 citations

Related papers

  1. Tracking Without Re-recognition in Humans and MachinesDrew Linsley, Girik Malik, Junkyung Kim, Lakshmi Narasimhan Govindarajan et al.NeurIPS 2021 · 21 citations
  2. Recurrent Complex-Weighted Autoencoders for Unsupervised Object DiscoveryAnand Gopalakrishnan, Aleksandar Stanic, Jürgen Schmidhuber, Michael C. MozerNeurIPS 2024 · 10 citations
  3. STaR: Self-Supervised Tracking and Reconstruction of Rigid Objects in Motion With Neural RenderingWentao Yuan, Zhaoyang Lv, Tanner Schmidt, Steven LovegroveCVPR 2021
  4. No-Reference Image Quality Assessment Using Dynamic Complex-Valued Neural ModelZihan Zhou, Yong Xu, Ruotao Xu, Yuhui QuanACM MM 2022 · 7 citations
  5. Recurrent neural circuits for contour detectionDrew Linsley, Junkyung Kim, Alekh Ashok, Thomas SerreICLR 2020 · 48 citations
Dusk over the sea between two cliffs drawn in fine vertical lines