AgentDet: A Shared-Blackboard Multi-Agent Framework for Zero-/Few-Shot Object Detection

Large multimodal language models have made rapid progress on vision-language tasks, yet their potential for zero-/few-shot object detection (ZSOD/FSOD) under a closed set of target classes has yet to be fully explored. ZSOD/FSOD is hampered by data scarcity and catastrophic forgetting. Although vision-language models (VLMs) demonstrated excellent performance on multiple benchmarks, they typically rely on large-scale visual pretraining, which is inconsistent with the goal of FSOD to test generalization to new categories under limited supervision. We introduce AgentDet, a shared-blackboard multiagent framework that unifies ZSOD and FSOD via pseudoincremental learning. AgentDet decouples detection into four cooperating roles-Agent-Scout, Agent-Pinner, Agent-Curator, and Agent-Judge-which collaboratively maintain a Shared Blackboard and a Knowledge Base. For efficiency, we only train Agent-Judge by updating its image encoder and LLM-based detection head, which is a lightweight recipe that encourages generalization to previously unseen categories. On PASCAL VOC and MS COCO ZSOD/FSOD datasets, AgentDet achieves strongly competitive performance with state-of-the-art results in several settings.