Unified Category and Style Generalization for Instance-Level Sketch Retrieval

Zero-shot instance-level sketch retrieval addresses a practical retrieval scenario in which sketches from unseen categories during training serve as queries to retrieve matching RGB images. The core challenges of this task lie in two aspects: unknown category generalization and subjective style adaptation. Existing methods either focus solely on category generalization or apply simplistic style elimination techniques within a specific category, leading to suboptimal performance when both challenges are present. To this end, we propose the Dual-Attentive Prompt (DAP) method, which unifies category generalization and style adaptation into a single, interpretable framework. Central to DAP is a dual-attentive prompt composer, consisting of two self-attention-based modules. This composer dynamically integrates pre-learned category-specific knowledge with instance-specific prompts that adapt to sketch-specific styles. By cooperating with additional style alignment loss, the proposed method ensures robust generalization of unseen categories while mitigating the impact of subjective style variations. Extensive experimental results demonstrate the state-of-the-art performance of the proposed method. Additionally, some insights are provided into the challenges of traditional training processes when handling multi-style sketches, along with quantitative and qualitative evidence showing how the proposed approach effectively mitigates the negative impact of subjective style variations.