Online Cross-Modal Hashing with Multi-Level Memory

Online cross-modal hashing has recently gained significant attention due to its remarkable capability to handle cross-modal streaming data retrieval. Despite promising progress, existing methods still face challenges in fully exploiting the intricate relations across heterogeneous modalities and streaming data chunks, limiting the retrieval performance. In this paper, a novel Online Cross-modal Hashing method with Multi-level Memory (OCH-MM) is proposed. OCH-MM captures the cross-modal consistency and sample semantic correlations for discrete hash learning with latent feature disentanglement, and designs a multi-level memory framework for effective knowledge transfer. Specifically, for discriminative hash learning, OCH-MM maps the multi-modal data into a latent feature space that is further disentangled into a common Hamming space and a modality-specific feature space. The semantic correlations among samples are also preserved into discrete hash codes without relaxation in a nonlinear manner. For effectively learning from streaming data, OCH-MM designs an intra-space feature association memory, an inter-space feature association memory, and a hash codes memory, which encode the historical feature correlations within original multi-modal spaces, the feature correlations between original and latent space, and a subset of hash codes, respectively. By dynamically updating and utilizing the multi-level memory, the data correlations between different chunks are well explored and the historical knowledge is effectively reused to guide future learning. The proposed model is solved by an efficient discrete optimization algorithm. Experimental results on three benchmark datasets demonstrate that our proposed method achieves better retrieval accuracy over the state-of-the-art baselines.