Localizing Visual Sounds the Hard Way

We localise sound sources in videos without manual annotation. Our key contribution is an automatic negative mining technique through differentiable thresholding of a cross-modal correspondence score map into a Tri-map. We use background regions with low correlation to the given sound as 'hard negatives' in a contrastive learning framework.