A Peer-review Look on Multi-modal Clustering: An Information Bottleneck Realization Method

Despite the superior capability in complementary information exploration and consistent clustering structure learning, most current weight-based multi-modal clustering methods still contain three limitations: 1) lack of trustworthiness in learned weights; 2) isolated view weight learning; 3) extra weight parameters. Motivated by the peer-review mechanism in the academia, we in this paper give a new peer-review look on the multi-modal clustering problem and propose to iteratively treat one modality as "author" and the remaining modalities as "reviewers" so as to reach a peer-review score for each modality. It essentially explores the underlying relationships among modalities. To improve the trustworthiness, we further design a new trustworthy score with a self-supervision working mechanism. Following that, we propose a novel Peer-review Trustworthy Information Bottleneck (PTIB) method for weighted multi-modal clustering, where both the above scores are simultaneously taken into account for accurate and parameter-free modality weight learning. Extensive experiments on eight multi-modal datasets suggest that PTIB can outperform the state-of-theart multi-modal clustering methods. A specific example Reviewer 2 Author review result result Accept Revise Reject Accept Revise Reject General peer-review Author /Reviewer Author /Reviewer Author /Reviewer "Peer-review" look on multimodal clustering Modality 3 Modality 1 Modality 2 A specific case Modality 2 "review" Modality 1 result result 0 1 The score 0 1 The score Modality 3