EMNLP Findings2022
Conditional Supervised Contrastive Learning for Fair Text Classification
Jianfeng Chi, William Shand, Yaodong Yu, Kai-Wei Chang, Han Zhao, Yuan Tian
Abstract
Contrastive representation learning has gained much attention due to its superior performance in learning representations from both image and sequential data. However, the learned representations could potentially lead to performance disparities in downstream tasks, such as increased silencing of underrepresented groups in toxicity comment classification. In light of this challenge, in this work, we study learning fair representations that satisfy a notion of fairness known as equalized odds for text classification via contrastive learning. Specifically, we first theoretically analyze the connections between learning representations with a fairness constraint and conditional supervised contrastive objectives, and then propose to use conditional supervised contrastive objectives to learn fair representations for text classification. We conduct experiments on two text datasets to demonstrate the effectiveness of our approaches in balancing the trade-offs between task performance and bias mitigation among existing baselines for text classification. Furthermore, we also show that the proposed methods are stable in different hyperparameter settings. 1
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
4 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Pushing the Accuracy-Group Robustness Frontier with Introspective Self-play cites A Simple Framework for Contrastive Learning of Visual Representations
- Pushing the Accuracy-Group Robustness Frontier with Introspective Self-play cites WILDS: A Benchmark of in-the-Wild Distribution Shifts
- Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere cites Data-Efficient Image Recognition with Contrastive Predictive Coding
- Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere cites A Simple Framework for Contrastive Learning of Visual Representations
- A Simple Framework for Contrastive Learning of Visual Representations cites Data-Efficient Image Recognition with Contrastive Predictive Coding
- DualCL: Principled Supervised Contrastive Learning as Mutual Information Maximization for Text Classification cites A Simple Framework for Contrastive Learning of Visual Representations
- DualCL: Principled Supervised Contrastive Learning as Mutual Information Maximization for Text Classification cites Supervised Contrastive Learning
- Supervised Contrastive Learning cites Data-Efficient Image Recognition with Contrastive Predictive Coding
- Supervised Contrastive Learning cites Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere
- Supervised Contrastive Learning cites A Simple Framework for Contrastive Learning of Visual Representations
- Factorized Contrastive Learning: Going Beyond Multi-view Redundancy cites A Simple Framework for Contrastive Learning of Visual Representations
- Factorized Contrastive Learning: Going Beyond Multi-view Redundancy cites Supervised Contrastive Learning
- MABEL: Attenuating Gender Bias using Textual Entailment Data cites Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere
- MABEL: Attenuating Gender Bias using Textual Entailment Data cites A Simple Framework for Contrastive Learning of Visual Representations
Cited by top-tier papers4
- Factorized Contrastive Learning: Going Beyond Multi-view RedundancyPaul Pu Liang, Zihao Deng, Martin Q. Ma, James Y. Zou et al.NeurIPS 2023 · 137 citations
- MABEL: Attenuating Gender Bias using Textual Entailment DataJacqueline He, Mengzhou Xia, Christiane Fellbaum, Danqi ChenEMNLP 2022 · 15 citations
- DualCL: Principled Supervised Contrastive Learning as Mutual Information Maximization for Text ClassificationJunfan Chen, Richong Zhang, Yaowei Zheng, Qianben Chen et al.WWW 2024 · 4 citations
- Pushing the Accuracy-Group Robustness Frontier with Introspective Self-playJeremiah Zhe Liu, Krishnamurthy (Dj) Dvijotham, Jihyeon Lee, Quan Yuan et al.ICLR 2023
Builds on21
- A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
- Supervised Contrastive LearningPrannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna et al.NeurIPS 2020 · 7,049 citations
- Understanding Contrastive Representation Learning through Alignment and Uniformity on the HypersphereTongzhou Wang, Phillip IsolaICML 2020 · 2,360 citations
- WILDS: A Benchmark of in-the-Wild Distribution ShiftsPang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie et al.ICML 2021 · 1,773 citations
- Data-Efficient Image Recognition with Contrastive Predictive CodingOlivier J. HénaffICML 2020 · 1,553 citations
Related papers
- Fair Contrastive Learning for Facial Attribute ClassificationSungho Park, Jewook Lee, Pilhyeon Lee, Sunhee Hwang et al.CVPR 2022 · 61 citations
- An Investigation of Representation and Allocation Harms in Contrastive LearningSubha Maity, Mayank Agarwal, Mikhail Yurochkin, Yuekai SunICLR 2024 · 2 citations
- Simple-Sampling and Hard-Mixup with Prototypes to Rebalance Contrastive Learning for Text ClassificationMengyu Li, Yonghao Liu, Fausto Giunchiglia, Ximing Li et al.WWW 2026 · 7 citations
- Conditional Contrastive Learning with KernelYao-Hung Hubert Tsai, Tianqin Li, Martin Q. Ma, Han Zhao et al.ICLR 2022 · 29 citations
- Conditional Learning of Fair RepresentationsHan Zhao, Amanda Coston, Tameem Adel, Geoffrey J. GordonICLR 2020 · 127 citations
