Lune

ACL Findings2023

TextObfuscator: Making Pre-trained Language Model a Privacy Protector via Obfuscating Word Representations

Xin Zhou, Yi Lu, Ruotian Ma, Tao Gui, Yuran Wang, Yong Ding, Yibo Zhang, Qi Zhang, Xuanjing Huang

Abstract

In real-world applications, pre-trained language models are typically deployed on the cloud, allowing clients to upload data and perform compute-intensive inference remotely. To avoid sharing sensitive data directly with service providers, clients can upload numerical representations rather than plain text to the cloud. However, recent text reconstruction techniques have demonstrated that it is possible to transform representations into original words, suggesting that privacy risk remains. In this paper, we propose TextObfuscator, a novel framework for preserving inference privacy by applying random perturbations to clustered representations. The random perturbations make each word representation indistinguishable from surrounding functionally similar representations, thus obscuring word information while retaining the original word functionality. To achieve this, we utilize prototypes to learn clustered representations, where words of similar functionality are encouraged to be closer to the same prototype during training. Additionally, we design different methods to find prototypes for token-level and sentencelevel tasks, which can improve performance by incorporating semantic and task information. Experimental results on token and sentence classification tasks show that TextObfuscator achieves improvement over compared methods without increasing inference cost.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

8 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Text Embedding Inversion Security for Multilingual Language Models cites Information Leakage in Embedding Models
  • Iron: Private Inference on Transformers cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • Anti-adversarial Learning: Desensitizing Prompts for Large Language Model cites Iron: Private Inference on Transformers
  • Recovering Private Text in Federated Learning of Language Models cites Information Leakage in Embedding Models
  • Cape: Context-Aware Prompt Perturbation Mechanism with Differential Privacy cites Information Leakage in Embedding Models
  • ObfusLM: Privacy-preserving Language Model Service against Embedding Inversion Attacks cites Information Leakage in Embedding Models
  • Information Leakage in Embedding Models cites Privacy Risks of General-Purpose Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers8

  1. Text Embedding Inversion Security for Multilingual Language ModelsYiyi Chen, Heather C. Lent, Johannes BjervaACL 2024 · 10 citations
  2. Leaky Thoughts: Large Reasoning Models Are Not Private ThinkersTommaso Green, Martin Gubri, Haritz Puerto, Sangdoo Yun et al.EMNLP 2025 · 2 citations
  3. ObfusLM: Privacy-preserving Language Model Service against Embedding Inversion AttacksYu Lin, Ruining Yang, Yunlong Mao, Qizhi Zhang et al.ACL 2025 · 2 citations
  4. Anti-adversarial Learning: Desensitizing Prompts for Large Language ModelXuan Li, Zhe Yin, Xiaodong Gu, Beijun ShenAAAI 2026
  5. Cape: Context-Aware Prompt Perturbation Mechanism with Differential PrivacyHaoqi Wu, Wei Dai, Li Wang, Qiang YanICML 2025

Builds on10

  1. An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
  2. Privacy Risks of General-Purpose Language ModelsXudong Pan, Mi Zhang, Shouling Ji, Min YangS&P 2020 · 291 citations
  3. Iron: Private Inference on TransformersMeng Hao, Hongwei Li, Hanxiao Chen, Pengzhi Xing et al.NeurIPS 2022 · 209 citations
  4. Information Leakage in Embedding ModelsCongzheng Song, Ananth RaghunathanCCS 2020 · 200 citations
  5. Recovering Private Text in Federated Learning of Language ModelsSamyak Gupta, Yangsibo Huang, Zexuan Zhong, Tianyu Gao et al.NeurIPS 2022 · 120 citations

Related papers

  1. TextFusion: Privacy-Preserving Pre-trained Model Inference via Token FusionXin Zhou, Jinzhu Lu, Tao Gui, Ruotian Ma et al.EMNLP 2022 · 12 citations
  2. TextMixer: Mixing Multiple Inputs for Privacy-Preserving InferenceXin Zhou, Yi Lu, Ruotian Ma, Tao Gui et al.EMNLP Findings 2023 · 3 citations
  3. Ensembler: Protect Collaborative Inference Privacy from Model Inversion Attack via Selective EnsembleDancheng Liu, Chenhui Xu, Jiajie Li, Amir Nassereldine et al.DAC 2025
  4. NOIR: Privacy-Preserving Generation of Code with Open-Source LLMsKhoa Nguyen, Khiem Ton, NhatHai Phan, Issa Khalil et al.USENIX Security 2026 · 2 citations
  5. Reconstruction Attack-Resistant Inference Paradigm for LLM Cloud ServicesZipeng Ye, Wenjian Luo, Qi Zhou, Yubo TangAAAI 2026
Dusk over the sea between two cliffs drawn in fine vertical lines