Lune

ACL Findings2026

Privacy Risks of Intermediate Representations: Attribute Inference in Distributed LLM Inference

Yang Lyu, Jin Cao, Yang Xiao, Zhe Sun, Ben Niu, Fenghua Li, Hui Li

Abstract

Distributed LLM inference avoids sending raw inputs by transmitting intermediate hidden states, a practice widely assumed to preserve privacy. We challenge this assumption and demonstrate that intermediate representations alone are sufficient to leak sensitive user attributes. This setting poses a fundamental obstacle for existing attribute inference attacks, which typically rely on auxiliary embeddingattribute pairs. To characterize this previously underexplored privacy risk, we reformulate attribute inference as zero-shot matching over candidate attributes directly in the intermediate representation space, and introduce a purely intermediate-representation-based attribute inference attack, termed IR-AIA. To address two structural challenges that hinder attribute inference from intermediate representations, we propose SG-APCR to address layer-dependent anisotropy in intermediate embeddings and a sliding-window similarity matching strategy to handle subword-level semantic fragmentation. Experiments across three LLMs and three realworld datasets show that sensitive attributes can be reliably inferred using only intermediate representations, achieving Top-1 accuracy of up to 0.997 on CMS, 0.980 on Skytrax, and 0.986 on ECHR. These results reveal that intermediate states commonly considered safe to share can expose sensitive personal attributes on their own.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Information Leakage in Embedding Models cites Privacy Risks of General-Purpose Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on19

  1. Privacy Risks of General-Purpose Language ModelsXudong Pan, Mi Zhang, Shouling Ji, Min YangS&P 2020 · 291 citations
  2. Beyond Memorization: Violating Privacy via Inference with Large Language ModelsRobin Staab, Mark Vero, Mislav Balunovic, Martin T. VechevICLR 2024 · 211 citations
  3. Information Leakage in Embedding ModelsCongzheng Song, Ananth RaghunathanCCS 2020 · 200 citations
  4. HexGen: Generative Inference of Large Language Model over Heterogeneous EnvironmentYouhe Jiang, Ran Yan, Xiaozhe Yao, Yang Zhou et al.ICML 2024 · 46 citations
  5. Are Attribute Inference Attacks Just Imputation?Bargav Jayaraman, David EvansCCS 2022 · 42 citations

Related papers

  1. Prompt Inference Attack on Distributed Large Language Model Inference FrameworksXinjian Luo, Ting Yu, Xiaokui XiaoCCS 2025
  2. Prompt Inversion Attack Against Collaborative Inference of Large Language ModelsWenjie Qu, Yuguang Zhou, Yongji Wu, Tingsong Xiao et al.S&P 2025
  3. Reconstruction Attack-Resistant Inference Paradigm for LLM Cloud ServicesZipeng Ye, Wenjian Luo, Qi Zhou, Yubo TangAAAI 2026
  4. R.R.: Unveiling LLM Training Privacy through Recollection and RankingWenlong Meng, Zhenyuan Guo, Lenan Wu, Chen Gong et al.ACL Findings 2025 · 2 citations
  5. Hidden No More: Attacking and Defending Private Third-Party LLM InferenceRahul Krishna Thomas, Louai Zahran, Erica Choi, Akilesh Potti et al.ICML 2025
Dusk over the sea between two cliffs drawn in fine vertical lines