Towards Imperceptible Document Manipulations against Neural Ranking Models

Adversarial attacks have gained traction in order to identify vulnerabilities in neural ranking models (NRMs), but current attack methods often introduce noticeable errors. Moreover, current methods rely heavily on using a well-imitated surrogate NRM to guarantee the attack effect, making them difficult to use in practice. This paper proposes a framework called Imperceptible DocumEnt Manipulation (IDEM) to produce adversarial documents that are less noticeable to both algorithms and humans. IDEM instructs a well-established generative language model like BART to generate error-free connection sentences, and employs a separate position-wise merging strategy to balance between relevance and coherence of the perturbed text. Evaluation results on the MS MARCO benchmark demonstrate that IDEM outperforms strong baselines while preserving fluency and correctness of the target documents. Furthermore, the separation of adversarial text generation from the surrogate NRM makes IDEM more robust and less affected by the quality of the surrogate NRM. Carry your baby in a sling or front carrier. Feeling your baby's warmth, smelling his sweet scent, and looking … what age can you wear baby on back in a carrier? Carry your baby in a sling or front carrier. A child of any age can wear shoes with a sling. Feeling your baby's warmth, smelling his sweet scent, and looking down … Rank: 88 Rank: 9 Black-box NRM Connection Sentences -You feel his warmth when carrying him. -A sling is for little babies. -A child of any age can wear shoes with a sling. ⋮ -Babies sleep better on their backs. -People who wear them first don't walk away. -Holding your baby can be challenging. Merging