Exploiting Positional Bias for Query-Agnostic Generative Content in Search

In recent years, research shows that neural ranking models (NRMs) substantially outperform their lexical counterparts in text retrieval. In traditional search pipelines, a combination of features leads to well-defined behaviour. However, as neural approaches become increasingly prevalent as the final scoring component of engines or as standalone systems, their robustness to malicious text and, more generally, semantic perturbation needs to be better understood. We posit that the transformer attention mechanism can induce exploitable defects in search models through sensitivity to token position within a sequence, leading to an attack that could generalise beyond a single query or topic. We demonstrate such defects by showing that non-relevant text-such as promotional contentcan be easily injected into a document without adversely affecting its position in search results. Unlike previous gradient-based attacks, we demonstrate the existence of these biases in a query-agnostic fashion. In doing so, without the knowledge of topicality, we can still reduce the negative effects of non-relevant content injection by controlling injection position. Our experiments are conducted with simulated on-topic promotional text automatically generated by prompting LLMs with topical context from target documents. We find that contextualisation of a non-relevant text further reduces negative effects whilst likely circumventing existing content filtering mechanisms. In contrast, lexical models are found to be more resilient to such content injection attacks. We then investigate a simple yet effective compensation for the weaknesses of the NRMs in search, validating our hypotheses regarding transformer bias. Position Generator α * nDCG@10 (∆) RR (∆) MRPR (∆) BM25 Abs Before GPT-3.5 0.1 0.414(+0.068) † 0.619(+0.14) 0.121(-0.136) † Abs Before Llama-2 0.1 0.414(+0.064) † 0.608(+0.050) 0.170(-0.154) † Abs Before Static 0.1 0.425(+0.031) 0.626(-0.025) 0.116(-0.081) † Abs Middle GPT-3.5 0.1 0.406(+0.060) † 0.616(+0.11) 0.130(-0.127) † Abs Middle Llama-2 0.1 0.406(+0.056) † 0.608(+0.050) 0.186(-0.138) † Abs Middle Static 0.1 0.417(+0.024) 0.625(-0.027) 0.120(-0.077) † Abs After GPT-3. 5 0.1 0.404(+0.058) † 0.616(+0.11) 0.139(-0.118) † Abs After Llama-2 0.1 0.406(+0.056) † 0.607(+0.049) 0 .192(-0.133) † Abs After Static 0.1 0.415(+0.021) 0.625(-0.027) 0.132(-0.065) † Rel Before GPT-3.5 0.1 0.414(+0.068) † 0.619(+0.14) 0.121(-0.136) † Rel Before Llama-2 0.1 0.414(+0.064) † 0.608(+0.050) 0.170(-0.154) † Rel Before Static 0.1 0.425(+0.031) 0.626(-0.025) 0.116(-0.081) † Rel After GPT-3.5 0.1 0.404(+0.058) † 0.616(+0.11) 0.139(-0.118) † Rel After Llama-2 0.1 0.406(+0.056) † 0.607(+0.049) 0.192(-0.133) † Rel After Static 0.1 0.415(+0.021) 0.625(-0.027) 0.132(-0.065) † ColBERT Abs Before GPT-3.5 0.3 0.648(+0.029) 0.862(+0.004) 0.119(-0.063) † Abs Before Llama-2 0.3 0.650(+0.021) 0.858(-0.002) 0.134(-0.032) † Abs Before Static 0.3 0.644(+0.028) 0.862(-0.002) 0.133(-0.039) † Abs Middle GPT-3.5 0.1 0.614(+0.060) † 0.815(-0.027) 0.112(-0.167) † Abs Middle Llama-2 0.2 0.617(+0.049) † 0.854(+0.008) 0.154(-0.084) † Abs Middle Static 0.2 0.618(+0.036) 0.853(-0.004) 0.142(-0.068) † Abs After GPT-3.5 0.1 0.607(+0.097) † 0.813(-0.024) 0.126(-0.208) † Abs After Llama-2 0.1 0.595(+0.075) † 0.798(-0.031) 0.155(-0.166) † Abs After Static 0.1 0.605(+0.053) † 0.815(-0.038) 0.105(-0.143) † Rel Before GPT-3.5 0.3 0.648(+0.029) 0.862(+0.004) 0.119(-0.063) † Rel Before Llama-2 0.3 0.650(+0.021) 0.858(-0.002) 0.134(-0.032) † Rel Before Static 0.3 0.644(+0.028) 0.862(-0.002) 0.133(-0.039) † Rel After GPT-3.5 0.1 0.607(+0.097) † 0.813(-0.024) 0.126(-0.208) † Rel After Llama-2 0.1 0.595(+0.075) † 0.798(-0.031) 0.155(-0.166) † Rel After Static 0.1 0.605(+0.053) † 0.815(-0.038) 0.105(-0.143) † monoT5 Abs Before GPT-3.5 0.9 0.634(+0.19) 0.838(-0.031) 0.143(-0.072) † Abs Before Llama-2 0.9 0.629(+0.15) 0.824(-0.042) 0.182(-0.042) † Abs Before Static 0.9 0.636(+0.000) 0.842(-0.031) 0.132(-0.039) † Abs Middle GPT-3.5 0.6 0.611(+0.051) † 0.782(-0.062) 0.127(-0.183) † Abs Middle Llama-2 0.6 0.605(+0.051) † 0.764(-0.072) † 0.165(-0.171) † Abs Middle Static 0.6 0.610(+0.036) 0.784(-0.075) † 0.111(-0.163) † Abs After GPT-3.5 0.6 0.606(+0.101) † 0.780(-0.035) 0.134(-0.272) † Abs After Llama-2 0.6 0.597(+0.091) † 0.759(-0.054) 0.172(-0.228) † Abs After Static 0.6 0.601(+0.073) † 0.782(-0.069) 0.127(-0.210) † Rel Before GPT-3.5 0.9 0.634(+0.19) 0.838(-0.031) 0.143(-0.072) † Rel Before Llama-2 0.9 0.629(+0.15) 0.824(-0.042) 0.182(-0.042) † Rel Before Static 0.9 0.636(+0.000) 0.842(-0.031) 0.132(-0.039) † Rel After GPT-3.5 0.6 0.606(+0.101) † 0.780(-0.035) 0.134(-0.272) † Rel After Llama-2 0.6 0.597(+0.091) † 0.759(-0.054) 0.172(-0.228) † Rel After Static 0.6 0.601(+0.073) † 0.782(-0.069) 0.127(-0.210) † Contriever Abs Before GPT-3.5 0.2 0.589(+0.054) † 0.783(+0.059) 0.151(-0.147) † Abs Before Llama-2 0.3 0.594(+0.051) 0.772(+0.047) 0.145(-0.155) † Abs Before Static 0.4 0.595(+0.0