Null-Shot Prompting: Rethinking Prompting Large Language Models With Hallucination

This paper investigates an interesting phenomenon where we observe performance increases in large language models (LLMs) when providing a prompt that causes and exploits hallucination.We propose null-shot prompting, a counter-intuitive approach where we deliberately instruct LLMs to reference a null, nonexistent, section.We evaluate null-shot prompting across a variety of tasks, including arithmetic reasoning, commonsense reasoning, and reading comprehension.Notably, we observe a substantial increase in performance in arithmetic reasoning tasks for various models, with up to a 44.62% increase compared to a baseline in one model.Additional experiments on more complex mathematical problem-solving and hallucination detection benchmarks also reveal similar benefits from this approach.Furthermore, we explore the effects of combining reasoning, which typically mitigates hallucination, with hallucination within the prompt and find several cases of performance improvements.We hope this paper stimulates further interest, investigation, and discussion on how hallucination in prompts may not only affect LLMs but, in certain cases, enhance their performance.GPT-4 Turbo -0.79% -5.35% -0.9% 1.22% -8.48% -4.08% -11.54%Claude 2.1 -8.53% -7.46% -7.81% -3.81% -6.36% 0.88% 11.11% Claude 3 Haiku -4.94% -1.22% 5.75% 2.34% -4.84% 3.01% 3.88% Claude 3 Sonnet 0.9% 1.59% -7.58% -3.7% -12.9% -0.65% -3.74% Claude 3 Opus -1.87% -2.35%