Think Again! The Effect of Test-Time Compute on Preferences, Opinions, and Beliefs of Large Language Models

As Large Language Models (LLMs) become deeply integrated into human life and increasingly influence decision-making, it's crucial to evaluate whether and to what extent they exhibit subjective preferences, opinions, and beliefs. These tendencies may stem from biases within the models, which may shape their behavior, influence the advice and recommendations they offer to users, and potentially reinforce certain viewpoints. This paper presents the Preference, Opinion, and Belief survey (POBs), a benchmark developed to assess LLMs' subjective inclinations across societal, cultural, ethical, and personal domains. We applied our benchmark to evaluate leading open-and closed-source LLMs, measuring desired properties such as reliability, neutrality, and consistency. In addition, we investigated the effect of increasing the test-time compute, through reasoning and self-reflection mechanisms, on those metrics. While effective in other tasks, our results show that these mechanisms offer only limited gains in our domain. Furthermore, we reveal that newer model versions are becoming less consistent and more biased toward specific viewpoints, highlighting a blind spot and a concerning trend. POBs: https://ibm.github.io/POBS C Prompts and Templates Direct Prompt Format Question: QUESTION. Options: OPTIONS. Choose the most appropriate answer by selecting the corresponding letter (A, B, C, etc.). Provide your final answer inside <answer> </answer>, using ONLY the corresponding letter (A, B, C, etc.). If you refuse to answer, select the designated refusal option. NEVER provide explanations or additional details in your response. Your response format must be as follows: <answer>YOUR ANSWER</answer> Choose only one answer (one letter) that best aligns with your opinion. ENSURE that you include both the opening and closing HTML tags, as they are essential for correctly parsing your response. Example Responses: Example 1: Question: Should taxes on high-income individuals be increased? Options: A.