SCULPT: Systematic Tuning of Long Prompts

Prompt optimization is essential for effective utilization of large language models (LLMs) across diverse tasks. While existing optimization methods are effective in optimizing short prompts, they struggle with longer, more complex ones, often risking information loss and being sensitive to small perturbations. To address these challenges, we propose SCULPT (Systematic Tuning of Long Prompts), a framework that treats prompt optimization as a hierarchical tree refinement problem. SCULPT represents prompts as tree structures, enabling targeted modifications while preserving contextual integrity. It employs a Critic-Actor framework that generates reflections and applies actions to refine the prompt. Evaluations demonstrate SCULPT's effectiveness on long prompts, its robustness to adversarial perturbations, and its ability to generate high-performing prompts even without any initial human-written prompt. Compared to existing state of the art methods, SCULPT consistently improves LLM performance by preserving essential task information while applying structured refinements. Both qualitative and quantitative analyses show that SCULPT produces more stable and interpretable prompt modifications, ensuring better generalization across tasks. # Task Your task is to identify the type of translation error in a given source-translation pair. You will be given sentences with specific errors manually introduced. Determine which of the six error classes the translation error belongs to. # Error Identification Analyze the source -translation pair and identify the error based on the following classes : * Named entities : Look for changes in names , places , locations , etc . Examples: A company name changed from Apple to Microsoft, A country name changed from France to Germany, A person's name changed from John to Jack, A city name changed from New York to Los Angeles * Numerical values : Check for alterations in numbers , dates , or units . Examples: A date changed from 2021 to 2020, A price changed from $50 to$55, A time changed from 3 PM to 4 PM, A measurement unit changed from meters to feet, A quantity changed from 100 to 150 * Modifiers or adjectives : Identify changes in descriptors pertaining to a noun . Examples: The adjective changed from big to small., The descriptor changed from red to blue., The modifier changed from happy to sad., The descriptor changed from old to new., The adjective changed from tall to short. * Negation or antonyms : Detect the introduction or removal of negation , or changes to comparatives . Examples: The comparative changed from 'less important' to 'more important'., The negation changed from 'is not' to 'is'., The phrase changed from 'He is not happy' to 'He is happy'., The comparative changed from 'better' to 'worse'., The sentence changed from 'She never goes to the gym' to 'She always goes to the gym'. * Facts : Spot trivial factual errors not covered by the above classes . Examples: The fact changed from The capital of France is Paris to The capital of France is Berlin, The fact changed from Humans have 206 bones in their body to Humans have 210 bones in their body, The fact changed from The Great Wall of China is visible from space to The Great Wall of China is not visible from space * Dropped content : Notice if a significant clause is missing from the translation . Examples : A city name changed from ' Berlin ' to ' Munich ' would be a ' Named entities ' error , A date changed from '1990 ' to '1989 ' would be a ' Numerical values ' error # Performance Analysis Understand that language models perform differently across error classes: * Models like XLM -Roberta may struggle with named entities , dropped content , and modifiers / adjectives . * XNLI models also show poor performance on named entities and dropped content . # Additional points Keep in mind the following points while identifying errors: * Ensure minimal impact on translation fluency while identifying errors . * Focus on salient source information to detect errors effectively . * Remember that each translation contains only one of the six error classes . # Options # Task Your task is to identify the type of translation error in a given source-translation pair. You will be provided with sentences where specific classes of errors have been manually introduced. Determine which of the six error classes the translation error belongs to: Named entities, Numerical values, Modifiers or adjectives, Negation or antonyms, Facts, and Dropped content. Examples: The name 'John' was changed to 'James' in the translation, which is a 'Named entities' error., The word 'happy' was translated as 'sad', which is a 'Negation or antonyms' error., The number '50' was translated as '15', which is a 'Numerical values' error. # Error Identification Analyze the provided source -translation pair and identify the error based on the following classes : * Named entities: Look for changes in names, places, locations, scientific names, classificati