Do Language Models Understand Honorific Systems in Javanese?

The Javanese language features a complex system of honorifics that vary according to the social status of the speaker, listener, and referent. Despite its cultural and linguistic significance, there has been limited progress in developing a comprehensive corpus to capture these variations for natural language processing (NLP) tasks. In this paper, we present UNGGAH-UNGGUH 1 , a carefully curated dataset designed to encapsulate the nuances of Unggah-Ungguh Basa, the Javanese speech etiquette framework that dictates the choice of words and phrases based on social hierarchy and context. Using UNGGAH-UNGGUH, we assess the ability of language models (LMs) to process various levels of Javanese honorifics through classification and machine translation tasks. To further evaluate cross-lingual LMs, we conduct machine translation experiments between Javanese (at specific honorific levels) and Indonesian. Additionally, we explore whether LMs can generate contextually appropriate Javanese honorifics in conversation tasks, where the honorific usage should align with the social role and contextual cues. Our findings indicate that current LMs struggle with most honorific levels, exhibiting a bias toward certain honorific tiers. * Equal contribution. † The work was conducted outside Capital One. ‡ Senior authors. 1 We released our models & dataset in https:// huggingface.co/JavaneseHonorifics Ngoko Mbak Srini mangan pecel ajange pincuk. Ngoko Alus Mbak Srini dhahar pecel ambenge pincuk. Krama Mbak Srini nedha pecel ajangipun pincuk. Krama Alus Mbak Srini dhahar pecel ambengipun pincuk. Translation Mbak Srini makan pecel dengan ajang pincuk. Translation Mbak Srini ate pecel with pincuk. Ngoko Bu Haiti arep tuku gula selawe kilo, wis dak-dolane . Ngoko Alus Bu Haiti arep mundhut gula selawe kilo, wis adalem dolane Krama Bu Haiti badhe tumbas gendhis selangkung kilo, sampun kula sadeanipun . Krama Alus Bu Haiti badhe mundhut gendhis selangkung kilo, sampun adalem sadeanipun . Translation Bu Haiti akan membeli gula dua puluh lima kilo, sudah saya jualnya. Translation Mrs. Haiti is going to buy twenty-five kilos of sugar, I have already sold it.