Language Models as an Alternative Evaluator of Word Order Hypotheses: A Case Study in Japanese

We examine a methodology using neural language models (LMs) for analyzing the word order of language. This LM-based method has the potential to overcome the difficulties existing methods face, such as the propagation of preprocessor errors in count-based methods. In this study, we explore whether the LMbased method is valid for analyzing the word order. As a case study, this study focuses on Japanese due to its complex and flexible word order. To validate the LM-based method, we test (i) parallels between LMs and human word order preference, and (ii) consistency of the results obtained using the LM-based method with previous linguistic studies. Through our experiments, we tentatively conclude that LMs display sufficient word order knowledge for usage as an analysis tool. Finally, using the LMbased method, we demonstrate the relationship between the canonical word order and topicalization, which had yet to be analyzed by largescale experiments. Model Verbs whose type-A examples prefer the ACC-DAT order Verbs whose type-B examples prefer the ACC-DAT order CLM "預ける" (deposit), "置く" (place), "持つ" (have), "入 れる" (put in), "納める" (pay), "郵送" (mail), "供 給" (supply), "出す" (put out), "運ぶ" (transport), "流 す" (shed), "掛ける" (hang), "飾る" (decorate), "広 げる" (spread), "移す" (transfer), "残す" (leave), "配 送" (deliver), "送る" (send), "投げる" (throw), "送 付" (send), "返却" (return), "届ける" (send), "戻す" (return), "着ける" (wear), "上げる" (increase), "落と す" (drop), "載せる" (publish), "変更" (change), "納 入" (deliver), "卸す" (unload), "掲載" (publish), "通 す" (get X through) "配 布" (distribute), "渡 す" (pass), "プ レ ゼ ン ト" (present), "合わせる" (match), "見せる" (show), "提 供" (offer), "与える" (give), "当てる" (hit), "回す" (turn), "追加" (add), "貸す" (lend), "展示" (exhibit), "据える" (lay), "依頼" (request), "挿入" (insert), "纏 める" (collect), "請求" (claim) SLM "預ける" (deposit), "置く" (place), "頼む" (ask), "入 れる" (put in), "納める" (pay), "郵送" (mail), "出 す" (put out), "運ぶ" (transport), "流す" (shed), "掛け る" (hang), "広げる" (spread), "移す" (transfer), "残 す" (leave), "リクエスト" (request), "配送" (deliver), "送る" (send), "投げる" (throw), "送付" (send), "求 める" (ask), "提出" (submit), "届ける" (deliver), "要 求" (request), "戻す" (return), "寄付" (donate), "寄贈" (donation), "着ける" (wear), "乗せる" (place), "上げ る" (increase), "落とす" (drop), "貼る" (stick), "分け る" (divide), "ばらまく" (scatter), "はめる" (fit), "支 払う" (pay), "配達" (deliver), "卸す" (unload), "纏め る" (collect), "通す" (get X through) "プレゼント" (present), "持つ" (have), "合わせる" (match), "見せる" (show), "向ける" (point), "提供" (offer), "装備" (equip), "追加" (add), "展示" (exhibit), "据える" (lay), "採用" (adopt)