Lune

ACL Findings2026

NASH: Numerically Aware Scoring Heuristic for Robust Semantic Similarity

Yu-Shiang Huang, Yun-Yu Lee, Tzu-Hsin Chou, Che Lin, Chuan-Ju Wang

Abstract

Numerical precision is critical in financial NLP, yet embedding-based semantic similarity metrics exhibit numerical blindness-failing to distinguish contradictory values within similar contexts.We introduce NASH (Numerically Aware Scoring Heuristic), a model-agnostic metric that decouples numerical verification from textual semantic evaluation through a three-stage pipeline: (1) modal separation via numeric masking, (2) dual-channel similarity estimation through masked-text similarity and context-aware numeric alignment, and (3) IDF-weighted aggregation.NASH functions as a drop-in enhancement to existing embedding-based metrics.Validated on our proposed NumFinE financial numerical evaluation benchmark and established semantic similarity datasets (STS-B, Financial-STS), NASH achieves substantial improvements in numerical sensitivity (up to +159.6% on listwise ranking) while preserving general semantic performance, establishing a reliable standard for numeracy-aware evaluation. 1 1 The evaluation code and benchmark datasets are available at https://github.com/cnclabs/codes.fin.bertscore.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Large Language Models are Zero-Shot Reasoners cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on8

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
  3. Deberta: decoding-Enhanced Bert with Disentangled AttentionPengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu ChenICLR 2021 · 3,729 citations
  4. MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained TransformersWenhui Wang, Furu Wei, Li Dong, Hangbo Bao et al.NeurIPS 2020 · 2,727 citations
  5. SimCSE: Simple Contrastive Learning of Sentence EmbeddingsTianyu Gao, Xingcheng Yao, Danqi ChenEMNLP 2021 · 2,496 citations

Related papers

  1. FinMTEB: Finance Massive Text Embedding BenchmarkYixuan Tang, Yi YangEMNLP 2025 · 4 citations
  2. Financial Language Model Evaluation (FLaME)Glenn Matlin, Mika Okamoto, Huzaifa Pardawala, Yang Yang et al.ACL Findings 2025 · 3 citations
  3. When FLUE Meets FLANG: Benchmarks and Large Pretrained Language Model for Financial DomainRaj Sanjay Shah, Kunal Chawla, Dheeraj Eidnani, Agam Shah et al.EMNLP 2022 · 63 citations
  4. FinMaster: A Holistic Benchmark for Full-Pipeline Financial Management with Large Language ModelsJunzhe Jiang, Chang Yang, Aixin Cui, Sihan Jin et al.ACL Findings 2026
  5. FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial ReasoningZhuohan Xie, Daniil Orel, Rushil Thareja, Dhruv Sahnan et al.ACL 2026 · 13 citations
Dusk over the sea between two cliffs drawn in fine vertical lines