Lune

ICML2026Top-tier venue

Learning the ESG Geometry with Domain Aware Language Models

Kunal Pradeep Pimparkhede, Chirayu Chaurasia, Jatin Roy, Mahesh Mohan Mohanachandran Radhamany

Abstract

Responsible investing aims to generate positive impact across Environment (E), Society (S), and Governance (G), and rating companies along these dimensions is now widespread, making ESG scores highly popular. Allocating retail capital with sustainability in mind could be transformational, yet it remains unclear how individual investors can do so in practice. Current ESG solutions cannot model high-dimensional, multi-modal time series capturing the joint evolution of ESG risks, financial returns, news, and sentiment, even though this domain requires jointly reasoning over distinct numerical signals where both numerical proximity and semantic type must be preserved. To bridge this gap, we introduce a novel domain-aware representation learning framework\textbf{representation learning framework} that learns geometry-preserving representations for heterogeneous time series using value-aware tokens with block-wise orthogonal embeddings\textbf{orthogonal embeddings}. To capture trajectory-level structure, we introduce FACET\textbf{FACET} tokens and train the model using a geometry-preserving loss. The resulting model jointly learns to forecast future values and to organize entities in a representation space that reflects their temporal evolution. Trained on ESG, returns, news, and sentiment, the domain-aware LLM learns a representation space that enables accurate ESG forecasting, trajectory-based grouping, and latent-space search for superior asset selection and downstream application like portfolio rebalancing

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting cites Reformer: The Efficient Transformer
  • TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis cites Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting
  • TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis cites Reformer: The Efficient Transformer
  • iTransformer: Inverted Transformers Are Effective for Time Series Forecasting cites Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting
  • iTransformer: Inverted Transformers Are Effective for Time Series Forecasting cites TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis
  • iTransformer: Inverted Transformers Are Effective for Time Series Forecasting cites Reformer: The Efficient Transformer
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on4

  1. Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series ForecastingHaixu Wu, Jiehui Xu, Jianmin Wang, Mingsheng LongNeurIPS 2021 · 5,824 citations
  2. Reformer: The Efficient TransformerNikita Kitaev, Lukasz Kaiser, Anselm LevskayaICLR 2020 · 2,878 citations
  3. iTransformer: Inverted Transformers Are Effective for Time Series ForecastingYong Liu, Tengge Hu, Haoran Zhang, Haixu Wu et al.ICLR 2024 · 1,703 citations
  4. TimesNet: Temporal 2D-Variation Modeling for General Time Series AnalysisHaixu Wu, Tengge Hu, Yong Liu, Hang Zhou et al.ICLR 2023 · 423 citations

Related papers

  1. DyMRL: Dynamic Multispace Representation Learning for Multimodal Event Forecasting in Knowledge GraphFeng Zhao, Kangzheng Liu, Teng Peng, Yu Yang et al.WWW 2026
  2. InvestESG: A multi-agent reinforcement learning benchmark for studying climate investment as a social dilemmaXiaoxuan Hou, Jiayi Yuan, Joel Z. Leibo, Natasha JaquesICLR 2025
  3. S2IP-LLM: Semantic Space Informed Prompt Learning with LLM for Time Series ForecastingZijie Pan, Yushan Jiang, Sahil Garg, Anderson Schneider et al.ICML 2024 · 23 citations
  4. Joint-Embedding Predictive Learning of Latent Market States in U.S. EquitiesSimon Mahns, Randall Balestriero, Mahmoud AssranICML 2026
  5. SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Drivingjingyu li, Junjie Wu, Dongnan Hu, Xiangkai Huang et al.CVPR 2026 · 36 citations
Dusk over the sea between two cliffs drawn in fine vertical lines