Implicit Fine-tuning via Context Engineering: A Curriculum Learning Framework for Multimodal Entity Alignment
Yunpeng Hong, Chenyang Bu, Di Wu, Yi He, Xindong Wu
摘要
Multimodal Entity Alignment (MMEA) aims to identify equivalent entities across different modalities. While existing methods enhance MMEA performance through black-box context engineering strategies, their reliance on LLM parameter capacity and lack of theoretical interpretability remain unresolved. To this end, we first theoretically validate the mathematical equivalence between context engineering and model fine-tuning in MMEA tasks, demonstrating that prompt components simulate contrastive learning-based sequential fine-tuning in MMEA. Building on this foundation, we then propose PTFEA, a curriculum-learning-inspired framework that translates fine-tuning strategies into interpretable context engineering. Specifically, adaptive difficulty modulation dynamically adjusts information injection stages using confidence thresholds, establishing mathematical equivalence between curriculum learning weights and context sample selection; and three-stage progressive inference incorporates entity information from simple to complex cases, mirroring the gradient descent process in fine-tuning. Experiments on five public datasets demonstrate that PTFEA consistently outperforms strong baselines. In particular, on the ICWIKI dataset, PTFEA narrows the H@1 gap between Qwen2.5-72B and 14B to 0.6%. Moreover, compared with the representative context-engineering-based MMEA method MM-ChatAlign, PTFEA reduces the runtime of Qwen2.5-72B from 21 hours to 1 hour and lowers token consumption from 2200–3000 to 200–400, achieving over 80% reduction on the ICWIKI dataset. This work provides the first theoretical framework unifying context engineering and fine-tuning in MMEA, paving the way for future research that seeks to translate additional fine-tuning strategies into context engineering paradigms. Our code is available at https://github.com/DMiC-Lab-HFUT/PTFEA.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper30
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 被引用 47,906 次
- Dynamic Curriculum Learning for Imbalanced Data ClassificationYiru Wang, Weihao Gan, Jie Yang, Wei Wu 等ICCV 2019 · 被引用 263 次
- Visual Pivoting for (Unsupervised) Entity AlignmentFangyu Liu, Muhao Chen, Dan Roth, Nigel CollierAAAI 2021 · 被引用 159 次
- Boosting the Speed of Entity Alignment 10 ×: Dual Attention Matching Network with Normalized Hard Sample MiningXin Mao, Wenting Wang, Yuanbin Wu, Man LanWWW 2021 · 被引用 148 次
- Exploring and Evaluating Attributes, Values, and Structures for Entity AlignmentZhiyuan Liu, Yixin Cao, Liangming Pan, Juanzi Li 等EMNLP 2020 · 被引用 110 次
相关 Paper
- PSQE: A Theoretical-Practical Approach to Pseudo Seed Quality Enhancement for Unsupervised Multimodal Entity AlignmentYunpeng Hong, Chenyang Bu, Jie Zhang, Yi He 等KDD 2026
- Beyond General Alignment: Fine-Grained Entity-Centric Image-Text Matching with Multimodal Attentive ExpertsYaxiong Wang, Lianwei Wu, Lechao Cheng, Zhun Zhong 等SIGIR 2025 · 被引用 1 次
- IBMEA: Exploring Variational Information Bottleneck for Multi-modal Entity AlignmentTaoyu Su, Jiawei Sheng, Shicheng Wang, Xinghua Zhang 等ACM MM 2024 · 被引用 7 次
- ZeroEA: A Zero-Training Entity Alignment Framework via Pre-Trained Language ModelNan Huo, Reynold Cheng, Ben Kao, Wentao Ning 等VLDB 2024 · 被引用 16 次
- Tackling Uncertain Correspondences for Multi-Modal Entity AlignmentLiyi Chen, Ying Sun, Shengzhe Zhang, Yuyang Ye 等NeurIPS 2024 · 被引用 20 次
