Aligning Actions Across Recipe Graphs
Lucia Donatelli, Theresa Schmidt, Debanjali Biswas, Arne Köhn, Fangzhou Zhai, Alexander Koller
Abstract
Recipe texts are an idiosyncratic form of instructional language that pose unique challenges for automatic understanding. One challenge is that a cooking step in one recipe can be explained in another recipe in different words, at a different level of abstraction, or not at all. Previous work has annotated correspondences between recipe instructions at the sentence level, often glossing over important correspondences between cooking steps across recipes. We present a novel and fully-parsed English recipe corpus, ARA (Aligned Recipe Actions), which annotates correspondences between individual actions across similar recipes with the goal of capturing information implicit for accurate recipe understanding. We represent this information in the form of recipe graphs, and we train a neural model for predicting correspondences on ARA. We find that substantial gains in accuracy can be obtained by taking fine-grained structural information about the recipes into account.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers6
- Differentiable Task Graph Learning: Procedural Activity Representation and Online Mistake Detection from Egocentric VideosLuigi Seminara, Giovanni Maria Farinella, Antonino FurnariNeurIPS 2024 · 36 citations
- Counterfactual Recipe Generation: Exploring Compositional Generalization in a Realistic ScenarioXiao Liu, Yansong Feng, Jizhi Tang, Chengang Hu et al.EMNLP 2022 · 6 citations
- Generating Coherent Sequences of Visual Illustrations for Real-World Manual TasksJoão Bordalo, Vasco Ramos, Rodrigo Valerio, Diogo Glória-Silva et al.ACL 2024 · 1 citation
- AutoDSL: Automated domain-specific language design for structural representation of procedures with constraintsYu-Zhe Shi, Haofei Hou, Zhangqian Bi, Fanxu Meng et al.ACL 2024
- CaT-Bench: Benchmarking Language Model Understanding of Causal and Temporal Dependencies in PlansYash Kumar Lal, Vanya Cohen, Nathanael Chambers, Niranjan Balasubramanian et al.EMNLP 2024
Builds on2
Related papers
- Multi-modal Cooking Workflow Construction for Food RecipesLiangming Pan, Jingjing Chen, Jianlong Wu, Shaoteng Liu et al.ACM MM 2020 · 20 citations
- CHEF: Cross-modal Hierarchical Embeddings for Food Domain RetrievalHai Xuan Pham, Ricardo Guerrero, Vladimir Pavlovic, Jiatong LiAAAI 2021 · 22 citations
- Learning Program Representations for Food Images and Cooking RecipesDim P. Papadopoulos, Enrique Mora, Nadiia Chepurko, Kuan Wei Huang et al.CVPR 2022 · 38 citations
- Mitigating Cross-modal Representation Bias for Multicultural Image-to-Recipe RetrievalQing Wang, Chong-Wah Ngo, Yu Cao, Ee-Peng LimACM MM 2025
- Hybrid Fusion with Intra- and Cross-Modality Attention for Image-Recipe RetrievalJiao Li, Xing Xu, Wei Yu, Fumin Shen et al.SIGIR 2021 · 21 citations
