DiGraphHal-Bench: Evaluating Multimodal Large Language Models on Complex Directed Graphs

Element-Centric Class 1 -What / Which Q (Non-Semantic): Develop Solution -> 4. According to the graph, find the 1st blue-filled with pink-bordered node in order in the subpath. Q (Semantic): I have successfully reached Deploy Solution, what is the first critical milestone I must encounter in the four steps after this node? All the nodes in the picture that are pink-bordered and have blue fill-color represent critical milestones. Q (Semantic-Rewrite): I'm at Solution Deployment now, what is the first key milestone I must have passed within its four preceding steps? All the nodes in the picture that have blue fill and pink outline represent key milestones. A: Evaluate Performance Path-Centric Class 2 -How Q (Non-Semantic): Start Project -> -> Test and Validate. According to the graph, find at least 1 paths that include most yellow-bordered solid edges. Q (Semantic): ..(Similar to Semantic-Rewrite.) Q (Semantic-Rewrite): I'm at Beginning Project now, how can I reach Evaluate and Verify? Show me at least one possible path that covers the most important steps marked with yellow-bordered solid edges? A: Gather Resources -> Develop Solution Semantic Category Q (Semantic): Imagine you're working on a project where you need to Refine Solution before clearly Deploy Solution, however, it seems some steps are missing in between. What are they? Q (Semantic-Rewrite): Picture you're handling a project where Refining an Initial Solution is required prior to precisely Implementing Solutions. What are the steps in between? A