Action-Sketcher: From Reasoning to Action via Visual Sketches for Robotic Manipulation

Temporal Reasoning: The high-level instruction is "Clean the objects on the table". Okay, let's analyze step by step and break down tasks to accomplish this. The scene contains … What I have done is Nothing. Now I need to: "Pick up the cola bottle with the left arm and place into left basket". … … Sketching Spatial Reasoning: The cola bottle … in front of a green basket on the left. … The bounding box [42, 76, 74, 140]clearly frames the cola bottle. … The arrow originates at [59, 105], the center at [53, 28], and extends to [61, 77]… "bbox": [42, 76, 74, 140], "arrow": [[59, 105], [53, 28], [61, 77]]