Exploring Mobile Touch Interaction with Large Language Models
Tim Zindulka, Jannek Maximilian Sekowski, Florian Lehmann, Daniel Buschek
Abstract
Interacting with Large Language Models (LLMs) for text editing on mobile devices currently requires users to break out of their writing environment and switch to a conversational AI interface. In this paper, we propose to control the LLM via touch gestures performed directly on the text. We first chart a design space that covers fundamental touch input and text transformations. In this space, we then concretely explore two control mappings: spread-to-generate and pinch-to-shorten, with visual feedback loops. We evaluate this concept in a user study (N=14) that compares three feedback designs: no visualisation, text length indicator, and length + word indicator. The results demonstrate that touch-based control of LLMs is both feasible and user-friendly, with the length + word indicator proving most effective for managing text generation. This work lays the foundation for further research into gesture-based interaction with LLMs on touch devices.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 8b2ac3be-c5ea-4b8c-9dea-2690980a1078Cited by top-tier papers1
Ask how each one uses itBuilds on11
- TaleBrush: Sketching Stories with Generative Pretrained Language ModelsJohn Joon Young Chung, Wooseok Kim, Kang Min Yoo, Hwaran Lee et al.CHI 2022 · 202 citations
- A Design Space for Intelligent and Interactive Writing AssistantsMina Lee, Katy Ilonka Gero, John Joon Young Chung, Simon Buckingham Shum et al.CHI 2024 · 133 citations
- DirectGPT: A Direct Manipulation Interface to Interact with Large Language ModelsDamien Masson, Sylvain Malacria, Géry Casiez, Daniel VogelCHI 2024 · 104 citations
- Will AI Console Me when I Lose my Pet? Understanding Perceptions of AI-Mediated Email WritingYihe Liu, Anushk Mittal, Diyi Yang, Amy S. BruckmanCHI 2022 · 86 citations
- Choice Over Control: How Users Write with Large Language Models using Diegetic and Non-Diegetic PromptingHai Dang, Sven Goller, Florian Lehmann, Daniel BuschekCHI 2023 · 80 citations
Related papers
- SHAPE-IT: Exploring Text-to-Shape-Display for Generative Shape-Changing Behaviors with LLMsWanli Qian, Chenfeng Gao, Anup Sathya, Ryo Suzuki et al.UIST 2024 · 12 citations
- Privacy Control in Conversational LLM Platforms: A Walkthrough StudyZhuoyang Li, Yanlai Wu, Yao Li, Xinning Gui et al.CHI 2026 · 1 citation
- How Beginning Programmers and Code LLMs (Mis)read Each OtherSydney Nguyen, Hannah McLean Babe, Yangtian Zi, Arjun Guha et al.CHI 2024 · 66 citations
- Analyzing Multimodal Interaction Strategies for LLM-Assisted Manipulation of 3D ScenesJunlong Chen, Jens Grubert, Per Ola KristenssonIEEE VR 2025 · 9 citations
- The HaLLMark Effect: Supporting Provenance and Transparent Use of Large Language Models in Writing with Interactive VisualizationMd. Naimul Hoque, Tasfia Mashiat, Bhavya Ghai, Cecilia D. Shelton et al.CHI 2024 · 48 citations
