MoiréMop: Lightweight and Pervasive Moiré Removal for Mobile Camera-to-Screen Interaction

The widespread use of mobile devices and electronic displays has made camera-to-screen interaction, where users capture display screens with smartphones or smart glasses, a common way of recording and sharing information across applications such as real-time document analysis and AR/VR. However, such interaction often introduces unwanted colored stripe distortions known as moiré patterns. These patterns severely degrade image quality and hinder user experience in downstream tasks. Existing moiré removal methods process entire images uniformly, leading to high computational costs and detail degradation in moiré-free regions, while lacking generalizability across diverse real-world scenarios. In this paper, we propose MoiréMop, a lightweight, high-fidelity, and pervasive moiré removal method for mobile devices. MoiréMop localizes moiré-affected regions robustly and performs selective removal for efficiency and fidelity. To achieve robust and fine-grained moiré region extraction, we analyze the spectral aliasing mechanism of moiré pattern and reveal a unique symmetry between high- and low-frequency components. Leveraging this, we propose a Reversed Spectrum Consistency Map (RSCM) to estimate moiré presence and a Temporal Consistency Mask Propagation (TCMP) for real-time moiré region propagation across frames. For high-fidelity moiré removal, a Cross Spectral Adaptive Demoiré Network (CSADN) is designed to suppress moiré adaptively by modeling frequency symmetry. Extensive experiments demonstrate that MoiréMop achieves high-fidelity moiré removal and enhances segmentation accuracy by 18% while maintaining real-time efficiency on pervasive mobile devices, significantly improving user experience.