Learning Nanoscale Motion Patterns of Vesicles in Living Cells

Detecting and analyzing nanoscale motion patterns of vesicles, smaller than the microscope resolution (∼ 250 nm), inside living biological cells is a challenging problem. Stateof-the-art CV approaches based on detection, tracking, optical flow or deep learning perform poorly on this problem. We propose an integrative approach built upon physics-based simulations, nanoscopy algorithms and shallow residual attention network to permit for the first time analysis of subresolution motion patterns in vesicles, also of sub-resolution diameter. Our results show state-of-the-art performance, 89% validation accuracy on simulated dataset and 82% testing accuracy on an experimental dataset of images of living heart muscle cells grown under three different pathophysiologically relevant conditions. We demonstrate automated analysis of the motion states and changes in them for over 9000 vesicles. Such analysis will enable large scale biological studies of vesicle transport and interactions in living cells in the future.