PColorizor: Re-coloring Ancient Chinese Paintings with Ideorealm-congruent Poems

Color restoration of ancient Chinese paintings plays a significant role in Chinese culture protection and inheritance. However, traditional color restoration is challenging and time-consuming because it requires professional restorers to conduct detailed literature reviews on numerous paintings for reference colors. After that, they have to fill in the inferred colors on the painting manually. In this paper, we present PColorizor, an interactive system that integrates advanced deep-learning models and novel visualizations to ease the difficulties of color restoration. PColorizor is established on the principle of poem-painting congruence. Given a color-faded painting, we employ both explicit and implicit color guidance implied by ideorealm-congruent poems to associate reference paintings. We propose a mountain-like visualization to facilitate efficient navigation of the color schemes extracted from the reference paintings. This visual representation allows users to easily see the color distribution over time at both the ideorealm and imagery levels. Moreover, we demonstrate the ideorealm understood by deep learning models through visualizations to bridge the communication gap between human restorers and deep learning models. We also adopt intelligent color-filling techniques to accelerate manual color restoration further. To evaluate PColorizor, we collaborate with domain experts to conduct two case studies to collect their feedback. The results suggest that PColorizor could be beneficial in enabling the effective restoration of color-faded paintings.