Promi: Progressive Live Migration in Distributed Database Systems

Data partitioning serves as a fundamental technique in distributed database systems, but skewed and dynamic work-loads often cause imbalanced load distribution among nodes. Live migration is crucial for addressing this imbalance by redistributing data partitions across nodes. However, existing migration methods either continue processing heavy transaction loads on overloaded nodes or block and abort live transactions during migration, failing to achieve both fast load balance and transactional zero downtime simultaneously. This paper introduces Promi, a live data migration method that progressively migrates data at the granularity of mini-partitions instead of entire partitions. To ensure fast load balance, we propose a graph-based migration scheduler that prioritizes the migration of hot mini - partitions and minimizes potential distributed transactions during migration. To achieve zero downtime and improve system performance, we propose a transaction manager that judiciously routes and schedules the involved transactions based on the current migration state. We conduct extensive experiments com-paring Promi against various live migration methods. The results show that Promi achieves up to 1.5 × higher throughput and reduces load balance time by up to 60% compared to state-of-the-art methods.