Accelerating Metadata Management of DFS via Speculative Permission Checking

Modern distributed file systems face a critical metadata bottleneck as cloud computing and large-scale machine learning applications generate massive datasets with billions of files. Traditional metadata managements suffer from expensive path resolution: each operation requires $O(N)$ remote procedure calls and database queries to recursively check permissions and locate inodes across hierarchical namespaces. This paper revisits real-world namespace structures and reveals that the vast majority of metadata exhibits a permission descending pattern, enabling safe shortcuts for permission checking. We leverage this insight to design SPMeta, a scalable metadata management that integrates two key techniques: (1) Speculative Permission Checking, which bypasses expensive permission checking while preserving POSIX semantics, and (2) MART, a metadata-optimized trie structure that accelerates both permission checking and inode indexing. By combining these, SPMeta reduces the number of remote procedure calls (RPCs) and database queries for most metadata operations from $O(N)$ to $O(1)$. Evaluation in five real-world namespaces shows that SPMeta achieves superior metadata performance in read-only and mixed workloads, improving aggregate throughput by $\mathbf{1. 4 4} \times$ to $\mathbf{2. 8 6} \times$ over state-of-the-art baselines while scaling to distributed file systems with tens of billions of files.