JuiceFS Enterprise 5.4: From 100 Billion Files to 1 Million Clients

JuiceFS Enterprise Edition 5.4, released recently, significantly enhances the file system's capabilities for large-scale deployments. The new version supports up to 1 million clients, a major improvement over previous versions. This is achieved by co-locating metadata nodes with network proxies, allowing clients to connect to any metadata node and reducing the number of physical connections.

The introduction of cross-zone directory cloning enables users to create independently modifiable copies of directories, even in a multi-zone architecture. This process only copies metadata, leaving the underlying object data shared, which improves efficiency. The clone takes approximately 1 minute and 40 seconds to clone 100 million files in a deployment with 30 metadata zones, averaging 1 million files per second.

Performance enhancements are another key feature of the new version. RDMA bandwidth utilization has been optimized, with a single client achieving 45 GB/s of read throughput in a test environment with 400 Gbps RDMA NICs. High-concurrency random-read IOPS has also seen a significant improvement, with a single client reaching up to 189,000 IOPS.

On-demand data synchronization is another feature introduced in JuiceFS Enterprise 5.4. This reduces data replication and is particularly useful in multi-region, multi-provider environments. The system's overall performance has been significantly improved, making it a robust solution for large-scale deployments.