Dragonfly

Dragonfly is an open-source, cloud-native file distribution and image acceleration system that leverages peer-to-peer (P2P) technology to optimize large-scale data delivery. It is designed to improve download speeds and bandwidth utilization by enabling peers to share idle bandwidth, reducing reliance on centralized sources. Dragonfly supports distribution of container images, files, AI models, datasets, caches, logs, and dependencies, making it suitable for cloud-native environments and AI infrastructure.

The system architecture includes four main components: Manager, Scheduler, Seed Peer, and Peer. The Manager oversees cluster relationships and configuration management, providing a front-end console for visual cluster operations. The Scheduler selects optimal download sources and coordinates data flow. Seed Peers act as root nodes for back-to-source downloads, while Peers handle upload and download tasks within the P2P network. This architecture ensures efficient, stable, and consistent file delivery with exception isolation at multiple levels to enhance reliability.

Dragonfly integrates non-intrusively with container runtimes, container registries, AI infrastructure, and download tools. It offers configurable peer limits for load, concurrency, and traffic, ensuring flexible resource management. The system guarantees file consistency even without explicit user checks. Dragonfly has been adopted in production by various companies and is a Cloud Native Computing Foundation (CNCF) incubating project. Developers can get started by deploying Dragonfly components according to the documentation, integrating with existing container or AI workflows, and leveraging the provided APIs and web console for management.