Introducing Sonic: High-performance S3-compatible object storage with hourly billing
From RAID to true object storage tech 🙏
Ever since our inception back in 2017 our most successful product has been our SFS scalable file storage solution. SFS allowed customers to upload their content with ease and store it directly inside the heart of our global CDN. From there their files would be delivered to their users via optimised routes to our edge network. SFS has been great as a managed content origin, but it was built on top of legacy technology - RAID10 arrays. As the popularity of SFS was growing some drawbacks become apparent:
🙊 No easy solution for adding new nodes to the storage cluster.
🙉 No efficient way to re-balance files when new node is added.
🙈 Downtime of a single server means downtime for some of the files.
🏳 During the lengthy RAID rebuild the entire array may be lost.
💰 Requires costly multi-region replication for durability.
🛠 Uses legacy APIs and outdated management tools.
We started looking for a solution to these problems back in 2021, and after a lot of R&D and testing we found the perfect open source project to build on top of. SeaweedFS is a state of the art object storage system which builds on ideas from Facebook's Haystack project. It is fast, resilient, designed for instant file access and virtually unlimited scalability. RAID is replaced with efficient Reed-Solomon erasure coding for warm data, while hot data is replicated for maximum speed and availability. Availability of all objects is no longer a subject to a single array failure and Sonic achieves the amazing nine-nines (99.9999999%👌) of durability, or a cumulative probability of 2.895e-13 of not losing a file, all from a single region. No multi-region replication needed!
Fast object storage powered by NVMe, a global edge network and ActivePush™ ⚡
As a content delivery network, speed is our top priority. Competitive object storage solutions have traditionally been built on the premises of scalability and long-term durability, while delivery of the stored files has always been an afterthought. Usually, an object storage service would either be too slow and inefficient when used for mass file downloads and media playback, or it would become too costly when used at scale. 🙅
Alternatively, existing performance-oriented storage solutions always come at high cost when durability is required, or use legacy technologies and while marketed as object storage, they bring none of the actual benefits of true object storage technologies.
Building on all the benefits of the great design of SeaweedFS, and soon to be bundled with our own tech like ActivePush™, Sonic does not only achieve great durability, but provides consistently high speeds globally. With in-memory maps and efficient storage of metadata file access is consistently instant requiring only a O(1) disk seek operations.
On a hardware level Sonic spots a 2-layer storage setup with NVMe and HDD disk drives across it's entire cluster of servers. NVMe drives are used to insure instant write and read operations for hot data, as well as durability via fast replication. HDDs store warm files and take care of the Reed-Solomon erasure coding for efficient and durable long term storage, equal to 3x replication, while occupying only 1.2x the space 🎯
Unbeatable price-to-performance ratio. And the lowest storage costs for hot content in the world 👊
We've designed Sonic for ultimate speed at scale and we want high-performance object storage to be affordable to everyone. Thanks to it's smart setup and great storage efficiency Sonic allows us to offer the lowest price for performance-oriented storage without sacrificing content durability and availability. Pricing is based on two tiers with an automatic discount:
➡️ Billed hourly so you never pay for unused space.
➡️ The first 50 000GB of stored data is billed at $0.008 per GB.
➡️ 50 001st and each next GB: $0.005/GB.
➡️ No API, file operations or request fees.
➡️ Egress traffic via our global CDN edge starts from $0.008/GB.
This makes Sonic up to 320% cheaper than AWS S3 per stored gigabyte, and up to 420% cheaper per GB of egress. With up to 20% higher egress fees and charges for API requests even Backblaze's B2 can't match Sonic's pricing at scale.
S3-compatible API and multiple file management tools 🛠🤩
Another drawback of our legacy storage service, SFS, was the lack of compatibility with the now de-facto standard for file operations - the AWS S3 API. Sonic comes with support for it and is a S3-compatible drop-in replacement for AWS S3 👏
In addition to S3 compatibility, there are also FTP, Rsync and Rclone options to manage content stored in Sonic. Our web file manager is also available to allow easy point-and-click file operations from any browser. Switching to Sonic from another storage service is just as easy. For FTP-based storage providers we offer a free storage mirroring functionality on demand to automatically migrate the contents of your current storage service to your new Sonic account. For S3-based storage services we are working to offer an on-the-fly mirroring of content that is served via PUSHR's CDN. This will allow for low-cost seamless migrations that save you costs from the very first day of your Sonic's usage.
Built for projects of any scale, anywhere 🌍
The core of Sonic is built on the concept of an ever-growing cluster of virtually endless storage space that remains efficient and fast regardless of the number of objects stored and their size. Sonic performs great even when orchestrating physical storage servers that are spread across different data centres. From a few small assets that power your website or app, to directories or buckets handling millions of objects, Sonic allows for strikingly cost effective storage and high-speed serving of data ranging from bytes to tens of petabytes.
Join the Sonic preview stage 🚀
Sonic is currently in operation from a single region - Germany 🇩🇪 - and is fully GDPR-compliant with storage nodes in 3 different ISO-27001 certified data centres in the country. We welcome you to join us and help us make Sonic even better with your feedback and feature requests.
Create a free trial account, head over to this quick tutorial to create your first CDN Push zone, or find out more about Sonic's S3 API compatibility and get started in less than 5 minutes! ↗️
Roadmap and upcoming features 📣
In 2023 we will be focusing on the following features to make Sonic even better and to help you manage and deliver your content in easier and even more compliant manner:
➡️ ActivePush integration. While porting our smart pre-caching system to Sonic has proven to be challenging, we continue to work towards having a complete integration as soon as possible to achieve the target performance numbers we are after.
➡️ More storage regions. Sonic does not require multiple regions for content to be delivered at lightning speed globally, but additional regions will allow our customers from different jurisdictions to store sensitive content in compliance with local laws that apply to them. We will be looking to expand the footprint in 2023.
➡️ Immutability, to protect customers' data from malware and ransomware attacks.
➡️ Time-to-live setting for objects, to automate compliance with GDPR provisions for storage duration of data.
🤞
