The Next Step in SSD Evolution: NVMe Zoned Namespaces Explained
by Billy Tallis on August 6, 2020 12:00 PM EST- Posted in
- SSDs
- Storage
- Western Digital
- NVMe
- SMR
- Radian Memory
Ecosystem Status: Users and Use Cases
The software changes required, both in firmware and OS support/software, will keep zoned SSDs in the datacenter for the foreseeable future. Most of the early interest and adoption will be with the largest cloud computing companies that have the resources to optimize their software stacks top to bottom for zoned storage. But a lot of software work has already been done: software targeting host-managed SMR hard drives or open-channel SSDs can be easily extended to also support zoned SSDs. This includes both applications and filesystem drivers that have been modified to work on devices that do not support in-place modification of data.
Linux Kernel version 5.9 will update the NVMe driver with ZNS support, which plugs in to the existing zoned block device framework. Multiple Linux filesystems either already support running directly on zoned devices, or such support has been developed but not yet merged into a stable kernel release. The device mapper framework already includes a component to emulate a regular block device on top of a zoned device like a ZNS SSD, so unmodified filesystems and applications can be used. Western Digital has released a userspace library to help applications interact directly with zoned devices without using one of the kernel's filesystems on the device.
Only a few applications have publicly released support for ZNS SSDs. The Ceph clustered storage system has a backend that supports zoned storage, including ZNS SSDs. Western Digital has developed a zoned storage backend for the RocksDB key-value database (itself used by Ceph), but the patches are still a work in progress. Samsung has released a cross-platform library for accessing NVMe devices, with support for ZNS SSDs. They've written their own RocksDB backend using this library. As with host-managed SMR hard drives, most production use of ZNS (at least early on) will be behind the scenes in large datacenters. Because ZNS gives the host system a great degree of control over data placement on the SSD, it allows for good isolation of competing tasks. This makes it easier to ensure good storage performance QoS in multi-tenant cloud environments, but the relative lack of zone-aware software means there isn't much demand for such a hosting environment yet.
The most enthusiastic and prolific supporter of ZNS and zoned storage in general has been Western Digital, which stands to benefit from the overlap between ZNS and SMR hard drives. But it is very much a multi-vendor effort. The ZNS standard lists authors from all the other major NAND flash manufacturers (Samsung, Intel, Micron, SK Hynix, Kioxia), controller vendors (Microchip), cloud computing hyperscalers (Microsoft, Alibaba), and other familiar names like Seagate, Oracle and NetApp.
Longtime zoned SSD provider Radian Memory recently published a case study conducted with IBM Research. They ported an existing software-based log-structured storage system to run on Radian's non-standard zoned SSDs, and measured significant improvements to throughput, QoS and write amplification compared to running on a block storage SSD.
Most SSD vendors have not yet announced production models supporting ZNS (Radian Memory being the exception), so it's hard to tell what market segments, capacities and form factors will be most common among ZNS SSDs. The most compelling opportunity is probably for ZNS-only QLC based drives with reduced DRAM and overprovisioning, but the earliest models to market will probably be more conventional hardware configurations with updated firmware supporting ZNS.
Overall, ZNS is one of the next steps in mirroring the use of SSDs in the way SSDs are actually designed, rather than an add-on to hard drive methodology. It is a promising new feature. It looks likely to see more widespread adoption than previous efforts like open-channel SSDs, and the cost and capacity advantages should be more significant than what SMR hard drives have offered relative to CMR hard drives.
45 Comments
View All Comments
Luminar - Thursday, August 6, 2020 - link
So if this is software, what's stopping this vendors from making this a retroactive firmware update?BinaryTB - Thursday, August 6, 2020 - link
$$$$$Luminar - Thursday, August 6, 2020 - link
If only drive controllers were jailbreakable.althaz - Friday, August 7, 2020 - link
I mean, they technically are. But it's not really worth the expense.close - Friday, August 7, 2020 - link
Storage is the kind of segment where I'd 100% take reliability over performance. Slow storage is still storage, unreliable storage is a data shredder.JlHADJOE - Friday, August 7, 2020 - link
ヽ༼ຈل͜ຈ༽ノSteven Wells - Saturday, August 8, 2020 - link
Great point @close. So many concepts of host managed storage all come down to “but Mr Drive Vendor this is fully reliable, right? Even if I totally screw up the host side management?”close - Sunday, August 9, 2020 - link
Well nothing is 100% reliable and you should always plan on the possibility of it failing. But for example "jailbreaking" the controller to use some form of custom firmware that would increase performance almost certainly at the cost of reliability is really like shooting all of your storage at its feet to make it run faster. Sooner or later you'll hit the foot.Go for something that's validated and proven if your data matters. There's no guarantee, just a better likelihood of not loosing it.
dotjaz - Friday, August 7, 2020 - link
If only jailbreak QA is better than OEM, oh wait, there is none.Billy Tallis - Thursday, August 6, 2020 - link
I think Radian might actually be doing that. In general though, adding a feature of this size and complexity in a firmware update is something your customers will want to put through a full QA/qualification cycle before deploying. And by the time a customer is done updating their software stack to use ZNS, it's probably time to buy (or start qualifying) new SSDs anyways.