Storage is getting Faster, Smarter, and Cheaper

By Andrew Boggs, Data Center Design Architect

What used to be the dullest technology in IT, hard disk storage and tape, is now innovating right and left. It started when we moved from 3.5-inch diskettes to USB Sticks and Chip Readers. It has evolved all the way to Storage Class Memory which is blurring the old lines between RAM Operations and Storage Operations in a new Tier 0. This evolution includes NVMe, a new Flash storage interface that does not abuse the memory architecture like a spinning disk with an unlimited re-write capacity. Another development is multi-site Storage-as-a-Service, which disrupts the financial model of storage acquisition and provides better options for legacy infrastructure that cannot be migrated to HCI or Cloud Native stacks. Finally, ransomware mitigation in primary and secondary storage is starting to protect these advances from sophisticated, profit-driven criminal hackers that are putting manufacturers, hospital systems, local governments and most other organizations at risk. This Storage Trends series looks to highlight these developments and take a deeper look into current and future storage trends.

Storage Class Memory (SCM)
Storage Class Memory evolved from persistent memory technology and provides a spacious intermediate cache between RAM and storage. This approach can protect data that has just been processed from a power failure but requires further processing before it is written to a “slower” media like NVMe or SSD. The most ubiquitous example is Intel Optane and it is being leveraged by many storage array providers. High-Performance Computing and High-End General-Purpose Servers are embedding more SCM. Hybrid devices with embedded SCM are starting to emerge, so watch for options in motherboards, processors, and IoT devices.

Non-Volatile Memory express (NVMe)
NVMe is another variation that evolved from persistent memory. The express part of that label refers to the PCIe bus; which provides CPUs, GPUs, and motherboard chip sets parallel access to Flash page storage using Direct Memory Access. The serial SAS or SATA buses, still used by some SSDs, were designed for magnetic and optical media (HDD, CD, DVD) with slow mechanical operations that could not be optimized to reduce latency. The flash storage media in NVMe devices and SAS or SATA SSDs is not radically different, but the NVMe interface performance, buffer structures and support for massively parallel access can reduce latency and dramatically increase throughput for some workflows.

While NVMe is an excellent enhancement for Direct Attached Storage on Servers or HCI Nodes, it can also be extended over Layer 2 networks using Remote Direct Memory Access. This allows vendors to form Petabyte and Exabyte scale storage arrays with disruptive performance characteristics using NVMe to communicate between nodes. NVMe over Layer 2 Fabrics is supported by an ever-expanding group of protocols including Fibre Channel, InfiniBand, Ethernet, iSCSI Extensions and more. This means NVMe not only enhances communications within a storage array, it is also starting to enhance array support for application workstation and server performance across hundreds of endpoints. Endpoint support for NVMe over Fabrics is widespread with Windows SMB Direct or Guest RDMA and Linux InfiniBand, RoCE, or NVMe/TCP. Look out for a future blog covering a High-Performance Computing system vendor that has expanded the PCIe bus extension concept to form a Composable Infrastructure that includes CPU/RAM/SCM, GPU, FPGA, NVMe, and more.

Storage as a Service (StaaS)
Storage as a Service is one of the biggest changes affecting storage investment over the last several years. There is a consistent movement away from large CapEx bets that hope to cover 3 to 5 years between forklift storage upgrades to OpEx buys that continuously cover variable storage requirements. The first iteration of the OpEx approach was Cloud Storage. For example, AWS provides different performance tiers or rates for both Block and File. These operations can be backed by low performance Object Archive storage with very attractive pennies-per-GB pricing. There are two major issues with this approach related to where the compute is attached to the storage and the total cost of ownership. Cloud storage usually means cloud computing, whether the applications or operations teams are ready to move to the cloud or not, which is an inherently risky scenario.

Compounding the hidden costs of a rushed compute migration, the pennies-per-GB pricing can be very deceptive unless the business analysis has carried out the recurring cost for the required capacity across the duration of the entire operation. Large capacities of cloud storage over 3 to 5 years can turn out to be 3 or 4 times more expensive than the equivalent CapEx investment. The true cost of a storage placement also requires a careful analysis of any egress fees for the subsequent movement of data around the cloud or away from the cloud.

Many “legacy” storage vendors are beginning to address these issues with Storage-as-a-Service offerings that provide both subscription pricing for long-term capacity and on-demand pricing for variable capacity. Many offer implementations for On-Premises, in CoLo facilities, and with Public Cloud Providers such as AWS, Azure and Google. The StaaS provider typically gets a special rate and will front the cloud storage with the same features and operating environment that is used On-Premises. The best part about this new approach is that if you make a bad guess about the capacity or performance of the tiers you are using, you can correct the situation. Corrections are immediate in the Cloud and may require some lead time On-Premises or in CoLos. The bottom-line benefits because you only pay for what you are using at the on-demand rate.

Ransomware
Faster, smarter, and cheaper storage may be on everyone’s list, but none of it matters if you suddenly lose access to all your data in a ransomware attack. This is a dangerous situation, especially when paying the ransom means you’ll likely become a “repeat customer” of attackers. Unless you uncover how they got in and what assets they compromised, it could easily happen again. Several storage vendors are introducing support for Immutable Copy features or are partnering with providers like Veeam or Rubrik to ensure clean backups are available for rapid restoration. Preventing the problem in the first place reduces the requirement for rapid restoration, although immutable copies of key data will likely remain a part of many compliance regimens. Watch for a future blog covering indicators of compromise and encrypted storage approaches using Enterprise Key Management systems to prevent encryption with unmanaged keys.

Partner Example – Pure Storage

Red River strategic partner Pure Storage has been innovating with all these storage trends for some time. They offer support for

• Metadata-aware operations enhanced with customer crowd-sourced Artificial Intelligence
• Massively parallel, very low-latency workflows via NVMe over Fabrics
• Pure-as-a-Service offerings for On-Premises, CoLo, and a growing list of Public Cloud Providers.
• Ransomware mitigation with partners such as Veeam and Rubrik

The FlashArray//X70 and //X90 series leverage Intel Optane SCM with Pure DirectMemory Module cache software to cut storage operations latency by 25 to 50% and reduce CPU utilization by up to 25%. This performance is ideal for Online Transaction Processing and Real-time Analytics. SAP HANA, which normally operates in host memory, can leverage these DirectMemory Modules to get 90% of host memory performance with a 65% cost savings.

At their recent Pure//Accelerate Digital 2020 online event in mid-June, Pure Storage announced disruptive VMware Cloud Foundation and vSphere 7.0 integrations. VMware vSAN is still maintained as the core storage service for the initial build and Management Domain, but FlashArray//X or FlashStack with Cisco UCS can be integrated as flexible scalable storage for Workload Domains using FC, iSCSI, and most significantly, NVMe over Fabrics.

You can follow this link to register for a webinar on this topic Wednesday, July 22.

NVMe over Fabrics has been a feature for ESXi Host kernel-mode operations on HCI nodes for some time. VMware Cloud Foundation 4.0 and vSphere 7.0 now support NVMe over Fabrics for user-mode VMs and Containers, greatly improving performance for both single-thread and highly parallel workflows. This innovation opens the VMware Cloud Foundation to a flexible Hybrid-HCI model On-Premises, in CoLos, and with Public Cloud Providers starting with AWS. The details of Azure support should emerge shortly after the Azure Beta of Cloud Block Store wraps up. VMware Cloud on Azure has already been announced.

Pure Storage announced extensive VMware Cloud Foundation feature support for

I recommend watching Pure Storage as a consistent storage innovator in hardware, software, and the total cost of ownership with their Evergreen Program. Their FlashArray//C QLC-Flash series for capacity-optimized storage adds an affordable option for less demanding workflows. Pure FlashBlade use cases for high performance NFS, SMB, and S3 Object storage are multiplying every day.

I hope you’ve enjoyed this Storage Trends installment. If you have a question regarding anything you read or about any other advancements in storage technology, please feel free to reach out. Look out for the next installment, “Intelligent Data Placement and Migration With Metadata Analytics”.

ACRONYMS