Storage performance is a complex topic. When evaluating disaggregated NVMe-oFTM storage it is important to understand the system bottlenecks that control performance under varying workloads and deployment scenarios, and to be aware of best practices for measuring performance and evaluating the results.
Historical Measurements of Performance: IOPS, Bandwidth and Latency
The performance of a storage system can be expressed in terms of IOPS, bandwidth and latency.
- IOPS measurements refer to the number of data transfers completed within one second. This metric typically uses small transfer sizes, i.e. 4KB data transfers, and can be used to predict an application's processing power, e.g. database applications processing records.
- Bandwidth performance refers to the throughput typically in Gigabytes per second. This metric normally uses large transfer sizes, i.e. 256KB data transfers, and is used to predict massive data transfers by an application, e.g. streaming video content delivery.
- Latency measurement refers to the time it takes to complete a single command. Latency can be useful to predict an application's delay in a single threaded process, i.e. Write Ahead Logging (WAL) delay. Further latency measurements refer to tail latency, i.e. 95% percentage of I/Os are completed within a certain amount of time. Often times it is instructive to compare latency vs. IOPS, measuring latency while varying the intensity of the workload.
The New Measurements of Performance: IOPS per Dollar and Latency
For a data storage architecture based on the NVMe-oF specification, the combination of high-density NVMeTM SSDs with cost effective flash memory pricing make storage dollars per IOPS and storage read latency the new critical metrics for data center customers. A recent white paper demonstrated a 10X performance advantage of KumoScale over CEPH storage, by showing significant cost savings relating to IOPS performance, as well as superior read access and retrieval. For these tests, the same benchmark and hardware cluster were used for both storage solutions.