This section describes the KumoScale software architecture and how its components work to provision storage in various environments. It outlines the terminology used throughout the document and defines components you will use for deployment.
KumoScale software is a clustered, scale-out NVMe™ protocol storage system designed for the modern data center. KumoScale software is installed and configured on the KumoScale Storage Cluster, a Kubernetes cluster with hosted services that provision and manage storage allocation within a data center orchestration framework. This includes the KumoScale Provisioner Service, interfaces, and mapping and monitoring tools for allocating, analyzing, and managing storage.
- KumoScale Storage Nodes, are storage appliances equipped with NVMe Solid State Drives (SSDs) and configured as worker nodes that manage the virtual volumes.
- KumoScale Control Operators, are installed as part of the storage cluster and used to create and support configuration of Custom Resources used in the storage environment.
- KumoScale Interfaces, are installed on the storage cluster to support integration into your current orchestration environment.
The main interface is a RESTful Application Programming Interface (API) designed for proprietary orchestration environments. To speed up automation, KumoScale software also supports the:
- KumoScale Command Line Interface (CLI) for storage control, monitoring, and user management. This manual includes examples using this tool. The KumoScale CLI Guide includes complete information.
- Kubernetes CSI Driver is an interface for Kubernetes containerized environments. The KumoScale CSI Driver User Manual provides details on how to use this interface.
- Ansible (Bare-Metal) Module is used for provisioning storage to support a cross-domain resiliency solution. The KumoScale Ansible User Manual provides details on how to use this interface.
- OpenStack™ Cinder Driver, which integrates with an OpenStack platform via the Cinder driver plug-in and the Nova connector. Cinder is a block storage service for OpenStack platforms, and designed to present storage resources to end users that can be consumed by the OpenStack Nova Compute Project. The KumoScale Manual for OpenStack provides details on how to use this interface.
- KumoScale Manager is also available for development testing and monitoring purposes.
These components work in a data center as shown in the high-level architecture below (Figure 1).
Figure 1. KumoScale Software High-Level Architecture
The data center infrastructure environment requests volumes from KumoScale software according to the requirements of hosted applications. Where needed, these requests are implemented using the KumoScale interface for the infrastructure.
The KumoScale Provisioner service is a service on the storage cluster that creates virtual clusters for different tenants and allocates volumes according to applications requirements and the storage nodes current status and utilization. It then creates the targets on KumoScale storage nodes for connecting to the application host, adds the volumes as namespaces, and maintains the connection status for application hosts.
KumoScale storage nodes maintain the persistence of the configuration and manage the volume virtualization. The nodes periodically send telemetry on the physical SSDs and virtual volumes.
KumoScale Time Series Databases (TSDBs) and Syslog servers collect the telemetry data for analysis. KumoScale software analyzes the data to produce recommendations on managing the environment.
Compute nodes within application hosts are nodes in clustered application servers that connect to their allocated volumes via NVMe connect. Telemetry and Syslog events are collected from them as well. An agent monitors the state of the replication layer and generates events to the system log.
2.2 KumoScale Software Provisioning
The KumoScale Provisioner service is a distributed, stateless, resilient service that accepts requests for volumes, along with a specification detailing the volume’s requirements known as the storage class. The KumoScale Provisioner service returns the logical identity and network location where the requested volume can be accessed.
Provisioning takes into account a variety of factors to arrive at an optimal placement decision, such as resilience and topology requirements, capacity, and node utilization. In addition, the Provisioner Service will take into account the desired Quality of Service (QoS) parameters. KumoScale software interfaces support additional functions for managing provisioning in specific orchestration environments.
2.2.1 Solid State Disk (SSD) Groups
The SSDs deployed within a KumoScale storage node are automatically sorted into groups. These SSD groups serve as a pool of storage. You can expand a group by adding new SSDs to it. The new capacity becomes available for mapping new volumes. See Adding and Removing an SSD to and from a Group for additional information.
2.3 Virtual Volumes
KumoScale software maps virtual volumes to physical drives. In order to optimize the memory utilization within KumoScale software and to enable the easiest and most intuitive integration with orchestration frameworks, KumoScale software automatically places the volumes within the storage node, taking into account the storage class defined by the user.
2.3.1 Targets and Access Control List (ACL)
Once volumes are created, they are exposed over an Ethernet fabric. Application hosts and compute nodes establish an NVMe-oF™ protocol connection to the volumes using a logical entity called a target.
Connectivity between application hosts and targets is controlled in the Access Control List (ACL). When a volume is created, it is added to a target with the corresponding ACL as a namespace, thus enabling the host to connect to it. The ACL is set per the host and is associated with the initiator/target pair. The permitted connection types are ‘Read Only’ and ‘Read/Write’ access.
An additional access control feature is a discovery ON/OFF setting. When discovery is set to OFF, the NVMe protocol resource cannot be discovered over the fabric using the NVMe Discover command. The only way to connect to it is to know ahead of time the unique worldwide NVMe Qualified Name (NQN) of that resource. This prevents the discovery of storage resources by application hosts, even when they are connected to a data center trusted network.
The relationships between KumoScale storage provisioning entities now follows (Figure 2):
Figure 2. Physical Volume Allocation vs. Virtual Volumes
The SSD Group image on the left depicts the physical allocation of volumes for the various targets on a group of two SSDs, whereas the target images on the right illustrates the volumes from an application hosts point of view. The arrows simulate the hosts/targets ACLs.
The target creation and ACL settings are managed by the KumoScale Provisioner service, derived from connecting host parameters, and do not require a user configuration.