This section describes the KumoScale software architecture and how its components work to provision storage in various environments. It outlines the terminology used throughout the document and defines components you will use for deployment.
KumoScale software is a clustered, scale-out NVMe-oF™ protocol storage system designed for the modern data center. KumoScale software is available in one of two modes both of which host services that provision and manage storage allocation within a data center orchestration framework. This includes the KumoScale Provisioner Service, interfaces, and mapping and monitoring tools for allocating, analyzing, and managing storage.
Appliance Mode software is installed and configured on the KumoScale Storage Cluster, a Kubernetes cluster.
Managed Mode software is installed and configured on your own OS and kernel. There are two ways to use Managed mode:
- As installed on the OS and kernel. In this case, provisioning and storage management is done through the REST API or CLI.
- With your own Kubernetes cluster. This is any healthy, high-availability cluster that is installed and configured per best practices and guidelines of the selected Kubernetes distribution.
This document will discuss the procedures for Appliance mode and Managed mode with a Kubernetes Cluster.
- KumoScale Storage Nodes, are storage appliances equipped with NVMe™ Solid State Drives (SSDs) and configured as worker nodes that manage the virtual volumes.
- KumoScale Control Operators, are used to create and support configuration of Custom Resources used in the storage environment. The control operators are installed as part of the storage cluster when using appliance mode, and are installed on top of your Kubernetes storage cluster when using managed mode.
- KumoScale Interfaces, are installed on the storage cluster to support integration into your current orchestration environment.
The main interface is a RESTful Application Programming Interface (API) designed for proprietary orchestration environments. To speed up automation, KumoScale software also supports:
- KumoScale Command Line Interface (CLI) for storage control, monitoring, and user management. The KumoScale CLI Guides for Provisioner and Storage Nodes include complete information and notes differences between appliance and managed mode.
- Kubernetes CSI Driver, an interface for Kubernetes containerized environments. The KumoScale CSI Driver User Manual provides details on how to use this interface.
- Ansible (Bare-Metal) Module, for provisioning storage to support a cross-domain resiliency solution. The KumoScale Ansible User Manual provides details on how to use this interface.
- OpenStack™ Cinder Driver, which integrates with an OpenStack platform via the Cinder driver plug-in and the Nova connector. Cinder is a block storage service for OpenStack platforms, and designed to present storage resources to end users that can be consumed by the OpenStack Nova Compute Project. The KumoScale Manual for OpenStack provides details on how to use this interface.
When KumoScale is installed in appliance mode, these components work in a data center as shown in the high-level architecture below.
KumoScale Software High-Level Architecture for Appliance Mode
In both Appliance and Managed modes with a Kubernetes cluster, the data center infrastructure environment requests volumes from KumoScale software according to the requirements of hosted applications. Where needed, these requests are implemented using the KumoScale interface for the infrastructure.
The KumoScale Provisioner service is a service on the storage cluster that creates virtual clusters for different tenants and allocates volumes according to applications requirements and the storage nodes current status and utilization. It then creates the targets on KumoScale storage nodes for connecting to the application host, adds the volumes as namespaces, and maintains the connection status for application hosts.
KumoScale storage nodes maintain the persistence of the configuration and manage the volume virtualization. The nodes periodically send telemetry on the physical SSDs and virtual volumes.
KumoScale Time Series Databases (TSDBs) and Syslog servers collect the telemetry data for analysis. KumoScale software analyzes the data to produce recommendations on managing the environment.
Compute nodes within application hosts are nodes in clustered application servers that connect to their allocated volumes via the nvme-connect command. Telemetry and Syslog events are collected from them as well. An agent monitors the state of the replication layer and generates events to the system log.
KumoScale Software Provisioning
The KumoScale Provisioner service is a distributed, stateless, resilient service that accepts requests for volumes, along with a specification detailing the volume’s requirements known as the storage class. The KumoScale Provisioner service returns the logical identity and network location where the requested volume can be accessed.
Provisioning takes into account a variety of factors to arrive at an optimal placement decision, such as resilience and topology requirements, capacity, and node utilization. In addition, the Provisioner Service will take into account the desired Quality of Service (QoS) parameters. KumoScale software interfaces support additional functions for managing provisioning in specific orchestration environments.
Solid State Disk (SSD) Groups
Every SSD deployed within a KumoScale storage node is automatically sorted into a group. The SSD group serves as a pool of storage. You can expand a group by adding new SSDs to it. The new capacity becomes available for mapping new volumes. See Adding an SSD to a Group for additional information.
KumoScale software maps virtual volumes to physical drives. In order to optimize the memory utilization within KumoScale software and to enable the easiest and most intuitive integration with orchestration frameworks, KumoScale software automatically places the volumes within the storage node, taking into account the storage class defined by the user.
Targets and Access Control List (ACL)
Once volumes are created, they are exposed over an Ethernet fabric. Application hosts and compute nodes establish an NVMe-oF protocol connection to the volumes using a logical entity called a target.
Connectivity between application hosts and targets is controlled in the Access Control List (ACL). When a volume is created, it is added to a target with the corresponding ACL as a namespace, thus enabling the host to connect to it. The ACL is set per the host and is associated with the initiator/target pair. The permitted connection types are ‘Read Only’ and ‘Read/Write’ access.
An additional access control feature is a discovery ON/OFF setting. When discovery is set to OFF, the NVMe-oF protocol resource cannot be discovered over the fabric using the NVMe Discover command. The only way to connect to it is to know ahead of time the unique worldwide NVMe Qualified Name (NQN) of that resource. This prevents the discovery of storage resources by application hosts, even when they are connected to a data center trusted network.
The relationships between KumoScale storage provisioning entities now follows:
Physical Volume Allocation vs. Virtual Volumes
The SSD Group image on the left depicts the physical allocation of volumes for the various targets on a group of two SSDs, whereas the target images on the right illustrates the volumes from an application hosts point of view. The arrows simulate the hosts/targets ACLs.
The target creation and ACL settings are managed by the KumoScale Provisioner service, derived from connecting host parameters, and do not require a user configuration.