The primary interface for KumoScale storage provisioning is the KumoScale Provisioner REST API. Several plugins provide prebuilt integration to the Provisioner API for Kubernetes™ CSI, OpenStack™ Cinder™ and Ansible™ bare metal environments. The Provisioner directly controls the NVMe-oF "data plane" in a storage deployment. The figure below shows how this works in a Kubernetes environment.
Figure 1: Example KumoScale Deployment showing control plane and data plane
Supporting the Provisioner is a KumoScale system "control plane" that is a set of Kubernetes Operators that control overall storage system state (whether 1 node or multiple nodes). These are deployed as a set of Kubernetes containers running in a cluster spread across the available KumoScale storage nodes (whether the storage system consists of 1 node or several nodes). KumoScale deploys several Kubernetes operators, detailed below, that control storage system configuration and operation.
KumoScale Provisioner REST API for Storage Management
The KumoScale Software Provisioner REST API is the primary interface for storage management. The KumoScale Provisioner REST API can be accessed via SSH or terminal simply by entering valid REST API credentials. Details on how to access the Provisioner service REST API are available in KumoScale REST API.
KumoScale Operators for Configuration Management
KumoScale Configuration Operators include
- Install Operator
- Upgrade Operator
- Storage Node Operator
- License Operator
- Master Cluster Operator
KumoScale Operators for Operations Management
Internal KumoScale operations management Operators include
- Telemetry Operator
- Syslog Operator
- KumoScale-internal Operators for Grafana™, Loki™, Fluentd™ and Prometheus that are configured by the KumoScale Master Cluster Operator.
Using KumoScale Operators - kubectl
For security reasons, users are not given direct access to the KumoScale storage cluster and must use a remote host to issue kubectl commands. kubectl is a command line tool for controlling Kubernetes clusters. For general information on kubectl, see https://kubernetes.io/docs/reference/kubectl/overview/. Users can use Secure Shell (SSH) or terminal access to access KumoScale Operators or access the KumoScale Provisioner REST API.
Storage administrators should use kubectl to tell KumoScale operators how to configure nodes, logging, and other components. You will need to use a remote host (based on Windows™, iOS ™, or Linux™ operating systems) that:
- Supports the Kubernetes kubectl client
- Supports a configuration that enables access to the KumoScale storage cluster.
This manual provides all the commands you will need for deployment within KumoScale software. For general information on kubectl, visit the Kubernetes web site. Configuring such a host is part of the installation. See the KumoScale Installation Guide for details on the procedure.
KumoScale Custom Resource Definitions
KumoScale software defines a number of CRDs that tell KumoScale Operators exactly how to manage the storage system. CRDs are implemented as yaml files to support the deployment environment as well as KumoScale components described in the Working with KumoScale Software.
This section summarizes the CRDs you will use and customize to support your implementation.
- Master CRD is used to define the masters on the KumoScale storage cluster. This CRD is configured and deployed during installation.
- Telemetry CRD and Syslog CRD are used to collect data for planning and support purposes and usually set up after the KumoScale Storage Cluster is configured.
- Storage Node CRD is used to configure storage nodes. You may set up different storage node CRD ‘yaml’ files depending on different application needs and environmental settings. Storage nodes can be configured any time after installation.
KumoScale Storage Cluster Master CRDs
There are several CRDs used to configure masters on the KumoScale storage cluster. The only time that you will use the CR files bulleted below is for installing, upgrading, or updating KumoScale software on the master nodes. See the Installation Guide for actions that require using these CRs and their specifications. A high-level summary of these actions follows:
- Master CRD is configured at install time with the number of masters and affinity/anti-affinity parameters. There is only one master CR file per KumoScale storage cluster. A skeleton of the CR file, yaml is provided with KumoScale software.
- License CRD is edited and deployed at install time with the license key provided to you by KIOXIA. You will not be able to provision or allocate volumes without a valid license key.
- Internal Services CRDs are associated with Grafana™, Loki™, Fluentd™, and Prometheus. They are edited at install time with the KumoScale storage cluster VIP and replication information. There is a CR file for Prometheus, Loki, and Fluentd. There is only one CR file for each service and it has the form:
<service name>.kumoscale.kioxia.com_v1_<service name>_cr.yaml
For example,
fluentd.kumoscale.kioxia.com_v1_fluentdservice_cr.yaml.
Telemetry and Syslog CRDs
In addition to the external tools above, there are several CRDs that can be used to collect data for planning and support purposes. These CRDs are defined in Telemetry and Syslog along with instructions and examples for creation.
Storage Node CRDs
Storage node CRD files are used to configure masters and workers on the storage cluster. You can create templates for storage nodes intended to be used by particular applications. Storage node parameters are explained with examples in Storage Node Custom Resources.