Configuring Storage Nodes

Once you have installed storage node software and configured the KumoScale Provisioner, you are ready to set up your storage nodes. If you are using

Adding Storage Nodes to a Kubernetes Cluster

You are now ready to add storage nodes to your environment using custom resource files. This section provides only the basic steps to adding storage nodes. The KumoScale User Manual provides complete documentation on parameters to use when adding and modifying storage nodes.

An example storage node Custom Resource (CR) file, kumoscale_v1_storagenode_cr.yaml is available in the example directory included with KumoScale software. To configure and deploy a new storage node, follow these steps:

  1. Make a copy of kioxia.com_v1_storagenode_cr.yaml for editing, and save to a separate directory (e.g., deploy/crds/myapp_storagenode_cr.yaml).
  2. Update myapp_yaml with values for the parameters that apply to your deployment, such as name and IP address. See the KumoScale User Manual for a complete list.
  3. Create the node with:
kubectl create -f myapp_storagenode_cr.yaml

Initializing Storage Nodes with KumoScale Provisioner on Bare Metal

After the KumoScale installation completes successfully, you need to configure your storage nodes for KumoScale software by completing the following steps on each node:

  1. Initialize KumoScale software with a unique name and administrative password.
  2. Select and configure an SSD group that will be used to host all volumes.
  3. Create portals to access KumoScale software via NVMe-oF by setting a persistent IP address on the platform and creating a portal with this IP address.

This section explains how to complete these steps using the CLI. For more details on how to use the KumoScale CLI, see the Storage Node CLI Reference Manual. If you prefer to use the REST API, see the Storage Node REST API Reference Manual for details on corresponding commands.

For each storage node complete the following:

  1. Start the CLI by issuing the following command:
ssh root@<node ip address> ks_cli -p 443
2.  Use the KumoScale CLI command quick-init to initialize the storage node with a new name and administrative password. At the CLI> prompt, issue the command below where the parameters with values in brackets are defined in the table.
quick-init --user-name <USER_NAME> --old-password <OLD_PASSWORD> --new-password <NEW_PASSWORD> --system-name <SYSTEM_NAME>

quick-init
Parameter Name

Description

user-name

The RBAC administrator user name. The default is admin.

old-password

The old password value of the RBAC administrator user. The default is admin.

new-password

The new password value of the RBAC administrator user.

system-name

The name of the system.

3.  Generate a token in order to execute remaining commands:
CLI> generate-token --name admin --password <pass>

A token will be returned for use in subsequent commands:

Token:        **********************……
4.  Identify the names of the available of the SSDs by using show-ssds to get a list of SSDs.

 

Note: System disk SSDs will NOT be listed and should NOT be selected since the operation will clear their content.

At the CLI> prompt, issue the following command:

show-ssds --detail

The system will return a list of SSDs similar to the example below.


Name:                         SSD4
Alias:                        <alias>
Persistent Id:                nvme0
SSD Product info:
  Model:                    <model>
  Product:                  <product>
  Vendor:                   <vendor>
  Raw Capacity:             <raw capacity>
  Capacity:                 <capacity>
SSD location related info:
    Path:                     /dev/nvme0
    PCIe Slot:                -1
    PCIe Shelf:               -1
SSD version related info:
    Driver Name:              nvme
    Driver Version:           3.20-2229
    Firmware Version:         1.0
    Bios Version:             0.0.0
SSD Allocated Capacity related info:
    Free Space:               17112760320 bytes; 15.938 GiB
    NS use:                   17179869184 bytes; 16.000 GiB
SSD Configuration related Info:
    Group Name:               group
    Volumes Count:            0
    Allocation Enabled:       Yes
    Max Namespaces:           256
    NS Management Supported:  Yes
    Single NS Delete Supported: Yes
    FW Upgrade Supported:     No
SSD State related Info:
    Life Left:                100 %
    Temperature:              N/A Celsius
    Locate:                   Yes
State:                        AVAILABLE
Severity:                     INFO

Note the Name of the SSDs you want to use to host volumes (for example, SSD4).

5.  Create a group using the SSDs you want to host volumes by issuing the KumoScale CLI command create-group where the parameters with values in brackets are defined in the table below:
create-group --name <NAME> --ssd <SSD>

create-group
Parameter Name

Description

name

The SSD group name.

ssd

A list of SSD names, separated by a comma.

6.  A portal is a single or aggregated network port with its own static IP address. You will need to identify the available interfaces by using the command show-interfaces:
show-interfaces --detail

This will list the available network interfaces. Below is an example of one returned value:

Interface ID:         361
Interface Name:       ens2
Vendor:               VMware
Product:              VMXNET3 Ethernet Controller
Maximum speed:        10000.0 Mb/s
Speed:                1000.0 Mb/s
MTU:                  1500
MAC Address:          ##:##:##:##:##:##
Link Device State:    AVAILABLE
Admin state:          UP
Device State:         AVAILABLE
Interface Addresses:
    Protocol:         INET
    Type:             ManagementData
    Ip Address:       192.###.###.7
    Subnet:           255.255.255.0
Interface Addresses:
    Protocol:         INET6
    Type:             Management
    Ip Address:       fe80::5054:##:####:####
    Subnet:           ffff:ffff:ffff:ffff::

Note the Interface Name and Ip Address of the interface(s) you would like to use for the NVMe-oF connection(s). You will use these in the next step.

7.  Create a portal with the KumoScale CLI command create-portal. The create-portal command will configure a new NVMe Portal with a specific IP address on a data Interface. At the CLI> prompt, issue the command below where the parameters with values in brackets are defined in the table.
create-portal --name <NAME> --ip <IP> --port <PORT> --transport {TCP,RoCEv2}

create-portal
Parameter Name

Description

name

The name of the portal.

ip

The IP address of the portal.

port

The port number of the portal.

transport

The selected NVMe-oF transport type, equal to either TCP or RoCEv2.

8.     For each storage node that needs to be managed by the Provisioner, add it to the Provisioner with the Provisioner CLI command add-backend, where backend refers to a storage node. At the CLI> prompt, issue the command below where the parameters with values in brackets are defined in the table.

add-backend --mgmt-ips <MGMT_IPS> [--mgmt-port <MGMT_PORT>] [--rack <RACK>] [--region <REGION>] [--zone <ZONE>] [--host-id <HOST_ID>]

add-backend
Parameter Name

Description

mgmt-ips

A list of management IPs or DNS names. This is a required value.

mgmt-port

The port number of the backend (storage node). (optional)

rack

The storage node location indicating the rack ID. (optional)

region

The accessible region ID. (optional)

zone

The accessible zone ID. (optional)

Host-id

The host ID. (optional)

Exit the CLI with the exit command.

 

Next: Installing NVMe Host Module Patch on Compute Nodes