An "operator" is a design pattern for implementing a control system in software. The operator approach was originally popularized by the Kubernetes® container orchestration system. Since the KumoScale™ control plane is built on a Kubernetes platform, it is natural that the key control constructs are expressed in a Kubernetes native way.
To understand the elegance of the operator pattern, it's necessary to first emphasize the distinction between "imperative" and "declarative" and styles of expressing a user's intent. Imperative programming consists of a list of commands, each of which directly change the state of the system, and the sequence of these commands is the direct embodiment of the algorithm. An everyday example of actions specified in an imperative way is driving directions. "Drive forward one kilometer, then turn left" is an imperative statement, and a list of such statements executed in order will progress the system state (position) from the start to the desired destination.
In this context, imperative expression can be seen to have two weaknesses: First, the directions are valid only for one starting point. Even for the same destination, a set of directions are useless if the current position is either unknown or different from the one they were written for. Secondly, this method assumes that the user of the system knows the best route (algorithm) to accomplish the goal, and is aware of any transient circumstances (closed roads, traffic) that might change that. The system executes imperative statements faithfully, whether they are in fact advisable or not.
By contrast, a declarative system is analogous to GPS navigation: With a GPS system, the user supplies only the destination, and the control system itself contains the ability to determine current location, and the logic to find a safe and efficient path to to that destination. The user need not know the current location (or system state), nor the best sequence of steps to reach the destination. This makes the statement of the desired final state both independent of the current state, and resilient to unexpected changes in the environment that may occur along the way.
An operator, in a Kubernetes and in KumoScale environment, is a software construct which implements a declarative control system. An operator consists of two key parts:
A CRD is simply a template for an entry in a database used to keep track of system state. A "resource" can be an atomic object (e.g. a Kubernetes pod, or a KumoScale SSD) or a composite object (a Kubernetes deployment containing many pods, a KumoScale storage node containing many SSDs). Each entry is called a "custom resource", and represents one such object.
The salient aspect of a CRD is that for each controllable aspect of the resource, it contains two descriptors. One, the "current state," is kept up to date by the system, and as its name implies, represents the current state of that variable for that object. "Desired state" is writable by the user, and is used to express the user's intent. "Current state" and "desired state" correspond to current location and destination in the GPS example.
The controller portion of an object is a software routine which embeds the business logic associated with changing the state of the system. In the GPS example, the controller contains the street map, plus a navigation algorithm. Whenever the current state of a CR does not match the desired state, the corresponding controller is triggered, and takes the actions necessary to bring them into alignment. So, the process of changing the configuration of KumoScale is largely accomplished by editing the "desired state" portion of various system objects.
The following table lists some of the KumoScale custom resources and their constituent configuration variables, which are supervised by controllers. This list is not exhaustive, and will grow over time as additional configuration resources are brought under operator control.
| CRD | VARIABLE(s) | FUNCTION |
| Management Cluster | ||
| Master CRD | ||
| numberOfMasters | Number of master nodes in the cluster (should be an odd number) | |
| affinity/antiAffinity | Controls which storage nodes can become masters based on topology constraints. | |
| Virtual IP | Service IP at which the management cluster can be reached | |
| License CRD | ||
| License key | ||
| External Services CRD | ||
| Prometheus® | ||
| LogStash | ||
| Monitoring | ||
| Telemetry CRD | ||
| name | Name for the telemetry configuration | |
| telemetryTSDBType | Which specific time series database is in use (e.g. Graphite®, Prometheus) | |
| transportType, ip, port | Transport protocol (TCP or UDP), database IP address & port number | |
| pushState, intervalMin | Whether the telemetry framework should spontaneously "push" data at some rate, or the telemetry server will poll for ("pull") the data. And, time interval between telemetry push events (N/A for pull-mode systems.) | |
| Syslog CRD | ||
| url, ip, port | ||
| tls, certFileName | ||
| Storage | ||
| Storage Node CRD | ||
| name | ||
| initMgmtIp | Initial management IP | |
| adminSecretName | ||
| timeSettings | Time zone, ntpServer | |
| portals | Name, IP:port, and transport time for one or more logical network interfaces | |
| team, vlan | Network portal teaming & vlan configuration | |
| topology | Location label strings (rack, zone, region) that enable specification of topology-based constraints on the placement of volumes. | |
