Volume

The Listener Operator acts as a CSI PersistentVolume, which helps it to stabilize network addresses, inject pod metadata and expose individual Pods.

The listener volume represents a Pod being exposed by a Listener. The backing Listener can either be created automatically by the Volume, or manually by the operator.

Stable addresses

Some ListenerClass strategies, such as NodePort, tie the public address to the Kubernetes node that the Pod is running on. When this address must be configured statically in clients (such as for HDFS NameNodes), then Kubernetes' default "floating" scheduling either requires all clients to be reconfigured every time something moves, or for all clients to proxy their traffic through a single static node, which then becomes a single point of failure (along with the node that the workload is running on).

Mounting listeners into Pods as PersistentVolume allows the Listener Operator to pin these workloads to one node. Note that this only happens for ListenerClasses that actually benefit from pinning.

Downwards API

Some services (such as Kafka) need to know their external address, so that they can advertize it to their own replica discovery mechanism. Listener volumes contain a file tree that exposes this information:

  • default-address/- A symlink to addresses/{primary address}

  • addresses/ - Contains information about all addresses associated with this Listener

    • {address}/ - A folder is created for each address

      • address - Contains the Pod’s address (IP address or hostname)

      • ports/ - Contains port numbers for each named port

        • {port name} - Contains the public port number for this named port

Individual pod exposure

Sometimes each replica must be exposed individually, for example because clients need to access data on a specific shard. PersistentVolumeClaim templates can be used to provision this automatically.

Listeners created by volumes share their volume’s lifetime; if the volume is deleted then so is the Listener.

StatefulSet volumeClaimTemplates

The volumeClaimTemplates allow volumes to be provisioned for each StatefulSet replica. These volumes are persistent, and will not be deleted when the Pod or StatefulSet is. This makes them useful for provisioning addresses that must be hard-coded into client configuration.

Pod-scoped ephemeral volumes

Pod.spec.volumes[].ephemeral allows volumes to be provisioned for each Pod. These volumes are tied to the lifetime of the Pod and will be deleted along with it. This makes them useful for provisioning temporary addresses that will be discovered out of band (such as for HDFS DataNodes).

Shared exposure

Multiple replicas can reference the same Listener, by creating the Listener manually, and then setting the listeners.stackable.tech/listener-name annotation on the volume.

In this case, the injected Pod metadata may still be specific to a particular Pod. For example, when binding NodePort Listeners, the Pod will only contain metadata about the Node that it is actually running on.

Pinning (if applicable) is managed on the volume scope, each replica binding a single Listener can be pinned to a different Node.

Reference

All configuration must be specified as annotations on the PersistentVolumeClaim. The following attributes are currently supported:

listeners.stackable.tech/listener-name

Required: If listeners.stackable.tech/listener-class is not specified

Provisions metadata about an existing Listener that was created manually.

listeners.stackable.tech/listener-class

Required: If listeners.stackable.tech/listener-name is not specified

Provisions a new Listener using the specified ListenerClass. The created Listener will expose all of the Pod’s ports.