Skip to content

Storage

A Stroom node needs two kinds of storage:

  • Local storage, private to the node: logs, LMDB reference data, search results, temporary files and so on. The operator gives each node its own PersistentVolumeClaim for this.
  • Shared storage for Stroom's data volumes, which hold the stream data, mounted into every node. You add this with extra volumes.

Local storage

localDataVolumeClaim is a standard PersistentVolumeClaim spec. The StatefulSet creates one claim per node, named data-stroom-<cluster>-node-<node set>-<ordinal>:

    - name: data
      localDataVolumeClaim:
        storageClassName: fast-ssd
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 50Gi

Use fast, low-latency storage: LMDB performance depends on it. If you pre-provision PersistentVolumes instead of using a StorageClass, create one per node and use selector or volumeName to bind them.

The claim is mounted at these paths in the Stroom container:

Path in container Sub-directory of the volume Contents
/stroom/logs logs Stroom application, access, user (audit) and migration logs, and the operator's lifecycle script logs in /stroom/logs/k8s
/stroom/output output Output written by pipelines to the local file system
/stroom/tmp tmp Stroom's temporary directory (STROOM_TEMP)
/stroom/lmdb lmdb LMDB stores
/stroom/lmdb_library lmdb_library The LMDB native library
/stroom/planb planb Plan B state stores
/stroom/proxy_repo proxy-repo Proxy repository
/stroom/reference_data reference-data Reference data (LMDB)
/stroom/reference_staging_data reference_staging_data Reference data staging
/stroom/search_results search-results Search result stores

When sizing the claim, allow for logs (the bundled configuration keeps up to a week of minute-rolled application logs), reference data and search results.

Paths that are not persisted

The Stroom 7.13 image declares other directories as volumes, including /stroom/volumes, /stroom/content_index, /stroom/git_repo and /stroom/data_feed_identities. The operator doesn't mount anything there, and Kubernetes doesn't create volumes for image VOLUME declarations, so anything Stroom writes there is lost when the Pod is recreated. If your Stroom configuration uses any of these paths, mount persistent storage there with extraVolumes and extraVolumeMounts.

Resizing local storage

Kubernetes doesn't allow a StatefulSet's volume claim template to change, so changing localDataVolumeClaim on an existing node set fails (see Applying changes). To grow existing volumes, expand each claim directly, if your StorageClass allows volume expansion:

kubectl -n stroom patch pvc data-stroom-dev-node-data-0 \
  -p '{"spec": {"resources": {"requests": {"storage": "100Gi"}}}}'

New nodes added later still get the size in the template.

Shared storage for Stroom volumes

Stroom stores stream data in data volumes, which you configure in Stroom (Volumes in the user interface). In a cluster, every node must be able to reach them, so put them on shared storage mounted into every node.

spec.extraVolumes defines volumes for every node Pod. spec.extraVolumeMounts mounts them into the Stroom container of every node set, and nodeSets[].extraVolumeMounts adds mounts for one node set only:

spec:
  extraVolumes:
    - name: shared
      nfs:
        server: nfs.example.com
        path: /exports/stroom
  extraVolumeMounts:
    - name: shared
      mountPath: /stroom/shared

Any volume type works, for example a persistentVolumeClaim for a ReadWriteMany claim that you create. Then, in Stroom, create the data volumes under the mount path, for example /stroom/shared/data.

You can use the same mechanism to provide other files to Stroom, such as certificates for a Kafka connection, from a Secret or ConfigMap.

Deleting PersistentVolumeClaims

By default, PersistentVolumeClaims are never deleted. When a node is recreated, or the cluster is deleted and created again, each node gets back the claim (and data) it had before. spec.volumeClaimDeletePolicy changes this:

volumeClaimDeletePolicy Scale-down Cluster deletion
Not set (default) Claims kept Claims kept
DeleteOnScaledownOnly Claims of the removed nodes deleted Claims kept
DeleteOnScaledownAndClusterDeletion Claims of the removed nodes deleted All the cluster's claims deleted

Whether the PersistentVolume, and its data, survives a deleted claim depends on the volume's reclaim policy. With Retain, the volume moves to Released and must be cleaned up and made Available again by hand before it can be reused.