Product image selection

All products of the Stackable Data Platform run on Kubernetes and are managed by Stackable Operators. One operator is responsible for exactly one product like Apache Spark. The products are deployed using Pods and their Containers to initialize and run the respective product. Containers require images (e.g. Docker) to run of. These images contain different tools for initialization jobs and/or the actual product itself. Images are prepared for different architectures and different product versions.

Stackable uses two separate versions to describe the images that are provided as part of the platform:

Product version
This is the version of the product which this image provides, so this could for example be Kafka 3.3.1

You can find all products and their supported versions in the supported versions overview. You can also find the supported versions per product on each operator page, for example for Apache Kafka. New versions, deprecations and removals are announced in the release notes.

Stackable version
This version is used to track changes to the structure of the image containing the product (in the version specified by product version). Stackable operators expect to find a very specific structure in the images they use to roll out the products. This can be things like startup scripts being present, parameters these startup scripts expect, presence or location of extra libraries and similar things. In order for our operators to work as intended every operator has to be used with images from the same release line as this operator.

What this means is, that for example the Stackable Operator for Apache HBase will by default try to use images with the same Stackable version, the following table shows a few examples to make this clearer:

Operator version	HBase version	Image
23.4.0	3.3.0	hbase:3.3.0-stackable23.4.0
23.4.0	3.3.1	hbase:3.3.1-stackable23.4.0
23.7.0	3.3.0	hbase:3.3.0-stackable23.7.0
23.7.0	3.3.1	hbase:3.3.1-stackable23.7.0

However, since the last digit of the Stackable version is considered to be a patch level indicator, operators will be compatible with all images from the same release line. So an operator of version 23.4.x will be compatible with all images of version 23.4.y. This is intended to allow shorter update cycles for users, when new image versions are released that may contain security fixes.

The following paragraphs explain the available settings and how they work.

At the bottom of this page in the common scenarios section some common update scenarios are explained as examples.

Stackable provided images

If your Kubernetes cluster has internet access, the easiest way is to use the publicly available images from the Image registry hosted by Stackable. If the Kubernetes cluster does not have internet access, a Custom docker registry or Custom images can be used.

Currently, you need to specify the product version. This can be found on the list of supported product versions or on the website of the product itself. This requirement might be relaxed in the future, as every platform release will ship wth a recommended product versions, which will be used by default.

Additionally, as images should be updated from time to time (e.g. new base image, security updates), a Stackable version can be provided. An image with the Stackable version 23.7.0 is fixed and will never change. Security updates within a release line will result in patch version bumps in the Stackable version to e.g. 23.7.1.

If you don’t specify the Stackable version, the operator will use its own version, e.g. 23.7.0. When using a nightly operator or a pr version, it will use the nightly 0.0.0-dev image.

All the available images (with their product and stackable version) can be found in our docker repository.

The versions need to be specified as follows:

spec:
  image:
    productVersion: 3.3.1
    # stackableVersion: 23.7.0 # optional

Custom docker registry

Custom docker registries can be used to fetch the image from a local image registry rather than from the internet. The perquisite is that you mirror all the required images the same way (with the same name and tag) as the images provided by Stackable.

Afterwards you can use the following snippet to configure your custom docker repo:

spec:
  image:
    productVersion: 3.3.1
    stackableVersion: 23.7.0
    repo: my.corp/myteam/stackable

This will change the image from the default Stackable repository docker.stackable.tech/stackable/kafka:3.3.1-stackable23.7.0 to my.corp/myteam/stackable/kafka:3.3.1-stackable23.7.0.

Custom images

Custom images can be used to fetch arbitrary images from local or public registries. In comparison to the Custom docker registry, this allows to provide self-hosted or user-created images (e.g. user extended Stackable images). If your image has other tags or names than the ones provided by Stackable you need to use this option.

spec:
  image:
    custom: my.corp/myteam/stackable/kafka:latest-and-greatest
    productVersion: 3.3.1

The Stackable Operators configure their respective products based on the product version. This affects e.g. configuration properties or available features. Therefore, the operators are dependent on the product and its product version contained in the custom image. It’s your responsibility to put in the correct product version.

Using custom images has a few limitations that users should be aware of.

The images will have to have the same structures that Stackable operators expect. This should usually be ensured by specifying a Stackable image in the FROM clause of the Dockerfile.
Images will have to be upgraded for every new Stackable release to follow structural changes that Stackable may have made to their images. When deriving images from official Stackable images this will mean updating the version of the image in the FROM clause to the correct Stackable release.
It is not possible to update the Stackable Platform to a new version without changing the deployed cluster definitions when using custom images. The recommended process here is:
- Tag clusters as "do not reconcile" (see Cluster operations)
- Update Stackable platform
- Change custom images in cluster specifications
- Remove "do not reconcile flag"

Common Scenarios

Planned platform updates

This is probably the most common scenario, users do not specify a Stackable version, and thus the operators always pick the image from their exact release. Updates happen by updating Stackable Operators, which will in turn restart the products with the new images.

Config

spec:
  image:
    productVersion: 3.3.1

Custom images / pinned images

When a setup requires the utmost stability, and it is preferable for things to break, rather than run with a different image version that for example has not been certified. Or when a user requires custom libraries / code in the images they run and build their own images derived from official Stackable images, this is the only possible way to do this.

Please see the warnings in custom images section above for how to upgrade in this scenario.

Config

spec:
  image:
    custom: my.corp/myteam/stackable/kafka:latest-and-greatest
    productVersion: 3.3.1