*This blog post has been updated with some new information and data can be found here
As we announced in this blog post on August 31st, Kubernetes has removed several APIs from version 1.22. Any Operators utilizing these deprecated APIs will not work on OpenShift 4.9+ (or any cluster vendor utilizing Kubernetes 1.22) and will block your customers from upgrading to OpenShift 4.9 (recently announced) unless you take actions summarized below.
To aid in this transition, OpenShift 4.8 currently raises events and alerts for cluster administrators when an Operator incompatible with OpenShift 4.9/Kubernetes 1.22 is present.
Now that the API removal has officially been completed, there are several additional steps and considerations that need to be taken to update your operator properly to support OpenShift 4.9+.
Updating my Operator to support OpenShift 4.9+ after the API removal
To ensure a smooth upgrade path for users, you will need version compatibility with OpenShift 4.8+ at a minimum (or beginning with the earliest version you want to support) . This way, for users that are on a 4.8 cluster, they can upgrade your Operator before upgrading their cluster to 4.9. If you need more information regarding how to make these specific updates, reference this Kubernetes blog post. This is important, as it helps to provide a safe upgrade path for users moving to 4.9.
Let’s start with the most simple way to update and republish.
If you do not need or want your users to install the previous versions of your operator and your operator does not contain any controller logic requiring users to follow a specific upgrade path, such as migrating from version A -> B -> C, follow these steps:
1. Since we have officially removed incompatible operators entirely, you cannot publish new bundles with the replaces field configuration in your CSVs, because no previous version exists to replace. However, you can still publish your new bundle using the skips or skipsRange options instead of replaces.
By using these field options, you allow your users to upgrade their installed operators by skipping a version or a range of versions:
metadata:
name: mynewoperator.v1.0.3
annotations:
olm.skipRange: "<1.0.3"2. Set the com.redhat.openshift.versions annotation in the annotations.yaml file of your updated operator metadata bundle to “v4.8”:
annotations:
# This annotation will allow your bundle to be published on
# 4.8 and beyond
com.redhat.openshift.versions: "v4.8"Do the same in your bundle image project. If using the OperatorSDK you will make the following addition in your bundle’s Dockerfile:
FROM scratch # The following annotation will allow your bundle to be published on # 4.8 and beyond LABEL com.redhat.openshift.versions="v4.8"
Note: For further information please reference Controlling Operator compatibility with OpenShift Container Platform versions
Additional steps
There are more steps needed and special considerations that might be required in specific scenarios. If you want or need the previous versions of your operator, including the deprecated APIs, to be installable AND you want to allow upgrades of your Operator after the customer has upgraded to OpenShift 4.9, follow these steps:
In this scenario, the following steps must be followed:
Create two separate operator metadata bundles, each with the same structure and data you typically include, but with two differences:
a. The first bundle must be published only for OpenShift 4.8
b. The second bundle must skip previous versions and publish only to OpenShift 4.9 or 4.9+
Bundle A (targeting 4.8)
1. Configure CSV as normal, using replaces:
metadata: name: mynewbundle.v1.0.3 spec: replaces: myoldbundle.v1.0.2
2. Properly configure your annotations.yaml and the bundle’s Dockerfile to ensure distribution only on OpenShift 4.8:
(annotations.yaml)
annotations:
# The following annotation will allow your bundle to be published on
# 4.8 only
com.redhat.openshift.versions: "=v4.8"
(bundle.Dockerfile)
FROM scratch
# The following annotation will allow your bundle be published on 4.8 LABEL com.redhat.openshift.versions="=v4.8"Bundle B (targeting 4.9)
1. Configure CSV to use skipsRange instead of replaces: name: mynewoperator.v1.0.3 annotations: olm.skipRange: "<1.0.3"
2. Configure the annotations.yaml and bundle.Dockerfile labels as above, only for 4.9.
com.redhat.openshift.versions: "=v4.9"
If you DO NOT want to use skips or skipsRange, use the following publishing option:
Publish two separate bundles.
The first bundle may have any configuration you normally create, however it must only be published to OpenShift 4.8.
The second bundle will not have any replaces, skips, or skipsRange configuration and will only be published to 4.9. By removing this field, it will be as if it is the first publication made for the project.
Bundle A (targeting 4.8)
This bundle configuration will be the same as in the previous example.
1. Configure CSV as normal, using replaces:
metadata: name: mynewbundle.v1.0.3 spec: replaces: myoldbundle.v1.0.2
2. Properly configure your annotations.yaml and bundle.Dockerfile files to ensure distribution only on OpenShift 4.8:
(annotations.yaml)
annotations:
# The following annotation will allow your bundle to be published on
# 4.8 only
com.redhat.openshift.versions: "=v4.8"
(bundle.Dockerfile)
FROM scratch
# The following annotation will allow your bundle be published on 4.8 LABEL com.redhat.openshift.versions="=v4.8"
Bundle B (targeting 4.9)
1. Ensure this bundle does not use replaces, skips, or skipsRange in the CSV
2. Configure the annotations.yaml and bundle.Dockerfile files, as you did above, to indicate distribution only to OpenShift 4.9.
FAQ
How can I ensure my project is compatible with Kubernetes 1.22 and OpenShift 4.9?
Please reference this blog post and this Kubernetes blog to see all the required changes to update your operator.
Why can’t I use replaces to publish my new operator version?
When using replaces, the bundle version you are replacing must exist in that specific version of the OpenShift catalog. skips and skipsRange options work similarly to replaces, but they do not have the same constraints.
However, skips and skipsRange can get a bit tricky in the following circumstances:
Example:
v4 replaces v3
v3 skips v2
v2 replaces v1
In this scenario, users will be stuck at version 1. This is because OLM will never upgrade to a version that is skipped by another version. Since v2 is skipped by v3, OLM will not upgrade to v2, thus prohibiting an upgrade to v3 and beyond.
skips is intended to be used in the following way:
v4 replaces v3
v3 skips v2 and v3 replaces v1
v2 replaces v1
The original intent of skips was for when a version (v2 in this example) is released but has problems or defects, v3 is then shipped to correct the defects in v2. Therefore, the project owner “skips” v2 when possible, only allowing an upgrade from v1 to v3, and if v2 is already installed, upgrading to v3.
Why do I need to add the OCP labels to the bundle.Dockerfile and annotations.yaml files?
Our pipelines will only look for this label in the bundle image (bundle.Dockerfile) to know what version should be added in each OCP/OLM index catalog. However, the bundle is the source of truth and we recommend ensuring the value is properly added to the annotations.yaml file as well.
