Technical resources
Exploring Certification Test Suites Integrated in DCI: A Comprehensive Review
Red Hat Distributed-CI (DCI) is a CI tool that can help you install Red Hat OpenShift (OCP) on different kinds of scenarios (baremetal, virtualized, etc.), automate the deployment of your plugins and workloads, and run a custom set of tests that may come from two different sources – tests defined by the own user or test suites provided by Red Hat, which can lead to the certification of the resources under test. Among other pieces of software, this automation is controlled by agents, written in Red Hat Ansible, that are deployed in users’ infrastructure and that perform all these tasks on a sequential basis, ensuring that all the configuration is correct before moving to subsequent steps.
This post will focus on the certification test suites that Red Hat has created for cloud-native applications to verify their adherence to best practices. In particular, we will see how DCI can simplify the setup required for these suites by handling all of the work related to the preparation of the configuration for the test suites, execution of the suites and report of the results.
The process of becoming certified
To become certification-ready, there are some necessary steps that need to be taken to reach that goal:
- Select the proper test suite that best suits the resources that we want to validate. It is not the same to test applications and services more oriented to the OpenShift infrastructure, compared to workloads (pods, operators, Helm charts, etc.) deployed on top of the OCP cluster.
- Understand what the requirements are for the specific certification test suite we want to execute. Do we need to prepare our resources in a specific way to run the certification tests? Do we need to create configuration for the execution? Do we need to install extra tools in our systems?
- Execute the test suites properly based on the previous requirements and understand what outputs from the test suites’ execution are useful for our particular case.
- Be able to submit the results so that they can be evaluated to check the suitability of the certification.
As you may imagine, each test suite may differ from others on each of these steps. What about having a tool to simplify this workflow and provide a single point of reference to users and partners for certification? Well, DCI is that tool.
In the next sections, we will focus on each certification test suite that can be found on DCI. Below, we present a summary of the test suites that are covered by DCI, and classified on infrastructure or workload tests, depending on the target. This is translated, in the end, in the type of agent that will run the certification tests: the DCI OpenShift Agent (or dci-openshift-agent), which takes care of all the configuration of the OCP cluster and, generally speaking, the underlying infrastructure, and the DCI OpenShift Application Agent (or dci-openshift-app-agent), which manages the workloads created on an already-running OCP cluster.
Agent - Infrastructure Tests
CNI Plugin Certification
The CNI Plugin Certification is intended for applications that offer network services on OpenShift through a CNI plugin. It enables continuous validation of CNI plugin compliance with the specified standards by utilizing the dci-openshift-agent during OpenShift cluster deployment.
To activate the tests, simply enable the dci_do_cni_tests flag. This will execute the recommended Red Hat tests and generate a comprehensive report for submission to the certification team for validation.
CSI Plugin Certification
The CSI Plugin Certification is designed for storage providers aiming to integrate their solutions with OpenShift using a CSI driver. The certification includes test suites that validate plugin compliance with CSI specifications and adhere to Red Hat's recommended best practices.
During cluster deployment using DCI, you need to provide a manifest file containing the plugin details and capabilities to run the certification tests. The job will generate a test report, which can be submitted to the certification team for validation. To enable the test, please refer to the dci_openshift_csi_test_manifest setting in the dci-openshift-agent documentation.
Application Agent - Workload Tests
Containers' certification: Preflight and OSCAP-podman
To become certification-ready, your container must pass all the test suites provided by two tools: Preflight for containers and OSCAP-podman.
- Preflight for containers checks for basic best practices: "do not run as root", "base on UBI", "provide license", "limit layer count", etc.
- OSCAP-podman scans for vulnerabilities, running over a thousand biweekly-updated tests.
To trigger these tests from DCI, you could customize an example configuration for your needs and use a debug guideline in case of any failing tests. You can also opt for end-to-end container certification from DCI. You only need to have a partner account then you can request DCI to automatically create a certification project and push the test results in that project. This allows you to simply click the publish button to add the container to the catalog.
Helm Chart Verifier
Chart Verifier allows you to validate the Helm chart against a configurable list of checks. The tool ensures that the Helm chart includes the associated metadata and formatting, and is distribution ready.
The ultimate goal of Helm chart certification is to get your chart merged into the OpenShift Helm Repository, and DCI can offer you a full service by running all the required tests and even automatically opening a pull request. Please note that not all Helm chart workloads can be certified; for example, the "certification-green" Helm chart must not contain any CRDs. CRDs should be defined using operators.
All Operators: Preflight Certification
General-purpose operators could be certified using the Preflight check-operator tool, which currently runs four tests. Three of these tests are basic formatting validations provided by the operator-sdk, and the fourth one, called DeployableByOLM, verifies if the operator could be deployed by OLM, having its Subscription and CustomServiceVersion up and running.
Similarly to Preflight check-container, here are several example configurations that you can customize for your needs, a debug guideline and the possibility to run an end-to-end certification process, which tests and merges your operator into the certified-operators repository.
Telco Workloads: CNF Certification
The Cloud-Native Network Functions (CNFs) certification suite is a set of CNF tests and a framework for building more. Its main goal is not to certify the workloads under test (which can be pods and operators), but rather to measure compliance with the good practices defined in the CNF Requirements document published by Red Hat.
This suite is run by the dci-openshift-app-agent, which uses DCI configuration to autodiscover the workloads. It then tests their interaction with OpenShift, and generates the report to be submitted to the Red Hat Certification Partner Connect portal (login required). We have also created a blog-post with an example configuration and an extended video presentation if you would like to learn more.
Next steps
Are you ready to start with your certification process using Red Hat test suites? Don’t hesitate to take a look at the DCI documentation to complete your view of the potential benefits that this CI tool can bring you to this journey towards certification. Both DCI and Telco Partner CI teams will be glad to help you in this process.
