Skip to content

Latest commit

 

History

History
59 lines (36 loc) · 8.67 KB

functionality-demonstrated.md

File metadata and controls

59 lines (36 loc) · 8.67 KB
Raw

Functionality demonstrated in this repository

While it is possible to limit the konflux-sample repositories to only illustrate a specific functionality or tip, many different changes are leveraged in this repository in order to improve the comprehension and maintenance of the repository. The key customizations and changes made are:

Using a pipelineRef to unify on a common PipelineDefinition

By default, Konflux will push two Tekton PipelineRun files for PAC to trigger on the cluster. Since these files share a large part of common specification, maintenance and customization can become harder especially if there are multiple artifacts being built in a repository with effectively the same repository.

There are two common pipelines defined in this repository, single-arch-build-pipeline and multi-arch-build-pipeline. Each of the PipelineRuns that are used by PAC then refer to one of the pipeline definitions with a pipelineRef, for example

  pipelineRef:
    name: multi-arch-build-pipeline

Building artifacts on multiple platforms

Tekton Pipelines are a collection of Tasks which run in pods in the cluster. The default Tekton PipelineRun that is pushed to repositories only build images for a single architecture -- the cluster's. This results in image artifacts only being built and pushed to the registry as an Image Manifest with the default docker-build pipeline. These images produced do not have an Image Index produced which enable the container execution environment to pull an image which matches the running system.

Since it is not currently possible to build on multiple architectures natively from within Tekton, a custom controller was developed for native building on multiple platforms in Konflux.

The pipeline definition supporting these builds in this repository is multi-arch-build-pipeline (in the future, a multi-platform pipeline will be available at onboard time). Remote build tasks can be triggered on any remote virtual machines that are configured via the PLATFORM parameter. A remote build is triggered for each specific architecture (for example, amd64) and then an OCI Image Index is generated referencing each of the architecture-specific Image Manifests.

Since we have a single PipelineDefinition for multiple component builds, some customization may be desired on a component level. In order to support this, parameters are added to the definition to allow specific architectures to be run as well as modifying the platform used (in case there a larger size is needed for a build).

EDIT: The original approach does not fully work as anticipated. While the common pipeline can be used, the parameters to enable the multi-arch builds are not sufficient to build on a subset of architectures. If all architectures are not produced, the values to the build-image-index cannot contain references to results from these skipped tasks. If skipped tasks are referenced, the Image Index generation will also be skipped. Use of matrix builds will improve the situation. This sample will be updated once the functionality is supported.

Trusted artifacts, removing the need for PVCs

The initial onboarding PRs from Konflux proposed a Tekton Pipeline that utilized PersistentVolumeClaims (PVC) for a shared workspace between tasks (see gatekeeper-operator, for an example). While these pipelines work for building artifacts, there are a couple primary limitations to the approach:

  • PVCs often have a quota within a workspace due to the infrastructure backing them. Removing a restriction on the PVC quotas increases the number of PipelineRuns that can progress in parallel.
  • All TaskRuns utilizing a shared PVC-backed workspace need to be run on the same node. This prevent k8s from scheduling each TaskRun from being able to effectively distribute workloads, especially when those workloads are long-running pipelines.
  • When content is shared in a PVC between tasks, the only way to ensure that the data hasn't been tampered with is to prevent any custom tasks that might access the workspace. This means that if users want to continue to pass Enterprise Contract policy verifications that they would be unable to add additional custom tasks to the pipeline like running unit tests on the repository or built artifacts.

Since data still needs to be shared between tasks, however, an alternative to PVCs are needed. Konflux enables pipelines to be able to leverage OCI-backed trusted artifacts. Instead of storing data in a PVC, tasks in Konflux can leverage the trusted artifact image to use and create OCI images to pass data between tasks. Since the digest of these images are recorded in the generated provenance, we can then verify the integrity of the data's chain of custody.

Utilization of on-cel-expressions for reducing redundant builds

After properly configuring the cel expressions, a PR which updates content that only affects a single component (for example, the only single-arch component) should not trigger builds for the rest of the components. This can be achieved by modifying the default on-cel-expressions to include filtering events based on the files changed. For example, the expressions for gatekeeper-operator-bundle which was built above is:

    pipelinesascode.tekton.dev/on-cel-expression: event == "pull_request" && target_branch == "main" && 
      (".tekton/single-arch-build-pipeline.yaml".pathChanged() || 
      ".tekton/gatekeeper-operator-bundle-pull-request.yaml".pathChanged() || 
      ".tekton/gatekeeper-operator-bundle-push.yaml".pathChanged() || 
      "Containerfile.gatekeeper-operator-bundle".pathChanged() ||
      "gatekeeper-operator".pathChanged())

This indicates that the PipelineRun will only be executed if the event is a PR to the main branch and if on of the 5 specified files were included in the changeset. Documentation for filtering events can be found in Pipelines as Code.

While this results in improved resource utilization (lower cloud spend), it does mean that not all artifacts will be built from every commit. If you need to identify the specific commit which produced an artifact, you can query the artifact's attestations for the build parameters.

Customization of files for component nudging

The gatekeeper and gatekeeper operator's "push" pipelines both specify the file to nudge component references in via the build.appstudio.openshift.io/build-nudge-files annotation. Since the location of the component references is atypical (i.e. not a Containerfile or yaml file), we need to configure this annotation. Once this annotation is set, the newly built operand and operator images will trigger a pull request against the update_bundle.sh file (i.e. konflux-ci/olm-operator-konflux-sample#21 and konflux-ci/olm-operator-konflux-sample#22).