Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

9769 fix docker push #9772

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions .github/workflows/container_app_push.yml
Original file line number Diff line number Diff line change
Expand Up @@ -99,8 +99,12 @@ jobs:
name: "Package & Publish"
runs-on: ubuntu-latest
# Only run this job if we have access to secrets. This is true for events like push/schedule which run in
# context of main repo, but for PRs only true if coming from the main repo! Forks have no secret access.
if: needs.check-secrets.outputs.available == 'true'
# context of the main repo, but for PRs only true if coming from the main repo! Forks have no secret access.
#
# Note: The team's decision was to not auto-deploy an image on any git push where no PR exists (yet).
# Accordingly, only run for push events on branches develop and master.
if: needs.check-secrets.outputs.available == 'true' &&
( github.event_name != 'push' || ( github.event_name == 'push' && contains(fromJSON('["develop", "master"]'), github.ref_name)))
steps:
- uses: actions/checkout@v3
- uses: actions/setup-java@v3
Expand All @@ -109,7 +113,7 @@ jobs:
distribution: temurin

# Depending on context, we push to different targets. Login accordingly.
- if: ${{ github.event_name != 'pull_request' }}
- if: github.event_name != 'pull_request'
name: Log in to Docker Hub registry
uses: docker/login-action@v2
with:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/spi_release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@ name: Dataverse SPI

on:
push:
branch:
branches:
- "develop"
paths:
- "modules/dataverse-spi/**"
pull_request:
branch:
branches:
- "develop"
paths:
- "modules/dataverse-spi/**"
Expand Down
109 changes: 109 additions & 0 deletions doc/sphinx-guides/source/container/auto-usage.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
Automation Usage
================

With containers, a lot of automation becomes easier or is even enabled in the first place. Using containers
for Continuous Integration, Continuous Deployment or even fancy Continuous Benchmarking opens up a complete new
range of possibilities.

This section describes how the upstream project ships ready to use container images in an automated fashion
using Github Actions.

.. contents:: |toctitle|
:local:

Continuous Deployment for Container Images
------------------------------------------

The following diagram highlights events on Github and how the CI pipeline delivers ready to use images.

Main Image Workflow
^^^^^^^^^^^^^^^^^^^

This workflow can be found in ``container_app_push.yml``. It provides a place to enable integration tests using
containers and can produce an :doc:`application image <app-image>` as well as a :doc:`configbaker image <configbaker-image>`.

It is triggered by a) pull requests with changes to the images source files or b) called from other workflows, namely
the workflows to produce the base image and run Maven unit tests.

The produces images are either meant to be pushed to Docker Hub for common use or to the Github Container Registry
for preview images of pull request code for testing and development purposes. For preview images, the job will leave
a comment on the pull request with details where to find and use them.

**Note 1:** Secrets are available only to jobs running with the organizational context, which means no pull request from a
fork can use this workflow to push images to some place. Any feature branch within the main repository has access to
these secrets. Accordingly, the deployment job needs to look closely when to run or not and how to tag images.

**Note 2:** The workflow file contains another job to sync image descriptions on Docker Hub on push events to the
``develop`` branch. It is left out of the diagram below to reduce complexity.

.. graphviz::

digraph shipit {
node [shape=rect]
rankdir=TB
splines=ortho

trigger [label="CI\ntrigger", shape=circle, margin=0]
trigger -> build [label="start"]

subgraph build_test {
cluster=true
label="Job: Build & Test"
rank=same

build [label="Build app and app image\nw/ runner architecture"]
testing [label="Testing (future)", style="dashed"]
}

build -> secretsCheck [label="on success"]

subgraph secrets {
cluster=true
label="Job: Check Secrets"

secretsCheck [label="Secrets available\nwithin jobs?", shape=diamond]
}

secretsCheck -> decideSkip [taillabel="yes"]
secretsCheck -> end1 [taillabel="no"]
end1 [label="End", shape=circle, margin=0]

subgraph deploy {
cluster=true
label="Job: Deploy Images"

decideSkip [label="Trigger was no push\nor on branches develop or master?", shape=diamond]
decideSkip -> decideHubLogin [taillabel="yes"]
decideSkip -> end2 [taillabel="no"]

decideHubLogin [label="Trigger was PR?", shape=diamond]
hubLogin [label="Login at Docker Hub"]
ghcrLogin [label="Login at GHCR"]

decideHubLogin -> hubLogin [taillabel="no"]
decideHubLogin -> ghcrLogin [taillabel="yes"]
hubLogin -> decideMaster
ghcrLogin -> decideMaster

decideMaster [label="On branch master?", shape=diamond]
setTagsMaster [label="Use 'alpha' as tag"]
decidePR [label="Trigger was PR?", shape=diamond]
setForPR [label="Use branch name as tag,\nSet registry to GHCR"]

decideMaster -> setTagsMaster [taillabel="yes"]
decideMaster -> decidePR [taillabel="no"]
setTagsMaster -> decidePR
decidePR -> setForPR [taillabel="yes"]

buildLocal [label="Build app and app image with runner architecture"]
buildPush [label="Build and push multiarch images to registry"]
comment [label="If PR, leave comment on whereabouts"]
end2 [label="End", shape=circle, margin=0]

setForPR -> buildLocal
decidePR -> buildLocal [taillabel="no"]
buildLocal -> buildPush
buildPush -> comment
comment -> end2
}
}
1 change: 1 addition & 0 deletions doc/sphinx-guides/source/container/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ develop and extend them further are provided.
.. toctree::

dev-usage
auto-usage
base-image
app-image
configbaker-image