Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use only docker-bake.hcl for both Linux and Windows image definitions #426

Closed
lemeurherve opened this issue Jun 15, 2024 · 2 comments
Closed

Use only docker-bake.hcl for both Linux and Windows image definitions #426

lemeurherve opened this issue Jun 15, 2024 · 2 comments
Labels
bake enhancement windows

Comments

@lemeurherve
Copy link
Member

lemeurherve commented Jun 15, 2024 •
edited
Loading

What feature do you want to see added?

I propose to have a unique source of truth for all image definitions (Linux and Windows), variables and helper functions.

Goals:

  • Avoid mistakes, oversights, inconsistencies or more work to do thanks to unique location of image definitions
  • Simplify build.ps1, which won't have to massage data for images definitions anymore
  • Pave the way for when we'll be able to use docker buildx bake to build and publish images on Windows

How?

Even if docker buildx bake can't be used for building images on Windows yet, it can already be used on Windows with the --print parameter, outputting Windows images definitions as a JSON.

docker buildx bake windows --print 
{
  "group": {
    "default": {
      "targets": [
        "windows"
      ]
    },
    "nanoserver": {
      "targets": [
        "nanoserver-1809_jdk11",
        "nanoserver-1809_jdk17",
        "nanoserver-1809_jdk21",
        "nanoserver-ltsc2019_jdk11",
        "nanoserver-ltsc2019_jdk17",
        "nanoserver-ltsc2019_jdk21",
        "nanoserver-ltsc2022_jdk11",
        "nanoserver-ltsc2022_jdk17",
        "nanoserver-ltsc2022_jdk21"
      ]
    },
    "windows": {
      "targets": [
        "nanoserver",
        "windowsservercore"
      ]
    },
    "windowsservercore": {
      "targets": [
        "windowsservercore-ltsc2019_jdk11",
        "windowsservercore-ltsc2019_jdk17",
        "windowsservercore-ltsc2019_jdk21",
        "windowsservercore-ltsc2022_jdk11",
        "windowsservercore-ltsc2022_jdk17",
        "windowsservercore-ltsc2022_jdk21"
      ]
    }
  },
  "target": {
    "nanoserver-1809_jdk11": {
      "context": ".",
      "dockerfile": "windows/nanoserver/Dockerfile",
      "args": {
        "JAVA_HOME": "C:/openjdk-11",
        "JAVA_VERSION": "11.0.23+9",
        "TOOLS_WINDOWS_VERSION": "1809",
        "WINDOWS_VERSION_TAG": "1809"
      },
      "tags": [
        "docker.io/jenkins/ssh-agent:nanoserver-1809-jdk11"
      ],
      "platforms": [
        "windows/amd64"
      ]
    },
    "nanoserver-1809_jdk17": {
      "context": ".",
      "dockerfile": "windows/nanoserver/Dockerfile",
      "args": {
        "JAVA_HOME": "C:/openjdk-17",
        "JAVA_VERSION": "17.0.11+9",
        "TOOLS_WINDOWS_VERSION": "1809",
        "WINDOWS_VERSION_TAG": "1809"
      },
      "tags": [
        "docker.io/jenkins/ssh-agent:nanoserver-1809-jdk17"
      ],
      "platforms": [
        "windows/amd64"
      ]
    },
    "nanoserver-1809_jdk21": {
      "context": ".",
      "dockerfile": "windows/nanoserver/Dockerfile",
      "args": {
        "JAVA_HOME": "C:/openjdk-21",
        "JAVA_VERSION": "21.0.3+9",
        "TOOLS_WINDOWS_VERSION": "1809",
        "WINDOWS_VERSION_TAG": "1809"
      },
      "tags": [
        "docker.io/jenkins/ssh-agent:nanoserver-1809-jdk21"
      ],
      "platforms": [
        "windows/amd64"
      ]
    },
    "nanoserver-ltsc2019_jdk11": {
      "context": ".",
      "dockerfile": "windows/nanoserver/Dockerfile",
      "args": {
        "JAVA_HOME": "C:/openjdk-11",
        "JAVA_VERSION": "11.0.23+9",
        "TOOLS_WINDOWS_VERSION": "1809",
        "WINDOWS_VERSION_TAG": "ltsc2019"
      },
      "tags": [
        "docker.io/jenkins/ssh-agent:nanoserver-ltsc2019-jdk11"
      ],
      "platforms": [
        "windows/amd64"
      ]
    },
    "nanoserver-ltsc2019_jdk17": {
      "context": ".",
      "dockerfile": "windows/nanoserver/Dockerfile",
      "args": {
        "JAVA_HOME": "C:/openjdk-17",
        "JAVA_VERSION": "17.0.11+9",
        "TOOLS_WINDOWS_VERSION": "1809",
        "WINDOWS_VERSION_TAG": "ltsc2019"
      },
      "tags": [
        "docker.io/jenkins/ssh-agent:nanoserver-ltsc2019-jdk17"
      ],
      "platforms": [
        "windows/amd64"
      ]
    },
    "nanoserver-ltsc2019_jdk21": {
      "context": ".",
      "dockerfile": "windows/nanoserver/Dockerfile",
      "args": {
        "JAVA_HOME": "C:/openjdk-21",
        "JAVA_VERSION": "21.0.3+9",
        "TOOLS_WINDOWS_VERSION": "1809",
        "WINDOWS_VERSION_TAG": "ltsc2019"
      },
      "tags": [
        "docker.io/jenkins/ssh-agent:nanoserver-ltsc2019-jdk21"
      ],
      "platforms": [
        "windows/amd64"
      ]
    },
    "nanoserver-ltsc2022_jdk11": {
      "context": ".",
      "dockerfile": "windows/nanoserver/Dockerfile",
      "args": {
        "JAVA_HOME": "C:/openjdk-11",
        "JAVA_VERSION": "11.0.23+9",
        "TOOLS_WINDOWS_VERSION": "ltsc2022",
        "WINDOWS_VERSION_TAG": "ltsc2022"
      },
      "tags": [
        "docker.io/jenkins/ssh-agent:nanoserver-ltsc2022-jdk11"
      ],
      "platforms": [
        "windows/amd64"
      ]
    },
    "nanoserver-ltsc2022_jdk17": {
      "context": ".",
      "dockerfile": "windows/nanoserver/Dockerfile",
      "args": {
        "JAVA_HOME": "C:/openjdk-17",
        "JAVA_VERSION": "17.0.11+9",
        "TOOLS_WINDOWS_VERSION": "ltsc2022",
        "WINDOWS_VERSION_TAG": "ltsc2022"
      },
      "tags": [
        "docker.io/jenkins/ssh-agent:nanoserver-ltsc2022-jdk17"
      ],
      "platforms": [
        "windows/amd64"
      ]
    },
    "nanoserver-ltsc2022_jdk21": {
      "context": ".",
      "dockerfile": "windows/nanoserver/Dockerfile",
      "args": {
        "JAVA_HOME": "C:/openjdk-21",
        "JAVA_VERSION": "21.0.3+9",
        "TOOLS_WINDOWS_VERSION": "ltsc2022",
        "WINDOWS_VERSION_TAG": "ltsc2022"
      },
      "tags": [
        "docker.io/jenkins/ssh-agent:nanoserver-ltsc2022-jdk21"
      ],
      "platforms": [
        "windows/amd64"
      ]
    },
    "windowsservercore-ltsc2019_jdk11": {
      "context": ".",
      "dockerfile": "windows/windowsservercore/Dockerfile",
      "args": {
        "JAVA_HOME": "C:/openjdk-11",
        "JAVA_VERSION": "11.0.23+9",
        "TOOLS_WINDOWS_VERSION": "1809",
        "WINDOWS_VERSION_TAG": "ltsc2019"
      },
      "tags": [
        "docker.io/jenkins/ssh-agent:windowsservercore-ltsc2019-jdk11"
      ],
      "platforms": [
        "windows/amd64"
      ]
    },
    "windowsservercore-ltsc2019_jdk17": {
      "context": ".",
      "dockerfile": "windows/windowsservercore/Dockerfile",
      "args": {
        "JAVA_HOME": "C:/openjdk-17",
        "JAVA_VERSION": "17.0.11+9",
        "TOOLS_WINDOWS_VERSION": "1809",
        "WINDOWS_VERSION_TAG": "ltsc2019"
      },
      "tags": [
        "docker.io/jenkins/ssh-agent:windowsservercore-ltsc2019-jdk17"
      ],
      "platforms": [
        "windows/amd64"
      ]
    },
    "windowsservercore-ltsc2019_jdk21": {
      "context": ".",
      "dockerfile": "windows/windowsservercore/Dockerfile",
      "args": {
        "JAVA_HOME": "C:/openjdk-21",
        "JAVA_VERSION": "21.0.3+9",
        "TOOLS_WINDOWS_VERSION": "1809",
        "WINDOWS_VERSION_TAG": "ltsc2019"
      },
      "tags": [
        "docker.io/jenkins/ssh-agent:windowsservercore-ltsc2019-jdk21"
      ],
      "platforms": [
        "windows/amd64"
      ]
    },
    "windowsservercore-ltsc2022_jdk11": {
      "context": ".",
      "dockerfile": "windows/windowsservercore/Dockerfile",
      "args": {
        "JAVA_HOME": "C:/openjdk-11",
        "JAVA_VERSION": "11.0.23+9",
        "TOOLS_WINDOWS_VERSION": "ltsc2022",
        "WINDOWS_VERSION_TAG": "ltsc2022"
      },
      "tags": [
        "docker.io/jenkins/ssh-agent:windowsservercore-ltsc2022-jdk11"
      ],
      "platforms": [
        "windows/amd64"
      ]
    },
    "windowsservercore-ltsc2022_jdk17": {
      "context": ".",
      "dockerfile": "windows/windowsservercore/Dockerfile",
      "args": {
        "JAVA_HOME": "C:/openjdk-17",
        "JAVA_VERSION": "17.0.11+9",
        "TOOLS_WINDOWS_VERSION": "ltsc2022",
        "WINDOWS_VERSION_TAG": "ltsc2022"
      },
      "tags": [
        "docker.io/jenkins/ssh-agent:windowsservercore-ltsc2022-jdk17"
      ],
      "platforms": [
        "windows/amd64"
      ]
    },
    "windowsservercore-ltsc2022_jdk21": {
      "context": ".",
      "dockerfile": "windows/windowsservercore/Dockerfile",
      "args": {
        "JAVA_HOME": "C:/openjdk-21",
        "JAVA_VERSION": "21.0.3+9",
        "TOOLS_WINDOWS_VERSION": "ltsc2022",
        "WINDOWS_VERSION_TAG": "ltsc2022"
      },
      "tags": [
        "docker.io/jenkins/ssh-agent:windowsservercore-ltsc2022-jdk21"
      ],
      "platforms": [
        "windows/amd64"
      ]
    }
  }
}

This output can then be converted with yq (already required for building Windows images) into the format expected by docker compose, still used (for now) for building and publishing Windows images.

Corresponding docker compose file content 
services:
  nanoserver-1809_jdk11:
    image: docker.io/jenkins/ssh-agent:nanoserver-1809-jdk11
    build:
      context: .
      dockerfile: windows/nanoserver/Dockerfile
      args:
        JAVA_HOME: C:/openjdk-11
        JAVA_VERSION: 11.0.23+9
        TOOLS_WINDOWS_VERSION: "1809"
        WINDOWS_VERSION_TAG: "1809"
      tags:
        - docker.io/jenkins/ssh-agent:nanoserver-1809-jdk11
  nanoserver-1809_jdk17:
    image: docker.io/jenkins/ssh-agent:nanoserver-1809-jdk17
    build:
      context: .
      dockerfile: windows/nanoserver/Dockerfile
      args:
        JAVA_HOME: C:/openjdk-17
        JAVA_VERSION: 17.0.11+9
        TOOLS_WINDOWS_VERSION: "1809"
        WINDOWS_VERSION_TAG: "1809"
      tags:
        - docker.io/jenkins/ssh-agent:nanoserver-1809-jdk17
  nanoserver-1809_jdk21:
    image: docker.io/jenkins/ssh-agent:nanoserver-1809-jdk21
    build:
      context: .
      dockerfile: windows/nanoserver/Dockerfile
      args:
        JAVA_HOME: C:/openjdk-21
        JAVA_VERSION: 21.0.3+9
        TOOLS_WINDOWS_VERSION: "1809"
        WINDOWS_VERSION_TAG: "1809"
      tags:
        - docker.io/jenkins/ssh-agent:nanoserver-1809-jdk21
  nanoserver-ltsc2019_jdk11:
    image: docker.io/jenkins/ssh-agent:nanoserver-ltsc2019-jdk11
    build:
      context: .
      dockerfile: windows/nanoserver/Dockerfile
      args:
        JAVA_HOME: C:/openjdk-11
        JAVA_VERSION: 11.0.23+9
        TOOLS_WINDOWS_VERSION: "1809"
        WINDOWS_VERSION_TAG: ltsc2019
      tags:
        - docker.io/jenkins/ssh-agent:nanoserver-ltsc2019-jdk11
  nanoserver-ltsc2019_jdk17:
    image: docker.io/jenkins/ssh-agent:nanoserver-ltsc2019-jdk17
    build:
      context: .
      dockerfile: windows/nanoserver/Dockerfile
      args:
        JAVA_HOME: C:/openjdk-17
        JAVA_VERSION: 17.0.11+9
        TOOLS_WINDOWS_VERSION: "1809"
        WINDOWS_VERSION_TAG: ltsc2019
      tags:
        - docker.io/jenkins/ssh-agent:nanoserver-ltsc2019-jdk17
  nanoserver-ltsc2019_jdk21:
    image: docker.io/jenkins/ssh-agent:nanoserver-ltsc2019-jdk21
    build:
      context: .
      dockerfile: windows/nanoserver/Dockerfile
      args:
        JAVA_HOME: C:/openjdk-21
        JAVA_VERSION: 21.0.3+9
        TOOLS_WINDOWS_VERSION: "1809"
        WINDOWS_VERSION_TAG: ltsc2019
      tags:
        - docker.io/jenkins/ssh-agent:nanoserver-ltsc2019-jdk21
  nanoserver-ltsc2022_jdk11:
    image: docker.io/jenkins/ssh-agent:nanoserver-ltsc2022-jdk11
    build:
      context: .
      dockerfile: windows/nanoserver/Dockerfile
      args:
        JAVA_HOME: C:/openjdk-11
        JAVA_VERSION: 11.0.23+9
        TOOLS_WINDOWS_VERSION: ltsc2022
        WINDOWS_VERSION_TAG: ltsc2022
      tags:
        - docker.io/jenkins/ssh-agent:nanoserver-ltsc2022-jdk11
  nanoserver-ltsc2022_jdk17:
    image: docker.io/jenkins/ssh-agent:nanoserver-ltsc2022-jdk17
    build:
      context: .
      dockerfile: windows/nanoserver/Dockerfile
      args:
        JAVA_HOME: C:/openjdk-17
        JAVA_VERSION: 17.0.11+9
        TOOLS_WINDOWS_VERSION: ltsc2022
        WINDOWS_VERSION_TAG: ltsc2022
      tags:
        - docker.io/jenkins/ssh-agent:nanoserver-ltsc2022-jdk17
  nanoserver-ltsc2022_jdk21:
    image: docker.io/jenkins/ssh-agent:nanoserver-ltsc2022-jdk21
    build:
      context: .
      dockerfile: windows/nanoserver/Dockerfile
      args:
        JAVA_HOME: C:/openjdk-21
        JAVA_VERSION: 21.0.3+9
        TOOLS_WINDOWS_VERSION: ltsc2022
        WINDOWS_VERSION_TAG: ltsc2022
      tags:
        - docker.io/jenkins/ssh-agent:nanoserver-ltsc2022-jdk21
  windowsservercore-ltsc2019_jdk11:
    image: docker.io/jenkins/ssh-agent:windowsservercore-ltsc2019-jdk11
    build:
      context: .
      dockerfile: windows/windowsservercore/Dockerfile
      args:
        JAVA_HOME: C:/openjdk-11
        JAVA_VERSION: 11.0.23+9
        TOOLS_WINDOWS_VERSION: "1809"
        WINDOWS_VERSION_TAG: ltsc2019
      tags:
        - docker.io/jenkins/ssh-agent:windowsservercore-ltsc2019-jdk11
  windowsservercore-ltsc2019_jdk17:
    image: docker.io/jenkins/ssh-agent:windowsservercore-ltsc2019-jdk17
    build:
      context: .
      dockerfile: windows/windowsservercore/Dockerfile
      args:
        JAVA_HOME: C:/openjdk-17
        JAVA_VERSION: 17.0.11+9
        TOOLS_WINDOWS_VERSION: "1809"
        WINDOWS_VERSION_TAG: ltsc2019
      tags:
        - docker.io/jenkins/ssh-agent:windowsservercore-ltsc2019-jdk17
  windowsservercore-ltsc2019_jdk21:
    image: docker.io/jenkins/ssh-agent:windowsservercore-ltsc2019-jdk21
    build:
      context: .
      dockerfile: windows/windowsservercore/Dockerfile
      args:
        JAVA_HOME: C:/openjdk-21
        JAVA_VERSION: 21.0.3+9
        TOOLS_WINDOWS_VERSION: "1809"
        WINDOWS_VERSION_TAG: ltsc2019
      tags:
        - docker.io/jenkins/ssh-agent:windowsservercore-ltsc2019-jdk21
  windowsservercore-ltsc2022_jdk11:
    image: docker.io/jenkins/ssh-agent:windowsservercore-ltsc2022-jdk11
    build:
      context: .
      dockerfile: windows/windowsservercore/Dockerfile
      args:
        JAVA_HOME: C:/openjdk-11
        JAVA_VERSION: 11.0.23+9
        TOOLS_WINDOWS_VERSION: ltsc2022
        WINDOWS_VERSION_TAG: ltsc2022
      tags:
        - docker.io/jenkins/ssh-agent:windowsservercore-ltsc2022-jdk11
  windowsservercore-ltsc2022_jdk17:
    image: docker.io/jenkins/ssh-agent:windowsservercore-ltsc2022-jdk17
    build:
      context: .
      dockerfile: windows/windowsservercore/Dockerfile
      args:
        JAVA_HOME: C:/openjdk-17
        JAVA_VERSION: 17.0.11+9
        TOOLS_WINDOWS_VERSION: ltsc2022
        WINDOWS_VERSION_TAG: ltsc2022
      tags:
        - docker.io/jenkins/ssh-agent:windowsservercore-ltsc2022-jdk17
  windowsservercore-ltsc2022_jdk21:
    image: docker.io/jenkins/ssh-agent:windowsservercore-ltsc2022-jdk21
    build:
      context: .
      dockerfile: windows/windowsservercore/Dockerfile
      args:
        JAVA_HOME: C:/openjdk-21
        JAVA_VERSION: 21.0.3+9
        TOOLS_WINDOWS_VERSION: ltsc2022
        WINDOWS_VERSION_TAG: ltsc2022
      tags:
        - docker.io/jenkins/ssh-agent:windowsservercore-ltsc2022-jdk21
Current docker compose file content for comparison
Notice that almost all args currently have to be injected by build.ps1:

docker-ssh-agent/build-windows.yaml

Lines 1 to 39 in bad0642

services:
jdk11:
image: ${DOCKERHUB_ORGANISATION}/${DOCKERHUB_REPO}:${WINDOWS_FLAVOR}-${WINDOWS_VERSION_TAG}-jdk11
build:
context: ./
dockerfile: ./windows/${WINDOWS_FLAVOR}/Dockerfile
args:
JAVA_HOME: "C:/openjdk-11"
JAVA_VERSION: 11.0.23+9
WINDOWS_VERSION_TAG: ${WINDOWS_VERSION_TAG}
TOOLS_WINDOWS_VERSION: ${TOOLS_WINDOWS_VERSION}
tags:
- "${DOCKERHUB_ORGANISATION}/${DOCKERHUB_REPO}:${VERSION}-${WINDOWS_FLAVOR}-${WINDOWS_VERSION_TAG}-jdk11"
jdk17:
image: ${DOCKERHUB_ORGANISATION}/${DOCKERHUB_REPO}:${WINDOWS_FLAVOR}-${WINDOWS_VERSION_TAG}-jdk17
build:
context: ./
dockerfile: ./windows/${WINDOWS_FLAVOR}/Dockerfile
args:
JAVA_HOME: "C:/openjdk-17"
JAVA_VERSION: 17.0.11+9
WINDOWS_VERSION_TAG: ${WINDOWS_VERSION_TAG}
TOOLS_WINDOWS_VERSION: ${TOOLS_WINDOWS_VERSION}
tags:
- "${DOCKERHUB_ORGANISATION}/${DOCKERHUB_REPO}:${VERSION}-${WINDOWS_FLAVOR}-${WINDOWS_VERSION_TAG}-jdk17"
- "${DOCKERHUB_ORGANISATION}/${DOCKERHUB_REPO}:${VERSION}-${WINDOWS_FLAVOR}-${WINDOWS_VERSION_TAG}"
- "${DOCKERHUB_ORGANISATION}/${DOCKERHUB_REPO}:${WINDOWS_FLAVOR}-${WINDOWS_VERSION_TAG}"
jdk21:
image: ${DOCKERHUB_ORGANISATION}/${DOCKERHUB_REPO}:${WINDOWS_FLAVOR}-${WINDOWS_VERSION_TAG}-jdk21
build:
context: ./
dockerfile: ./windows/${WINDOWS_FLAVOR}/Dockerfile
args:
JAVA_HOME: "C:/openjdk-21"
JAVA_VERSION: 21.0.3+9
WINDOWS_VERSION_TAG: ${WINDOWS_VERSION_TAG}
TOOLS_WINDOWS_VERSION: ${TOOLS_WINDOWS_VERSION}
tags:
- "${DOCKERHUB_ORGANISATION}/${DOCKERHUB_REPO}:${VERSION}-${WINDOWS_FLAVOR}-${WINDOWS_VERSION_TAG}-jdk21"

Details

Avoid mistakes, oversights, inconsistencies or more work to do thanks to unique location of image definitions

Having only one place to store image definitions, build variables and helper functions would decrease the risk of mistakes or oversights, and reduce the work needed to change parameters common to both Linux and Windows, currently defined in two distinct files and a bit of data manipulations defined in build.ps1.

Ex: the work I've done for jenkinsci/docker#1891 to remove Java 11 from Weekly release line would have been much more simpler.

Something like that: 
-variable "jdks_to_build" {
-  default = [11, 17, 21]
+# Return the JDKs to build depending if the version matches LTS versionning pattern
+function "jdkstobuild" {
+  param  = [version]
+  # Return all JDKs only if the version matches LTS versionning pattern
+  result = equal(regex(version, ltsRegexPattern), "") ? [17, 21] : [11, 17, 21]
 }

 target "alpine" {
   matrix = {
-    jdk = jdks_to_build
+    jdk = jdkstobuild(VERSION)
   }

 target "debian" {
   matrix = {
-    jdk = jdks_to_build
+    jdk = jdkstobuild(VERSION)
   }

 target "nanoserver" {
   matrix = {
-    jdk = jdks_to_build
+    jdk = jdkstobuild(VERSION)
   }

 target "windowsservercore" {
   matrix = {
-    jdk = jdks_to_build
+    jdk = jdkstobuild(VERSION)
   }

Here is the code that would be required to remove Java 11 from Linux and Windows images here with the proposed enhancement:

variable "jdks_to_build" {
-   default = [11, 17, 21]
+   default = [17, 21]
}

Another place where it reduces complexity: the updatecli manifests, reducing the number of targets.

Implementing something like jenkinsci/docker-agent#816 would result in a new Dockerfile argument and a docker bake function returning the correct WINDOWS_VERSION_DIGEST depending on the Windows version, no blocker from the enhancement.

Simplify build.ps1, which won't have to massage data for images definitions anymore

Currently, build.ps1 is used to massage the data for the image definitions and is in charge of the special case like this:

docker-ssh-agent/build.ps1

Lines 39 to 51 in bad0642

# Ensure constant env vars used in the docker compose file are defined
$env:DOCKERHUB_ORGANISATION = "$Organisation"
$env:DOCKERHUB_REPO = "$Repository"
$env:VERSION = "$VersionTag"
$items = $ImageType.Split('-')
$env:WINDOWS_FLAVOR = $items[0]
$env:WINDOWS_VERSION_TAG = $items[1]
$env:TOOLS_WINDOWS_VERSION = $items[1]
if ($items[1] -eq 'ltsc2019') {
# There are no eclipse-temurin:*-ltsc2019 or mcr.microsoft.com/powershell:*-ltsc2019 docker images unfortunately, only "1809" ones
$env:TOOLS_WINDOWS_VERSION = '1809'
}

With this enhancement, build.ps1 would just be used for:

  • (new) Optionally setting up the docker compose file generation from the docker bake definition without having to work on them
  • Calling docker compose
  • Optionally setting up the tests harness and calling Pester to run it

Apart from this generation, the rest of the process is unchanged, just an additional init step to generate the docker compose file, optional if it already exists.

When docker buildx will be able to build images on Windows, build.ps1 role will be:

  • Calling docker buildx bake
  • Optionally setting up the tests harness and calling Pester to run it

Contributors will be able to either use build.ps1 to (re)generate their docker compose file, or a command line that will be explained in the README to generate themselves the docker compose file. (See "miscellaneous" below)

With this proposed change, build.ps1 isn't mandatory for contributors anymore, they will be able to build and publish their own images by using docker bake and docker compose only.

Pave the way for when we'll be able to use docker buildx bake to build and publish images on Windows

BuildKit seems to be able-ish to build images on Windows:

But docker buildx isn't yet.

Main issues to follow the progress:

When it will be ready, we'll be able to remove the docker compose file generation and call and just use docker buildx bake to build and publish all images.

Miscellaneous

Example of the call to generate the docker compose file from a Windows machine:

$ docker buildx bake --progress=plain --file=docker-bake.hcl windows --print `
    | yq --prettyPrint '.target[] | del(.output) | {(. | key): {\"image\": .tags[0], \"build\": .}}' | yq '{\"services\": .}' `
    | Out-File -FilePath build-windows.yaml
  • Use docker buildx bake to output image definitions from the "windows" bake target
  • Convert with yq to the format expected by docker compose
  • Store the result in the docker compose file

The only new requirements is docker buildx as docker, docker compose and yq are already required.

Corresponding change (missing the doc, incoming)

ed751ae

Working build locally and on ci.jenkins.io

https://ci.jenkins.io/job/Packaging/job/docker-ssh-agent/job/PR-415/88/

Upstream changes

No response

Are you interested in contributing this feature?

Yes:
This was referenced Jun 15, 2024
Merged
Closed
@lemeurherve
Copy link
Member Author

lemeurherve commented Jun 19, 2024 •
edited
Loading

Build https://ci.jenkins.io/job/Packaging/job/docker-ssh-agent/job/master/444/ successful 🎉

I forgot to remove the docker compose target from updatecli manifests, opened #430 that has been merged.

I'm triggering a new release to verify the publish process.

cc @jenkinsci/team-docker-packaging FTR.

@lemeurherve
Copy link
Member Author

lemeurherve commented Jun 19, 2024 •
edited
Loading

Release https://github.com/jenkinsci/docker-ssh-agent/releases/tag/5.41.0 completed on trusted.ci.jenkins.io with success 🥳

https://hub.docker.com/r/jenkins/ssh-agent/tags?page=&page_size=&ordering=&name=5.41.0

@lemeurherve lemeurherve mentioned this issue Jun 23, 2024
Merged
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bake enhancement windows
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@lemeurherve