update main and quincy to include NFS on ARM #2102

BlaineEXE · 2023-03-07T00:24:04Z

Update main and quincy build configs to include now-existing packages for NFS on ARM architectures.

Latest update:
I am chatting with Andrew Schoen to get a change into Shaman in the next few weeks to allow ceph-container to more reliably create ARM builds of images to facilitate my NFS v5 development.

Description of your changes:

Which issue is resolved by this Pull Request:
Resolves #

Checklist:

Documentation has been updated, if necessary.
Pending release notes updated with breaking and/or notable changes, if necessary.

BlaineEXE · 2023-03-07T00:24:54Z

.github/workflows/container.yml

@@ -39,4 +39,16 @@ jobs:
        run: docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

      - name: build the ceph container arm64 image
-        run: make BASEOS_REGISTRY=quay.io BASEOS_TAG=stream8 RELEASE="demo" BASEOS_REPO=centos/arm64v8 DAEMON_BASE_TAG="daemon-base:demo-centos-8-aarch64" DEMO_TAG="demo:demo-centos-stream8-aarch64" DAEMON_TAG="daemon:demo-centos-stream8-aarch64" FLAVORS="main,centos-arm64,8" build
+        run: make BASEOS_REGISTRY=quay.io/centos BASEOS_TAG=stream8 RELEASE="demo" BASEOS_REPO=centos DAEMON_BASE_TAG="daemon-base:demo-centos-8-aarch64" DEMO_TAG="demo:demo-centos-stream8-aarch64" DAEMON_TAG="daemon:demo-centos-stream8-aarch64" FLAVORS="main,centos-arm64,8" build


Updated because we no longer need centos/arm64v8 and can just use centos/centos

Also, because of the shaman issue I mention in the description, I expect this to fail on main at least on the first run, but it may not be a consistent failure.

I realize this is succeeding because the github runner is still an x86 system.

It might be good to make the arm builds do so via emulation, but it may be a fair bit of work: https://wiki.debian.org/QemuUserEmulation

BlaineEXE · 2023-03-07T00:26:47Z

.github/workflows/container.yml

-        run: make BASEOS_REGISTRY=quay.io BASEOS_TAG=stream8 RELEASE="demo" BASEOS_REPO=centos/arm64v8 DAEMON_BASE_TAG="daemon-base:demo-centos-8-aarch64" DEMO_TAG="demo:demo-centos-stream8-aarch64" DAEMON_TAG="daemon:demo-centos-stream8-aarch64" FLAVORS="main,centos-arm64,8" build
+        run: make BASEOS_REGISTRY=quay.io/centos BASEOS_TAG=stream8 RELEASE="demo" BASEOS_REPO=centos DAEMON_BASE_TAG="daemon-base:demo-centos-8-aarch64" DEMO_TAG="demo:demo-centos-stream8-aarch64" DAEMON_TAG="daemon:demo-centos-stream8-aarch64" FLAVORS="main,centos-arm64,8" build
+
+  arm64-quincy:


I expect this test to succeed since the quincy build takes NFS packages from the CentOS build service, and Kaleb got those builds working today.

guits · 2023-04-17T07:39:35Z

Update main and quincy build configs to include now-existing packages for NFS on ARM architectures.

@guits I need to look into who can help me fix what I consider to be a bug with shaman's build system as it reports the latest next version.

@ktdreyer @andrewschoen do any of you know who can help regarding shaman build system?

This site is used in the docker install to get the latest repo, but the latest repos don't always include ARM builds. For example, this is what I see today:

[
  {
    "status": "ready",
    "sha1": "91dd6865b71bbe99ad828c9c8bae1827922cd2a6",
    "extra": {
      "build_url": "https://jenkins.ceph.com/job/nfs-ganesha/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=huge/3427/",
      "root_build_cause": "TIMERTRIGGER",
      "version": "5-dev.3",
      "node_name": "172.21.2.6+braggi06",
      "job_name": "nfs-ganesha/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=huge",
      "package_manager_version": "54.34.3-"
    },
    "url": "https://2.chacra.ceph.com/r/nfs-ganesha/next/91dd6865b71bbe99ad828c9c8bae1827922cd2a6/centos/8/flavors/ceph_main/",
    "distro_codename": null,
    "modified": "2023-03-04 01:05:01.711522",
    "distro_version": "8",
    "project": "nfs-ganesha",
    "flavor": "ceph_main",
    "ref": "next",
    "chacra_url": "https://2.chacra.ceph.com/repos/nfs-ganesha/next/91dd6865b71bbe99ad828c9c8bae1827922cd2a6/centos/8/flavors/ceph_main/",
    "archs": [
      "source",
      "x86_64"
    ],
    "distro": "centos"
  },
  {
    "status": "ready",
    "sha1": "91dd6865b71bbe99ad828c9c8bae1827922cd2a6",
    "extra": {
      "build_url": "https://jenkins.ceph.com/job/nfs-ganesha/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=huge/3425/",
      "root_build_cause": "SCMTRIGGER",
      "version": "5-dev.3",
      "node_name": "172.21.2.6+braggi06",
      "job_name": "nfs-ganesha/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=huge",
      "package_manager_version": "54.34.3-"
    },
    "url": "https://3.chacra.ceph.com/r/nfs-ganesha/next/91dd6865b71bbe99ad828c9c8bae1827922cd2a6/centos/8/flavors/ceph_main/",
    "distro_codename": null,
    "modified": "2023-03-03 23:21:03.852603",
    "distro_version": "8",
    "project": "nfs-ganesha",
    "flavor": "ceph_main",
    "ref": "next",
    "chacra_url": "https://3.chacra.ceph.com/repos/nfs-ganesha/next/91dd6865b71bbe99ad828c9c8bae1827922cd2a6/centos/8/flavors/ceph_main/",
    "archs": [
      "x86_64",
      "source"
    ],
    "distro": "centos"
  }
]

Searching for arm returns no result

$ curl -s -L "https://shaman.ceph.com/api/repos/nfs-ganesha/next/latest/centos/8/flavors/ceph_main/repo?arch=aarch64"
<html>
 <head>
  <title>504 Gateway Timeout</title>
 </head>
 <body>
  <h1>504 Gateway Timeout</h1>
  The gateway has timed out.<br /><br />
no repository is ready for: nfs-ganesha/next


 </body>
</html>

Whereas searching for x86_64 does return a result:

curl -s -L "https://shaman.ceph.com/api/repos/nfs-ganesha/next/latest/centos/8/flavors/ceph_main/repo?arch=x86_64" 
[nfs-ganesha]
name=nfs-ganesha packages for $basearch
baseurl=https://2.chacra.ceph.com/r/nfs-ganesha/next/91dd6865b71bbe99ad828c9c8bae1827922cd2a6/centos/8/flavors/ceph_main/$basearch
enabled=1
gpgcheck=0
type=rpm-md

[nfs-ganesha-noarch]
name=nfs-ganesha noarch packages
baseurl=https://2.chacra.ceph.com/r/nfs-ganesha/next/91dd6865b71bbe99ad828c9c8bae1827922cd2a6/centos/8/flavors/ceph_main/noarch
enabled=1
gpgcheck=0
type=rpm-md

[nfs-ganesha-source]
name=nfs-ganesha source packages
baseurl=https://2.chacra.ceph.com/r/nfs-ganesha/next/91dd6865b71bbe99ad828c9c8bae1827922cd2a6/centos/8/flavors/ceph_main/SRPMS
enabled=1
gpgcheck=0
type=rpm-md

Even if it's not the absolute latest, I think shaman should keep at least a record of the last time arm was built so the curl command won't fail.

yes, in fact it might be valid not only for nfs-ganesha? I guess it is still better to test with a recent build than nothing ?

BlaineEXE · 2023-05-18T16:43:06Z

I am chatting with Andrew Schoen to get a change into Shaman in the next few weeks to allow ceph-container to more reliably create ARM builds of images to facilitate my NFS v5 development.

Shaman feature request tracked here: ceph/shaman#138

Update main and quincy build configs to include now-existing packages for NFS on ARM architectures. Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>

Change to a case-statement-based method of choosing the NFS ganesha repo. This should keep bloat down while allowing reliable arm builds from shaman for ceph main. Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>

BlaineEXE · 2023-10-09T21:06:31Z

Closing in favor of #2162

BlaineEXE requested a review from guits March 7, 2023 00:24

BlaineEXE commented Mar 7, 2023

View reviewed changes

BlaineEXE force-pushed the use-nfs-5-for-quincy branch from 89c7942 to 267639f Compare May 17, 2023 23:04

BlaineEXE mentioned this pull request May 18, 2023

Need way to get most recent ARM build reliably ceph/shaman#138

Open

BlaineEXE force-pushed the use-nfs-5-for-quincy branch 12 times, most recently from 0f432f5 to 5c7a499 Compare May 31, 2023 19:55

BlaineEXE marked this pull request as ready for review May 31, 2023 21:04

BlaineEXE added 2 commits July 12, 2023 10:17

update main and quincy to include NFS on ARM

8130458

Update main and quincy build configs to include now-existing packages for NFS on ARM architectures. Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>

change how centos selects NFS repos

9b969a0

Change to a case-statement-based method of choosing the NFS ganesha repo. This should keep bloat down while allowing reliable arm builds from shaman for ceph main. Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>

BlaineEXE marked this pull request as draft July 12, 2023 16:26

BlaineEXE force-pushed the use-nfs-5-for-quincy branch from 5c7a499 to 9b969a0 Compare July 12, 2023 16:26

BlaineEXE closed this Oct 9, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update main and quincy to include NFS on ARM #2102

update main and quincy to include NFS on ARM #2102

BlaineEXE commented Mar 7, 2023 •

edited

Loading

BlaineEXE Mar 7, 2023

BlaineEXE Mar 7, 2023

BlaineEXE Mar 7, 2023

BlaineEXE Mar 7, 2023

guits commented Apr 17, 2023

BlaineEXE commented May 18, 2023

BlaineEXE commented Oct 9, 2023

update main and quincy to include NFS on ARM #2102

update main and quincy to include NFS on ARM #2102

Conversation

BlaineEXE commented Mar 7, 2023 • edited Loading

BlaineEXE Mar 7, 2023

Choose a reason for hiding this comment

BlaineEXE Mar 7, 2023

Choose a reason for hiding this comment

BlaineEXE Mar 7, 2023

Choose a reason for hiding this comment

BlaineEXE Mar 7, 2023

Choose a reason for hiding this comment

guits commented Apr 17, 2023

BlaineEXE commented May 18, 2023

BlaineEXE commented Oct 9, 2023

BlaineEXE commented Mar 7, 2023 •

edited

Loading