branch reef

[guits]

This add the required changes in order to build Ceph Reef
container images.

[ljflores]

Sloppy bash check failed from:

In /build-push-ceph-container-imgs.sh line 257:
         OSD_FLAVOR=${OSD_FLAVOR} \
                    ^-----------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
         OSD_FLAVOR="${OSD_FLAVOR}" \


In /build-push-ceph-container-imgs.sh line 263:
    make BASEOS_TAG=stream"${CENTOS_RELEASE}" CEPH_DEVEL=${DEVEL} RELEASE="${RELEASE}" BASEOS_REGISTRY="${CONTAINER_REPO_HOSTNAME}/centos" BASEOS_REPO=centos TAG_REGISTRY="${CONTAINER_REPO_ORGANIZATION}" build.parallel
                                                         ^------^ SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean: 
    make BASEOS_TAG=stream"${CENTOS_RELEASE}" CEPH_DEVEL="${DEVEL}" RELEASE="${RELEASE}" BASEOS_REGISTRY="${CONTAINER_REPO_HOSTNAME}/centos" BASEOS_REPO=centos TAG_REGISTRY="${CONTAINER_REPO_ORGANIZATION}" build.parallel

[ljflores]

The x86_64 container check failed with:

++ docker exec ceph-demo ceph --connect-timeout 3 --cluster ceph health detail
HEALTH_WARN too many PGs per OSD (329 > max 250)
[WRN] TOO_MANY_PGS: too many PGs per OSD (329 > max 250)
++ return 1
+ return
Error: Process completed with exit code 1.

[ljflores]

Here's where we define the nfs-ganesha version

[ljflores]

We verified in a reef container that nfs-ganesha v5.2 is used:

[root@3a5f3c37ef30 /]# ganesha.nfsd -version
NFS-Ganesha Release = V5.2

The filepath defined here leads to:
https://buildlogs.centos.org/centos/8-stream/storage/x86_64/nfsganesha-5/Packages/n/
https://buildlogs.centos.org/centos/8-stream/storage/aarch64/nfsganesha-5/Packages/n/

[ljflores]

x86_64 check failed from:

++ docker exec ceph-demo ceph --connect-timeout 3 --cluster ceph -s
  cluster:
    id:     27032162-06e4-481d-ab44-a3d8bc783ede
    health: HEALTH_WARN
            too many PGs per OSD (329 > max 250)
 
  services:
    mon:        1 daemons, quorum 64ed54fe4b66 (age 4m)
    mgr:        64ed54fe4b66(active, since 4m)
    mds:        1/1 daemons up
    osd:        2 osds: 2 up (since 4m), 2 in (since 4m)
    rbd-mirror: 1 daemon active (1 hosts)
    rgw:        1 daemon active (1 hosts, 1 zones)
    rgw-nfs:    1 daemon active (1 hosts, 1 zones)
 
  data:
    volumes: 1/1 healthy
    pools:   9 pools, 329 pgs
    objects: 247 objects, 459 KiB
    usage:   99 MiB used, 200 GiB / 200 GiB avail
    pgs:     328 active+clean
             1   active+clean+scrubbing
 
++ docker exec ceph-demo ceph --connect-timeout 3 --cluster ceph health detail
HEALTH_WARN too many PGs per OSD (329 > max 250)
[WRN] TOO_MANY_PGS: too many PGs per OSD (329 > max 250)
++ return 1
+ return

[ljflores]

Looks like this test is failing on other PRs too, so it's not caused by anything in this PR specifically.

[guits]

Looks like this test is failing on other PRs too, so it's not caused by anything in this PR specifically.

indeed, this has nothing to do

[ljflores]

@guits is this check required to merge the PR?

[guits]

@guits is this check required to merge the PR?

not really although I would prefer this test passes before merging but I get we are late in publishing the RC so we can discard it.
That being said, http://download.ceph.com/rpm-reef/el8//noarch/ceph-release-1-1.el8.noarch.rpm is missing so the Ceph Reef container image can't be built anyway.

[BlaineEXE]

This all looks good to me. Just one extremely small note that probably isn't worth changing in this PR.

[BlaineEXE]

I'm pretty sure buildlogs.centos.org repos support gpg signing, but this is a very small nit.

[BlaineEXE]

FWIW, I'm seeing the same demo cluster failure in this PR I'm working on: https://github.com/ceph/ceph-container/actions/runs/5137111619/jobs/9265592076?pr=2102

---

Update:

We "resolved" this in Rook by creating RGW pools with the min allowable number of PGs (I think that's 8 by default) before deploying RGWs: rook/rook#5098

That's the RGW root pool (.rgw.root), the RGW metadata pool, and the RGW data pool.

I think the all the standard pools would be .rgw.root, default.rgw.control, default.rgw.meta, default.rgw.buckets.index, default.rgw.buckets.data.

[ljflores]

Thank you @BlaineEXE !