Add support for arm64, armV7, armV8 #426

utegental · 2021-01-21T09:36:00Z

What would you like to be added:
There is no arm support. It would be good to see support this for those architectures.

Why is this needed:
Getting "standard_init_linux.go:219: exec user process caused: exec format error" for pods scheduled on arm nodes.
Usually it's easy to do - just making image multi-architecture =)

marquiz · 2021-01-21T14:10:11Z

I agree. There have been attempts in the past e.g. #203 and #327 at least. Now that we have K8s test-infra (gcb as the image builder) as the CI pipeline this might be doable

Diaoul · 2021-02-27T15:04:27Z

I would be interested in this as well. Can you elaborate on what kind of work is required to make this happen? From the previous PRs, it doesn't seem that a great deal of code is required.

xunholy · 2021-03-14T11:36:16Z

One is currently being maintained in parity with upstream docker.io/raspbernetes/node-feature-discovery which is built for armv7 & arm64

Diaoul · 2021-03-14T11:49:17Z

Great! Maybe this can be merged upstream?

zvonkok · 2021-03-16T09:54:00Z

@marquiz We can easily do this with github actions

jobs:
  build_job:
    runs-on: ubuntu-latest
    name: Build on ${{ matrix.arch }}

    strategy:
      matrix:
        include:
          - arch: armv7
            distro: ubuntu20.04
          - arch: aarch64
            distro: ubuntu20.04
          - arch: s390x
            distro: ubuntu20.04
          - arch: ppc64le
            distro: ubuntu20.04
    steps:

zvonkok · 2021-03-16T09:54:51Z

The only question is does gcr.io support manifest lists?

marquiz · 2021-03-16T10:04:10Z

@marquiz We can easily do this with github actions

Yeah, sure. But we use prow for building images

anthr76 · 2021-06-13T13:53:31Z

Any updates on this?

anta5010 · 2021-06-30T12:11:56Z

@marquiz Are there any plans to progress with this issue?
It looks like the main problem is not in updating the Makefile. This change is quite simple if buildx can be used. But, before creating a PR the CI pipelines need to be updated to support buildx. Do you know if it's possible and what help is required?

marquiz · 2021-07-07T11:24:01Z

@marquiz Are there any plans to progress with this issue?
It looks like the main problem is not in updating the Makefile. This change is quite simple if buildx can be used. But, before creating a PR the CI pipelines need to be updated to support buildx. Do you know if it's possible and what help is required?

Hmm, kubernetes test-infra might actually support docker buildx nowadays so we could do this. What needs to be done is

update Makefile
(probably) update scripts/test-infra/* to build images on all architectures
update docs (docs/get-started/deployment-and-usage.md and docs/advanced/developer-guide.md quickly come to mind)

Patches are welcome 😄

ArangoGutierrez · 2021-08-05T01:50:07Z

Best example I could find is https://github.com/kubernetes-sigs/service-catalog/blob/master/Makefile#L343
but I am not big fan of cross compilation tho, so the question is, do we want to do it like that?

ArangoGutierrez · 2021-08-05T01:50:25Z

https://kubernetes.slack.com/archives/C01A82FKSQK/p1628088075000500

marquiz · 2021-08-09T11:02:38Z

I am not big fan of cross compilation tho, so the question is, do we want to do it like that?

With docker buildx you shouldn't need that. Just something like docker buildx build --platform linux/arm64 . (haven't experimented this myself, though). Thus, I think we should be ok with relatively simple modifications to the Makefile

ArangoGutierrez · 2021-08-09T21:44:07Z

kubernetes/test-infra#22977 enables docker buildx on prow

ArangoGutierrez · 2021-08-09T22:05:49Z

We need to watch the output of

Multi arch for test-infra/images kubernetes/test-infra#16588
And wait for PR
images/bootstrap: build multi-archs amd64 & arm64 kubernetes/test-infra#23044
to be merged, once docker buildx support is fully supported, we can make the needed modifications to our Makefile

utegental · 2021-08-10T05:40:31Z

https://hub.docker.com/r/raspbernetes/node-feature-discovery - these guys have done multi-arch support several months ago. I don't know how exactly that was done, but this container works without issues.

marquiz · 2021-08-10T06:20:52Z

I don't know how exactly that was done, but this container works without issues.

I think they use docker buildx: https://github.com/raspbernetes/multi-arch-images

ArangoGutierrez · 2021-08-10T13:06:49Z

https://hub.docker.com/r/raspbernetes/node-feature-discovery is not a kubernetes-sigs repo, using prow, I think the main idea is to use prow as our official image builder right?

anthr76 · 2021-08-10T21:24:48Z

If prow isn't sufficient to emulate other arch's and there isn't infra to natively build on other arch's why use it? Is there a requirement? I know of other sigs building multi-arch.

Other arch's exist and this helps adoption greatly <3

marquiz · 2021-08-11T06:42:07Z

I think the main idea is to use prow as our official image builder right?

Yes. We use k8s test infra (prow) for CI and we're staying there as that tightly coupled with k8s container image hosting (k8s.gcr.io). Moreover, we want to serve all the container images from a single registry.

If prow isn't sufficient to emulate other arch's and there isn't infra to natively build on other arch's why use it? Is there a requirement? I know of other sigs building multi-arch.

I'm a bit lost here 🧐 As I commented earler, afaiu prow nowadays supports multiarch builds via docker buildx. And afaiu, Raspbernetes uses the same build method.

We just need a PR in NFD for enabling multiarch builds with docker buildx

kmhaeren · 2021-08-17T14:46:27Z

+1

This would be really handy to discover features on Jetson Xaviers

jonkerj · 2021-08-23T11:28:42Z

FWIW, I build (local) multiarch (arm64/amd64) docker images of NFS using this commandline:

$ git checkout v0.9.0
$ make IMAGE_BUILD_CMD="docker buildx build --push --platform linux/amd64,linux/arm64" IMAGE_REGISTRY="docker.io/jonkerj"

Should not be too difficult to patch up the makefile to do this by default. Maybe it needs some var containing the supported platforms, it's a matter of taste.

ArangoGutierrez · 2021-08-23T12:25:49Z

@jonkerj running your command returns

 Done setting up docker in docker.
+ WRAPPED_COMMAND_PID=174
+ wait 174
+ scripts/test-infra/build-image.sh
namespace: node-feature-discovery
image: k8s.gcr.io/nfd/node-feature-discovery:v0.10.0-devel-2-g58e147d
docker buildx build --platform linux/amd64,linux/arm64 --build-arg VERSION=v0.10.0-devel-2-g58e147d \
    --target full \
    --build-arg HOSTMOUNT_PREFIX=/host- \
    --build-arg BASE_IMAGE_FULL=debian:buster-slim \
    --build-arg BASE_IMAGE_MINIMAL=gcr.io/distroless/base \
    -t k8s.gcr.io/nfd/node-feature-discovery:v0.10.0-devel-2-g58e147d \
     \
     ./
error: multiple platforms feature is currently not supported for docker driver. Please switch to a different driver (eg. "docker buildx create --use")
make: *** [Makefile:70: image] Error 1

ArangoGutierrez · 2021-08-23T12:34:07Z

After splitting out the build by platform, is seems to work ok -> cbd42af

jonkerj · 2021-08-23T12:40:12Z

It could be a feature/limitation of your docker environment or buildx initialization, above invocation works for me (docker 20.10.8 / buildx v0.6.1 / moby/buildkit:buildx-stable-1)

jonkerj · 2021-08-23T13:51:06Z

The Raspbernetes project automatically builds multi-arch images of several projects, including NFD. You could take a look at the workflow file, which essentially wraps docker buildx build

ArangoGutierrez · 2021-08-23T18:14:30Z

Is not "my" environment, is kubernetes/test-infra prow environment, the one we use for all NFD CI and image building

zvonkok · 2022-06-01T19:42:52Z

@disconn3ct We're talking here about arm32v7 and arm64v8 right? Those are the terms docker buildx understands. Just to be at the same page.

eero-t · 2022-08-15T13:39:23Z

With #698 we now have ARM64

@marquiz That was merged over half a year ago, to a release done on March.

@disconn3ct We're talking here about arm32v7 and arm64v8 right? Those are the terms docker buildx understands. Just to be at the same page.

Is this ticket still open until all ARM targets supported by dockerx have been added?

liupeng0518 · 2022-10-24T08:09:30Z

hi guys,
when i build arm64 bin, An error occurred:

╭─  ~/Dev-env/node-feature-discovery   master                                                                                                 ✔ ─╮
╰─ GOOS=linux GOARCH=arm64 go install ./cmd/...                                                                                                      ─╯
# sigs.k8s.io/node-feature-discovery/source/cpu
source/cpu/cpu.go:203:63: undefined: getCpuidFlags

marquiz · 2022-10-24T19:11:42Z

when i build arm64 bin, An error occurred:

Probably because you don't have cross-build tools for c code. Try docker buildx (make image-all)

liupeng0518 · 2022-10-25T05:57:05Z

when i build arm64 bin, An error occurred:

Probably because you don't have cross-build tools for c code. Try docker buildx (make image-all)

ok , thanks.

k8s-triage-robot · 2023-01-23T06:54:19Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

ArangoGutierrez · 2023-01-23T10:43:51Z

/remove-lifecycle stale

k8s-triage-robot · 2023-04-23T11:03:40Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot · 2023-05-23T11:29:09Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle rotten
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot · 2023-06-22T12:12:28Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen
Mark this issue as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot · 2023-06-22T12:12:32Z

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied

After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied

After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen

Mark this issue as fresh with /remove-lifecycle rotten

Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Links2004 · 2024-04-01T17:46:56Z

/reopen

v0.15.3 does not work on armv7l

# arch
armv7l

# crictl --runtime-endpoint=unix:///run/containerd/containerd.sock pull registry.k8s.io/nfd/node-feature-discovery:v0.15.3
E0401 19:41:41.168147   29602 remote_image.go:171] "PullImage from image service failed" err="rpc error: code = NotFound desc = failed to pull and unpack image \"registry.k8s.io/nfd/node-feature-discovery:v0.15.3\": no match for platform in manifest: not found" image="registry.k8s.io/nfd/node-feature-discovery:v0.15.3"
FATA[0001] pulling image: rpc error: code = NotFound desc = failed to pull and unpack image "registry.k8s.io/nfd/node-feature-discovery:v0.15.3": no match for platform in manifest: not found

k8s-ci-robot · 2024-04-01T17:46:59Z

@Links2004: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

v0.15.3 does not work on armv7l

# arch
armv7l

# crictl --runtime-endpoint=unix:///run/containerd/containerd.sock pull registry.k8s.io/nfd/node-feature-discovery:v0.15.3
E0401 19:41:41.168147   29602 remote_image.go:171] "PullImage from image service failed" err="rpc error: code = NotFound desc = failed to pull and unpack image \"registry.k8s.io/nfd/node-feature-discovery:v0.15.3\": no match for platform in manifest: not found" image="registry.k8s.io/nfd/node-feature-discovery:v0.15.3"
FATA[0001] pulling image: rpc error: code = NotFound desc = failed to pull and unpack image "registry.k8s.io/nfd/node-feature-discovery:v0.15.3": no match for platform in manifest: not found

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

marquiz · 2024-04-02T07:41:21Z

@Links2004 would you be willing to work on this?

Links2004 · 2024-04-02T08:15:43Z

sure, can make some free time for this next week.
only need some pointers on how the build system works.
looks like some external tooling is used (at lease I dont see a github workflow for the image build)

ArangoGutierrez · 2024-04-02T08:28:56Z

/reopen
/assign Links2004

k8s-ci-robot · 2024-04-02T08:29:01Z

@ArangoGutierrez: Reopened this issue.

In response to this:

/reopen
/assign Links2004

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-triage-robot · 2024-05-02T08:35:44Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen
Mark this issue as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot · 2024-05-02T08:35:48Z

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied

After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied

After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen

Mark this issue as fresh with /remove-lifecycle rotten

Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

utegental added the kind/feature Categorizes issue or PR as related to a new feature. label Jan 21, 2021

Diaoul mentioned this issue Feb 28, 2021

Use official node-feature-discovery image (with arm) Diaoul/home-ops#120

Closed

ArangoGutierrez mentioned this issue Aug 23, 2021

WIP: Test if buildx is enabled at Prow #582

Closed

k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label May 27, 2022

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 23, 2023

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 23, 2023

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 23, 2023

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels May 23, 2023

k8s-ci-robot closed this as not planned Won't fix, can't repro, duplicate, stale Jun 22, 2023

k8s-ci-robot assigned Links2004 Apr 2, 2024

k8s-ci-robot reopened this Apr 2, 2024

Links2004 mentioned this issue Apr 6, 2024

add ARMv7 support #1659

Merged

k8s-ci-robot closed this as not planned Won't fix, can't repro, duplicate, stale May 2, 2024

Add support for arm64, armV7, armV8 #426

Add support for arm64, armV7, armV8 #426

Comments

utegental commented Jan 21, 2021

marquiz commented Jan 21, 2021

Diaoul commented Feb 27, 2021

xunholy commented Mar 14, 2021

Diaoul commented Mar 14, 2021

zvonkok commented Mar 16, 2021

zvonkok commented Mar 16, 2021

marquiz commented Mar 16, 2021

anthr76 commented Jun 13, 2021

anta5010 commented Jun 30, 2021

marquiz commented Jul 7, 2021

ArangoGutierrez commented Aug 5, 2021

ArangoGutierrez commented Aug 5, 2021

marquiz commented Aug 9, 2021

ArangoGutierrez commented Aug 9, 2021

ArangoGutierrez commented Aug 9, 2021 • edited Loading

utegental commented Aug 10, 2021

marquiz commented Aug 10, 2021

ArangoGutierrez commented Aug 10, 2021

anthr76 commented Aug 10, 2021

marquiz commented Aug 11, 2021

kmhaeren commented Aug 17, 2021 • edited Loading

jonkerj commented Aug 23, 2021 • edited Loading

ArangoGutierrez commented Aug 23, 2021

ArangoGutierrez commented Aug 23, 2021

jonkerj commented Aug 23, 2021 • edited Loading

jonkerj commented Aug 23, 2021

ArangoGutierrez commented Aug 23, 2021

zvonkok commented Jun 1, 2022

eero-t commented Aug 15, 2022

liupeng0518 commented Oct 24, 2022

marquiz commented Oct 24, 2022

liupeng0518 commented Oct 25, 2022

k8s-triage-robot commented Jan 23, 2023

ArangoGutierrez commented Jan 23, 2023

k8s-triage-robot commented Apr 23, 2023

k8s-triage-robot commented May 23, 2023

k8s-triage-robot commented Jun 22, 2023

k8s-ci-robot commented Jun 22, 2023

Links2004 commented Apr 1, 2024

k8s-ci-robot commented Apr 1, 2024

marquiz commented Apr 2, 2024

Links2004 commented Apr 2, 2024

ArangoGutierrez commented Apr 2, 2024

k8s-ci-robot commented Apr 2, 2024

k8s-triage-robot commented May 2, 2024

k8s-ci-robot commented May 2, 2024

ArangoGutierrez commented Aug 9, 2021 •

edited

Loading

kmhaeren commented Aug 17, 2021 •

edited

Loading

jonkerj commented Aug 23, 2021 •

edited

Loading

jonkerj commented Aug 23, 2021 •

edited

Loading