Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new --remote-builder flag for remote builds via the Function Builder API #961

Merged
merged 1 commit into from
Jun 2, 2023

Conversation

NikhilSharmaWe
Copy link
Contributor

@NikhilSharmaWe NikhilSharmaWe commented Apr 29, 2023

Description

This PR works on adding the new remote-builder flag for faas-cli to be able to perform build operations from Function Builder API in Openfaas Pro. A new payload-secret flag is also added, in which the path of the payload.txt file containing the payload secret is specified.

Demo's :

Motivation and Context

How Has This Been Tested?

  • publish command with --remote-builder flag is successfully run
> faas-cli publish --remote-builder http://127.0.0.1:8081/build \
  -f homepage.yml \
  --payload-secret $HOME/.openfaas/payload.txt

Created buildx node: "multiarch"
[0] > Building homepage.
Clearing temporary build folder: ./build/homepage/
Preparing: ./homepage/ build/homepage/function
Building: docker.io/nikhilsharmawe/homepage:latest with golang-http template. Please wait..
1fa0f5174095a059ac57915d6838b0ec41d4223f154bfcdd9cd8068890dc507b
2023/05/10 17:20:41 homepage invoking the API for build at http://127.0.0.1:8081/build
v: 2023-05-10T17:20:41Z [internal] load .dockerignore
v: 2023-05-10T17:20:41Z [internal] load build definition from Dockerfile
v: 2023-05-10T17:20:41Z [internal] load build definition from Dockerfile 0.00s
v: 2023-05-10T17:20:41Z [internal] load .dockerignore 0.00s
v: 2023-05-10T17:20:41Z [internal] load .dockerignore
v: 2023-05-10T17:20:41Z [internal] load build definition from Dockerfile
s: 2023-05-10T17:20:41Z transferring context: 0
s: 2023-05-10T17:20:41Z transferring context: 32
s: 2023-05-10T17:20:41Z transferring context: 41
s: 2023-05-10T17:20:41Z transferring dockerfile: 0
s: 2023-05-10T17:20:41Z transferring dockerfile: 30
s: 2023-05-10T17:20:41Z transferring dockerfile: 1821
v: 2023-05-10T17:20:41Z [internal] load .dockerignore 0.03s
v: 2023-05-10T17:20:41Z [internal] load build definition from Dockerfile 0.05s
v: 2023-05-10T17:20:41Z [internal] load metadata for docker.io/library/alpine:3.17.2
v: 2023-05-10T17:20:41Z [internal] load metadata for ghcr.io/openfaas/of-watchdog:0.9.11
v: 2023-05-10T17:20:41Z [internal] load metadata for docker.io/library/golang:1.19-alpine
v: 2023-05-10T17:20:41Z [internal] load metadata for ghcr.io/openfaas/of-watchdog:0.9.11 0.55s
v: 2023-05-10T17:20:41Z [internal] load metadata for docker.io/library/alpine:3.17.2 2.41s
v: 2023-05-10T17:20:41Z [internal] load metadata for docker.io/library/golang:1.19-alpine 2.66s
v: 2023-05-10T17:20:44Z [ship 1/7] FROM docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517
v: 2023-05-10T17:20:44Z [ship 2/7] RUN apk --no-cache add ca-certificates     && addgroup -S app && adduser -S -g app app
v: 2023-05-10T17:20:44Z [ship 3/7] RUN mkdir -p /home/app     && chown app /home/app
v: 2023-05-10T17:20:44Z [ship 4/7] WORKDIR /home/app
v: 2023-05-10T17:20:44Z [build  1/14] FROM docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103
v: 2023-05-10T17:20:44Z [build  2/14] RUN apk --no-cache add git
v: 2023-05-10T17:20:44Z [watchdog 1/1] FROM ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4
v: 2023-05-10T17:20:44Z [build  3/14] COPY --from=watchdog /fwatchdog /usr/bin/fwatchdog
v: 2023-05-10T17:20:44Z [build  4/14] RUN chmod +x /usr/bin/fwatchdog
v: 2023-05-10T17:20:44Z [build  5/14] RUN mkdir -p /go/src/handler
v: 2023-05-10T17:20:44Z [build  6/14] WORKDIR /go/src/handler
v: 2023-05-10T17:20:44Z [internal] load build context
v: 2023-05-10T17:20:44Z [build  7/14] COPY . .
v: 2023-05-10T17:20:44Z [build  8/14] RUN test -z "$(gofmt -l $(find . -type f -name '*.go' -not -path "./vendor/*" -not -path "./function/vendor/*"))" || { echo "Run "gofmt -s -w" on your Golang code"; exit 1; }
v: 2023-05-10T17:20:44Z [build  9/14] RUN sh ./modules-cleanup.sh
v: 2023-05-10T17:20:44Z [build 10/14] WORKDIR /go/src/handler/function
v: 2023-05-10T17:20:44Z [build 11/14] RUN mkdir -p /go/src/handler/function/static
v: 2023-05-10T17:20:44Z [build 12/14] RUN GOOS=linux GOARCH=amd64 GOFLAGS= go test ./... -cover
v: 2023-05-10T17:20:44Z [build 13/14] WORKDIR /go/src/handler
v: 2023-05-10T17:20:44Z [build 14/14] RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GOFLAGS=     go build --ldflags "-s -w" -o handler .
v: 2023-05-10T17:20:44Z [ship 5/7] COPY --from=build --chown=app /go/src/handler/handler           .
v: 2023-05-10T17:20:44Z [ship 6/7] COPY --from=build --chown=app /usr/bin/fwatchdog                .
v: 2023-05-10T17:20:44Z [ship 7/7] COPY --from=build --chown=app /go/src/handler/function/static   static
v: 2023-05-10T17:20:44Z [build  1/14] FROM docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103
v: 2023-05-10T17:20:44Z [internal] load build context 0.00s
v: 2023-05-10T17:20:44Z [ship 1/7] FROM docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517
v: 2023-05-10T17:20:44Z [watchdog 1/1] FROM ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4
s: 2023-05-10T17:20:44Z resolve docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103 0
s: 2023-05-10T17:20:44Z resolve docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517 0
s: 2023-05-10T17:20:44Z resolve ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4 0
s: 2023-05-10T17:20:44Z resolve ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4 0
v: 2023-05-10T17:20:44Z [watchdog 1/1] FROM ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4 0.11s
s: 2023-05-10T17:20:44Z resolve docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103 0
v: 2023-05-10T17:20:44Z [build  1/14] FROM docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103 0.12s
v: 2023-05-10T17:20:44Z [internal] load build context
v: 2023-05-10T17:20:44Z [ship 1/7] FROM docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517 0.13s
s: 2023-05-10T17:20:44Z resolve docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517 0
s: 2023-05-10T17:20:44Z transferring context: 0
s: 2023-05-10T17:20:44Z transferring context: 32
s: 2023-05-10T17:20:44Z transferring context: 12810
v: 2023-05-10T17:20:44Z [internal] load build context 0.02s
v: 2023-05-10T17:20:44Z [ship 2/7] RUN apk --no-cache add ca-certificates     && addgroup -S app && adduser -S -g app app 0.00s
v: 2023-05-10T17:20:44Z [ship 3/7] RUN mkdir -p /home/app     && chown app /home/app 0.00s
v: 2023-05-10T17:20:44Z [ship 4/7] WORKDIR /home/app 0.00s
v: 2023-05-10T17:20:44Z [build  2/14] RUN apk --no-cache add git 0.00s
v: 2023-05-10T17:20:44Z [build  3/14] COPY --from=watchdog /fwatchdog /usr/bin/fwatchdog 0.00s
v: 2023-05-10T17:20:44Z [build  4/14] RUN chmod +x /usr/bin/fwatchdog 0.00s
v: 2023-05-10T17:20:44Z [build  5/14] RUN mkdir -p /go/src/handler 0.00s
v: 2023-05-10T17:20:44Z [build  6/14] WORKDIR /go/src/handler 0.00s
v: 2023-05-10T17:20:44Z [build  7/14] COPY . . 0.00s
v: 2023-05-10T17:20:44Z [build  8/14] RUN test -z "$(gofmt -l $(find . -type f -name '*.go' -not -path "./vendor/*" -not -path "./function/vendor/*"))" || { echo "Run "gofmt -s -w" on your Golang code"; exit 1; } 0.00s
v: 2023-05-10T17:20:44Z [build  9/14] RUN sh ./modules-cleanup.sh 0.00s
v: 2023-05-10T17:20:44Z [build 10/14] WORKDIR /go/src/handler/function 0.00s
v: 2023-05-10T17:20:44Z [build 11/14] RUN mkdir -p /go/src/handler/function/static 0.00s
v: 2023-05-10T17:20:44Z [build 12/14] RUN GOOS=linux GOARCH=amd64 GOFLAGS= go test ./... -cover 0.00s
v: 2023-05-10T17:20:44Z [build 13/14] WORKDIR /go/src/handler 0.00s
v: 2023-05-10T17:20:44Z [build 14/14] RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GOFLAGS=     go build --ldflags "-s -w" -o handler . 0.00s
v: 2023-05-10T17:20:44Z [ship 5/7] COPY --from=build --chown=app /go/src/handler/handler           . 0.00s
v: 2023-05-10T17:20:44Z [ship 6/7] COPY --from=build --chown=app /usr/bin/fwatchdog                . 0.00s
v: 2023-05-10T17:20:44Z [ship 7/7] COPY --from=build --chown=app /go/src/handler/function/static   static 0.00s
v: 2023-05-10T17:20:44Z exporting to image
s: 2023-05-10T17:20:44Z exporting layers 0
s: 2023-05-10T17:20:44Z exporting layers 0
s: 2023-05-10T17:20:44Z exporting manifest sha256:bdd4df0c1bab88fad637972342d4d946bcdab79892fb1d7dc5efc6870ad22eb8 0
s: 2023-05-10T17:20:44Z exporting manifest sha256:bdd4df0c1bab88fad637972342d4d946bcdab79892fb1d7dc5efc6870ad22eb8 0
s: 2023-05-10T17:20:44Z exporting config sha256:af35980857a8bc06d203f69883ffa90bb4114871f73a76818e0f0b2c116db6b0 0
s: 2023-05-10T17:20:44Z exporting config sha256:af35980857a8bc06d203f69883ffa90bb4114871f73a76818e0f0b2c116db6b0 0
s: 2023-05-10T17:20:44Z pushing layers 0
s: 2023-05-10T17:20:46Z pushing layers 0
s: 2023-05-10T17:20:46Z pushing manifest for docker.io/nikhilsharmawe/homepage:latest@sha256:bdd4df0c1bab88fad637972342d4d946bcdab79892fb1d7dc5efc6870ad22eb8 0
s: 2023-05-10T17:20:46Z pushing manifest for docker.io/nikhilsharmawe/homepage:latest@sha256:bdd4df0c1bab88fad637972342d4d946bcdab79892fb1d7dc5efc6870ad22eb8 0
v: 2023-05-10T17:20:44Z exporting to image 1.78s
2023/05/10 17:20:46 homepage success building and pushing image: docker.io/nikhilsharmawe/homepage:latest
[0] < Building homepage done in 4.75s.
[0] Worker done.

Total build time: 4.75s
  • and the created image is successfully deployed:
> faas-cli deploy --image nikhilsharmawe/homepage:latest

Deployed. 202 Accepted.
URL: http://127.0.0.1:8080/function/homepage
  • in case of a syntax error in the handler func:
> faas-cli publish --remote-builder http://127.0.0.1:8081/build \
  -f homepage.yml \
  --payload-secret $HOME/.openfaas/payload.txt

Created buildx node: "multiarch"
[0] > Building homepage.
Clearing temporary build folder: ./build/homepage/
Preparing: ./homepage/ build/homepage/function
Building: docker.io/nikhilsharmawe/homepage:latest with golang-http template. Please wait..
2b2a1a5b83e81c1a724ccf80a8871086b595ddba08ed24e63669288f056b6df0
2023/05/10 17:23:29 homepage invoking the API for build at http://127.0.0.1:8081/build
v: 2023-05-10T17:23:29Z [internal] load .dockerignore
v: 2023-05-10T17:23:29Z [internal] load build definition from Dockerfile
v: 2023-05-10T17:23:29Z [internal] load .dockerignore 0.00s
v: 2023-05-10T17:23:29Z [internal] load build definition from Dockerfile 0.00s
v: 2023-05-10T17:23:29Z [internal] load build definition from Dockerfile
v: 2023-05-10T17:23:29Z [internal] load .dockerignore
s: 2023-05-10T17:23:29Z transferring dockerfile: 0
s: 2023-05-10T17:23:29Z transferring dockerfile: 30
s: 2023-05-10T17:23:29Z transferring dockerfile: 1821
s: 2023-05-10T17:23:29Z transferring context: 0
s: 2023-05-10T17:23:29Z transferring context: 32
s: 2023-05-10T17:23:29Z transferring context: 41
v: 2023-05-10T17:23:29Z [internal] load build definition from Dockerfile 0.11s
v: 2023-05-10T17:23:29Z [internal] load .dockerignore 0.17s
v: 2023-05-10T17:23:29Z [internal] load metadata for docker.io/library/alpine:3.17.2
v: 2023-05-10T17:23:29Z [internal] load metadata for ghcr.io/openfaas/of-watchdog:0.9.11
v: 2023-05-10T17:23:29Z [internal] load metadata for docker.io/library/golang:1.19-alpine
v: 2023-05-10T17:23:29Z [internal] load metadata for ghcr.io/openfaas/of-watchdog:0.9.11 0.38s
v: 2023-05-10T17:23:29Z [internal] load metadata for docker.io/library/alpine:3.17.2 2.24s
v: 2023-05-10T17:23:29Z [internal] load metadata for docker.io/library/golang:1.19-alpine 2.29s
v: 2023-05-10T17:23:31Z [ship 1/7] FROM docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517
v: 2023-05-10T17:23:31Z [ship 2/7] RUN apk --no-cache add ca-certificates     && addgroup -S app && adduser -S -g app app
v: 2023-05-10T17:23:31Z [ship 3/7] RUN mkdir -p /home/app     && chown app /home/app
v: 2023-05-10T17:23:31Z [ship 4/7] WORKDIR /home/app
v: 2023-05-10T17:23:31Z [build  1/14] FROM docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103
v: 2023-05-10T17:23:31Z [build  2/14] RUN apk --no-cache add git
v: 2023-05-10T17:23:31Z [watchdog 1/1] FROM ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4
v: 2023-05-10T17:23:31Z [build  3/14] COPY --from=watchdog /fwatchdog /usr/bin/fwatchdog
v: 2023-05-10T17:23:31Z [build  4/14] RUN chmod +x /usr/bin/fwatchdog
v: 2023-05-10T17:23:31Z [build  5/14] RUN mkdir -p /go/src/handler
v: 2023-05-10T17:23:31Z [build  6/14] WORKDIR /go/src/handler
v: 2023-05-10T17:23:31Z [internal] load build context
v: 2023-05-10T17:23:31Z [build  7/14] COPY . .
v: 2023-05-10T17:23:31Z [build  8/14] RUN test -z "$(gofmt -l $(find . -type f -name '*.go' -not -path "./vendor/*" -not -path "./function/vendor/*"))" || { echo "Run "gofmt -s -w" on your Golang code"; exit 1; }
v: 2023-05-10T17:23:31Z [build  9/14] RUN sh ./modules-cleanup.sh
v: 2023-05-10T17:23:31Z [build 10/14] WORKDIR /go/src/handler/function
v: 2023-05-10T17:23:31Z [build 11/14] RUN mkdir -p /go/src/handler/function/static
v: 2023-05-10T17:23:31Z [build 12/14] RUN GOOS=linux GOARCH=amd64 GOFLAGS= go test ./... -cover
v: 2023-05-10T17:23:31Z [build 13/14] WORKDIR /go/src/handler
v: 2023-05-10T17:23:31Z [build 14/14] RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GOFLAGS=     go build --ldflags "-s -w" -o handler .
v: 2023-05-10T17:23:31Z [ship 5/7] COPY --from=build --chown=app /go/src/handler/handler           .
v: 2023-05-10T17:23:31Z [ship 6/7] COPY --from=build --chown=app /usr/bin/fwatchdog                .
v: 2023-05-10T17:23:31Z [ship 7/7] COPY --from=build --chown=app /go/src/handler/function/static   static
v: 2023-05-10T17:23:31Z [build  1/14] FROM docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103
v: 2023-05-10T17:23:31Z [internal] load build context 0.00s
v: 2023-05-10T17:23:31Z [watchdog 1/1] FROM ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4
v: 2023-05-10T17:23:31Z [ship 1/7] FROM docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517
s: 2023-05-10T17:23:31Z resolve docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103 0
s: 2023-05-10T17:23:32Z resolve ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4 0
s: 2023-05-10T17:23:32Z resolve docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517 0
v: 2023-05-10T17:23:31Z [build  1/14] FROM docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103 0.22s
s: 2023-05-10T17:23:32Z resolve docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103 0
s: 2023-05-10T17:23:32Z resolve ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4 0
v: 2023-05-10T17:23:31Z [watchdog 1/1] FROM ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4 0.25s
v: 2023-05-10T17:23:32Z [internal] load build context
v: 2023-05-10T17:23:31Z [ship 1/7] FROM docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517 0.26s
s: 2023-05-10T17:23:32Z resolve docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517 0
s: 2023-05-10T17:23:32Z transferring context: 0
s: 2023-05-10T17:23:32Z transferring context: 32
s: 2023-05-10T17:23:32Z transferring context: 12809
v: 2023-05-10T17:23:32Z [internal] load build context 0.05s
v: 2023-05-10T17:23:32Z [build  2/14] RUN apk --no-cache add git 0.00s
v: 2023-05-10T17:23:32Z [build  3/14] COPY --from=watchdog /fwatchdog /usr/bin/fwatchdog 0.00s
v: 2023-05-10T17:23:32Z [build  4/14] RUN chmod +x /usr/bin/fwatchdog 0.00s
v: 2023-05-10T17:23:32Z [build  5/14] RUN mkdir -p /go/src/handler 0.00s
v: 2023-05-10T17:23:32Z [build  6/14] WORKDIR /go/src/handler 0.00s
v: 2023-05-10T17:23:32Z [build  7/14] COPY . . 0.00s
v: 2023-05-10T17:23:32Z [build  8/14] RUN test -z "$(gofmt -l $(find . -type f -name '*.go' -not -path "./vendor/*" -not -path "./function/vendor/*"))" || { echo "Run "gofmt -s -w" on your Golang code"; exit 1; } 0.00s
v: 2023-05-10T17:23:32Z [build  9/14] RUN sh ./modules-cleanup.sh 0.00s
v: 2023-05-10T17:23:32Z [build 10/14] WORKDIR /go/src/handler/function 0.00s
v: 2023-05-10T17:23:32Z [build 11/14] RUN mkdir -p /go/src/handler/function/static 0.00s
v: 2023-05-10T17:23:32Z [build 12/14] RUN GOOS=linux GOARCH=amd64 GOFLAGS= go test ./... -cover
l: 2023-05-10T17:23:32Z go: downloading github.com/openfaas/templates-sdk/go-http v0.0.0-20220408082716-5981c545cb03

l: 2023-05-10T17:23:38Z # handler/function
./handler.go:15:3: syntax error: unexpected return, expecting expression

v: 2023-05-10T17:23:32Z [build 12/14] RUN GOOS=linux GOARCH=amd64 GOFLAGS= go test ./... -cover 7.64s
500
[0] < Building homepage done in 10.54s.
[0] Worker done.

Total build time: 10.54s
Errors received during build:
- homepage failure while building or pushing image docker.io/nikhilsharmawe/homepage:latest: failure: failed to solve: process "/bin/sh -c GOOS=${TARGETOS} GOARCH=${TARGETARCH} GOFLAGS=${GOFLAGS} go test ./... -cover" did not complete successfully: exit code: 2

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I've read the CONTRIBUTION guide
  • I have signed-off my commits with git commit -s
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@NikhilSharmaWe NikhilSharmaWe force-pushed the remote-builder branch 2 times, most recently from 92d679c to 7ad71bc Compare April 29, 2023 08:33
@alexellis
Copy link
Member

Thanks for the contribution.

Generally, we need to see "How Has This Been Tested?" fully populated before we can consider a review.

builder/publish.go Outdated Show resolved Hide resolved
builder/publish.go Outdated Show resolved Hide resolved
@NikhilSharmaWe NikhilSharmaWe force-pushed the remote-builder branch 2 times, most recently from 441cdcf to 895a8be Compare April 29, 2023 11:33
@alexellis
Copy link
Member

How Has This Been Tested? Is still missing.

@alexellis
Copy link
Member

is run successfully with logging message: Success building image

Please share the output and verify that faas-cli deploy can use the image.

I'd expect you to use a stack.yml file without giving any extra arguments - this all gets loaded by the YAML file

@alexellis
Copy link
Member

The output looks a lot better. But I can't see the response from the server? There should be build logs.

We'd also want to do a build for a Go function with a syntax error so that we can prove the error flow works and gives the user the problem on the console whilst exiting non zero.

@NikhilSharmaWe
Copy link
Contributor Author

NikhilSharmaWe commented Apr 30, 2023

@alexellis When we try to use go function with a syntax error in the handler code, this error is logged:

Created buildx node: "multiarch"
[0] > Building homepage.
Clearing temporary build folder: ./build/homepage/
Preparing: ./homepage/ build/homepage/function
Building: docker.io/nikhilsharmawe/homepage:latest with golang-http template. Please wait..
/tmp/builder-3367007414/req.tar
20bd31cea19229a440689707dd1c922317f27e8747772f88da69a5723c98bb00
500
2023/04/30 09:35:54 Unable to build image docker.io/nikhilsharmawe/homepage:latest: failure: failed to solve: process "/bin/sh -c GOOS=${TARGETOS} GOARCH=${TARGETARCH} GOFLAGS=${GOFLAGS} go test ./... -cover" did not complete successfully: exit code: 2

Is this what you expect?

or we expect detailed logs something like we get in the case of build, which also gives user info about the statement responsible for the issue and solution:

Step 18/37 : RUN test -z "$(gofmt -l $(find . -type f -name '*.go' -not -path "./vendor/*" -not -path "./function/vendor/*"))" || { echo "Run \"gofmt -s -w\" on your Golang code"; exit 1; }
 ---> Running in 22a6770aa843
./function/handler.go:15:3: expected operand, found 'return'

But since we are not doing docker build / buildx here in remote-builder, these will not be logged in that case.

Please give your thoughts.

@alexellis
Copy link
Member

The API call that you're making should have both a status code to determine errors and a set of logs. Look at the docs for the format.

I also thought that authentication was required for the builder API - the payload secret?

@NikhilSharmaWe
Copy link
Contributor Author

NikhilSharmaWe commented Apr 30, 2023

@alexellis The payload secret is fetched through:

kubectl get secret \
    -n openfaas payload-secret -o jsonpath='{.data.payload-secret}' \
    | base64 --decode \
    > payload.txt

I currently do it manually.

@NikhilSharmaWe NikhilSharmaWe force-pushed the remote-builder branch 3 times, most recently from ed9ed28 to 1f3eb4f Compare April 30, 2023 16:26
@alexellis
Copy link
Member

I'm just wondering how you are able to only pass in a single argument here?

Shouldn't there be an additional argument for the payload secret?

@NikhilSharmaWe NikhilSharmaWe force-pushed the remote-builder branch 3 times, most recently from 419204b to a1c8f99 Compare May 3, 2023 06:03
@alexellis
Copy link
Member

We discussed the approach on the weekly call, I think you may have dropped off by then.

faas-cli shouldn't be calling any exec commands to kubectl.

In this case, have the user pass in the password via a file.

I'm not sure that having it go in via a flag is a good idea as it will get leaked in bash history.

--payload-secret $HOME/.openfaas/payload-secret

Could you make the change and demo it to us on the community call in ~ 1.5 hours from now?

https://docs.openfaas.com/community/

builder/publish.go Outdated Show resolved Hide resolved
builder/publish.go Outdated Show resolved Hide resolved
builder/publish.go Outdated Show resolved Hide resolved
builder/publish.go Outdated Show resolved Hide resolved
@alexellis
Copy link
Member

Could you try this also with faas-cli up --remote-builder etc?

@NikhilSharmaWe NikhilSharmaWe force-pushed the remote-builder branch 2 times, most recently from 03d17f2 to 6c6de6b Compare May 10, 2023 17:28
@NikhilSharmaWe NikhilSharmaWe force-pushed the remote-builder branch 4 times, most recently from baa6a2e to 9bf96f6 Compare May 17, 2023 02:59
@NikhilSharmaWe NikhilSharmaWe marked this pull request as ready for review May 18, 2023 10:00
@alexellis
Copy link
Member

Hi, please could you rebase this PR?

Also run git commit --amend -s to set the title to: Add --remote-builder flag to publish command

In the commit body, you can write anything else you want to summarise - including wrapping the text to 80 chars.

This is what I strive to follow myself - https://cbea.ms/git-commit/

builder/publish.go Outdated Show resolved Hide resolved
builder/publish.go Outdated Show resolved Hide resolved
builder/publish.go Outdated Show resolved Hide resolved
@alexellis
Copy link
Member

@welteki I'd like you to try this out on your machine for a regular build/publish (unaffected) and for a remote build - with one single build and multiple - also trying out build-args.

Thanks

@NikhilSharmaWe NikhilSharmaWe force-pushed the remote-builder branch 2 times, most recently from 052f112 to fac7588 Compare May 24, 2023 14:53
@welteki
Copy link
Member

welteki commented May 31, 2023

@welteki I'd like you to try this out on your machine for a regular build/publish (unaffected) and for a remote build - with one single build and multiple - also trying out build-args.

Tested this:

  • ✅ build/publish without build args
  • ✅ remote build/publish without build args
  • ✅ build/publish with build args
  • ❌ remote build/publish with build args

Build args are not working for remote builds.

I used the python-http template and tried setting a custom python version through build args.

faas-cli template store pull python3-http

faas-cli new python-fn --lang python3-http --prefix ttl.sh
faas-cli up -f python-fn.yml \
  --build-arg PYTHON_VERSION=3.10 \
  --remote-builder http://127.0.0.1:8081/build \
  --payload-secret ./payload.txt

The remote builder builds the function but in the logs I see lines like:

v: 2023-05-31T11:09:55Z [build  1/16] FROM docker.io/library/python:3.11-alpine@sha256:4e8e9a59bf1b3ca8e030244bc5f801f23e41e37971907371da21191312087a07

It uses the default python version 3.11 instead of the version set in the build arg. This indicates the build arg is not passed to the remote builder.

@NikhilSharmaWe
Copy link
Contributor Author

NikhilSharmaWe commented May 31, 2023

@welteki I have found the fix.
Currently, I am traveling but will fix it asap and will ping you for a review after.

Thank you for testing and informing the issue.

Signed-off-by: Nikhil Sharma <nikhilsharma230303@gmail.com>
BuildOptPackages: buildOptPackages,
BuildLabelMap: buildLabelMap,
}
if err := makeTar(builderConfig{Image: imageName, BuildArgs: buildArgMap}, path.Join("build", functionName), tarPath); err != nil {
Copy link
Contributor Author

@NikhilSharmaWe NikhilSharmaWe May 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@welteki
Now I also add buildArgMap in the builderConfig for remote-builder, which I was not doing before.

Can you try it out now?

@alexellis
Copy link
Member

@NikhilSharmaWe can you try the example provided by Han please?

@NikhilSharmaWe
Copy link
Contributor Author

NikhilSharmaWe commented Jun 2, 2023

@alexellis I have tested and it works fine now with Build Args.

Copy link
Member

@alexellis alexellis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

@alexellis alexellis merged commit 5190205 into openfaas:master Jun 2, 2023
@alexellis alexellis changed the title add new remote-builder flag to build with the Function Builder API through faas-cli Add new --remote-builder flag to build with the Function Builder API through faas-cli Jun 2, 2023
@alexellis
Copy link
Member

@NikhilSharmaWe please could you add example commands and usage under the curl instructions in the docs?

Clone the repo and run it locally so you can test the layout etc.

https://github.com/openfaas/docs/blob/master/docs/openfaas-pro/builder.md

https://docs.openfaas.com/openfaas-pro/builder/

@alexellis alexellis changed the title Add new --remote-builder flag to build with the Function Builder API through faas-cli Add new --remote-builder flag for remote builds via the Function Builder API Jun 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remote build command using the Function Builder API
3 participants