Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement docker multi stage build and use common Go image #191

Merged
merged 2 commits into from
Feb 22, 2024
Merged

Implement docker multi stage build and use common Go image #191

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
98a9d4f
Implement docker multi stage build
chimanjain Feb 7, 2024
3001c0f
Update actions and dockerfile
chimanjain Feb 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 13 additions & 15 deletions .github/workflows/actions.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout the code
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Checkout dell-csi-extensions
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
repository: 'dell/dell-csi-extensions'
path: dell-csi-extensions
Expand All @@ -25,7 +25,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout the code
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Run the forbidden words scan
uses: dell/common-github-actions/code-sanitizer@main
with:
Expand All @@ -35,9 +35,9 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout the code
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Checkout dell-csi-extensions
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
repository: 'dell/dell-csi-extensions'
path: dell-csi-extensions
Expand All @@ -52,7 +52,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout the code
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Run Go Security
uses: securego/gosec@master
with:
Expand All @@ -62,7 +62,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout the code
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Run malware scan
uses: dell/common-github-actions/malware-scanner@main
with:
Expand All @@ -73,15 +73,15 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Set up Go 1.21+
uses: actions/setup-go@v4
uses: actions/setup-go@v5
with:
go-version: ^1.21
cache: false
id: go
- name: Checkout the code
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Checkout dell-csi-extensions
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
repository: 'dell/dell-csi-extensions'
path: dell-csi-extensions
Expand All @@ -93,10 +93,9 @@ jobs:
run: go mod tidy
- name: Build podmon Docker Image
run: |
cd cmd/podmon
chmod +x ../../scripts/buildubimicro.sh
make clean build-base-image build
podman build -t docker.io/podmon . --build-arg BASEIMAGE="localhost/resiliency-ubimicro"
chmod +x ./scripts/buildubimicro.sh
make build-base-image
podman build -t docker.io/podmon -f ./Dockerfile --build-arg GOIMAGE=golang:1.21 --build-arg BASEIMAGE="localhost/resiliency-ubimicro"
podman save docker.io/library/podmon -o /tmp/podmon.tar
docker load -i /tmp/podmon.tar
- name: Image scanner
Expand All @@ -106,4 +105,3 @@ jobs:
severity-threshold: HIGH
env:
DOCKLE_HOST: "unix:///var/run/docker.sock"

8 changes: 4 additions & 4 deletions .github/workflows/linters.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,17 +14,17 @@ jobs:
name: golangci-lint
runs-on: ubuntu-latest
steps:
- uses: actions/setup-go@v4
- uses: actions/setup-go@v5
with:
go-version: "1.21"
cache: false
- name: Checkout the code
uses: actions/checkout@v3.2.0
uses: actions/checkout@v4
- name: Vendor packages
run: |
go mod vendor
- name: golangci-lint
uses: golangci/golangci-lint-action@v3
uses: golangci/golangci-lint-action@v4
with:
version: v1.53
version: latest
skip-cache: true
16 changes: 15 additions & 1 deletion cmd/podmon/Dockerfile → Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,27 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
ARG GOIMAGE
ARG BASEIMAGE

# Build the module binary
FROM $GOIMAGE as builder

WORKDIR /workspace
COPY . .

# Build the binary
RUN GOOS=linux CGO_ENABLED=0 go build -o podmon ./cmd/podmon/

# Stage to build the module image
FROM $BASEIMAGE AS final
LABEL vendor="Dell Inc." \
name="csm-resiliency" \
summary="Dell Container Storage Modules (CSM) for Resiliency" \
description="Makes Kubernetes applications, including those that utilize persistent storage, more resilient to various failures" \
version="2.0.0" \
license="Apache-2.0"
COPY podmon /podmon

COPY --from=builder /workspace/podmon /

ENTRYPOINT [ "/podmon" ]
30 changes: 28 additions & 2 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,37 @@
# See the License for the specific language governing permissions and
# limitations under the License.
# Includes the following generated file to get semantic version information
all:
(cd cmd/podmon; make clean build-base-image build podman push)

MAJOR=0
MINOR=0
PATCH=54
VERSION?="v$(MAJOR).$(MINOR).$(PATCH)"
REGISTRY?="${REGISTRY_HOST}:${REGISTRY_PORT}/podmon"
BASEIMAGE?="resiliency-ubimicro:latest"

all: clean podman push

check:
@scripts/check.sh ./internal/monitor ./internal/k8sapi ./internal/csiapi ./internal/criapi ./cmd/podmon

unit-test:
(cd cmd/podmon; make unit-test)

clean:
go clean ./...

build:
GOOS=linux CGO_ENABLED=0 go build -o podmon ./cmd/podmon/

build-base-image: download-csm-common
$(eval include csm-common.mk)
sh ./scripts/buildubimicro.sh $(DEFAULT_BASEIMAGE)

podman: build-base-image
podman build --no-cache -t "$(REGISTRY):$(VERSION)" --build-arg GOIMAGE=$(DEFAULT_GOIMAGE) --build-arg BASEIMAGE=$(BASEIMAGE) -f ./Dockerfile --label commit=$(shell git log --max-count 1 --format="%H") .

push:
podman push "$(REGISTRY):$(VERSION)"

download-csm-common:
curl -O -L https://raw.githubusercontent.com/dell/csm/main/config/csm-common.mk
16 changes: 9 additions & 7 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
-->

# Dell Container Storage Modules (CSM) for Resiliency

[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-v2.0%20adopted-ff69b4.svg)](https://github.com/dell/csm/blob/main/docs/CODE_OF_CONDUCT.md)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://www.apache.org/licenses/LICENSE-2.0)
[![Podmam Pulls](https://img.shields.io/docker/pulls/dellemc/podmon)](https://hub.docker.com/r/dellemc/podmon)
Expand Down Expand Up @@ -45,30 +46,31 @@ For documentation, please visit [Container Storage Modules documentation](https:

If you wish to clone and build CSM for Resiliency, a Linux host is required with the following installed:

| Component | Version | Additional Information |
| --------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| Podman | v4.4.1+ | [Podman installation](https://podman.io/docs/installation) |
| Component | Version | Additional Information |
| --------------- | --------- | ---------------------------------------------------------------------- |
| Podman | v4.4.1+ | [Podman installation](https://podman.io/docs/installation) |
| Buildah | v1.29.1+ | [Buildah installation](https://www.redhat.com/sysadmin/getting-started-buildah) |
| Golang | v1.18+ | [Golang installation](https://github.com/travis-ci/gimme) |
| git | latest | [Git installation](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) |
| Golang | v1.21+ | [Golang installation](https://go.dev/dl/) |
| git | latest | [Git installation](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) |

Once all prerequisites are on the Linux host, follow the steps below to clone, build and deploy CSM for Resiliency:

1. Clone the repository: `git clone https://github.com/dell/karavi-resiliency.git`
2. Define and export the following environment variables to point to your Podman registry:

```
```sh
export REGISTRY_HOST=<registry host>
export REGISTRY_PORT=<registry port>
export VERSION=<version>
```

3. At the root of the source tree, run the following to build and deploy: `make`

## Testing CSM for Resiliency

From the root directory where the repo was cloned, the unit tests can be executed as follows:

```
```sh
make unit-test
```

Expand Down
25 changes: 0 additions & 25 deletions cmd/podmon/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,35 +12,10 @@
# See the License for the specific language governing permissions and
# limitations under the License.
# Includes the following generated file to get semantic version information
MAJOR=0
Copy link
Collaborator

@alikdell alikdell Feb 20, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reason for this approach is to avoid building other folders that are not part of image build. We should keep these here, and remove from top Makefile, and top Makefile should have only "all" target that cd into cmd/podmon and build the actual image

Copy link
Contributor Author

@chimanjain chimanjain Feb 21, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reason for this approach was because the go.mod and go.sum is present in parent directory and we can't cd into the podmon directory, otherwise the go.mod and go.sum file will not be copied into the first stage of dockerfile and there is a build failure. Previously we were building the binary in our local machine, so they were present, and the binary was being created successfully. And in the second stage we are only copying the binary and no other files. This is the right approach.

MINOR=0
PATCH=54
VERSION?="v$(MAJOR).$(MINOR).$(PATCH)"
REGISTRY?="${REGISTRY_HOST}:${REGISTRY_PORT}/podmon"
BASEIMAGE?="resiliency-ubimicro:latest"

clean:
go clean ./...

build:
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -ldflags '-w' ./...

build-base-image: download-csm-common
$(eval include csm-common.mk)
sh ../../scripts/buildubimicro.sh $(DEFAULT_BASEIMAGE)

podman:
podman build --no-cache -t "$(REGISTRY):$(VERSION)" --build-arg BASEIMAGE=$(BASEIMAGE) --label commit=$(shell git log --max-count 1 --format="%H") .

push:
podman push "$(REGISTRY):$(VERSION)"

unit-test:
go test -race -v -coverprofile=c.out ./...

godog:
go clean -cache
go test -v -coverprofile=c.out -test.run TestMain ./...

download-csm-common:
curl -O -L https://raw.githubusercontent.com/dell/csm/main/config/csm-common.mk
Loading