Building Amazon Linux RPMs and a Windows executable #4220
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Commenting out GOPATH adjustment block Uncomments GOPATH adjustment, adds symlink
Remove trailing space in original GOPATH if/then function Update pr-build.yml to modify go path logic Added an echo to check the github username. Removed the If/then statement that was creating the codebuild failures. GO path: remove if/then statement
This commit adds forked username to pr-build-ubuntu and echos go version Comment out golang dependency in ecs-agent.spec
Replace 2 with 2023 for AL name/path Correct AMZN_LINUX_2023_RPM path (remove 'int')
Add Windows Make target and comment out 'generic' build commands Fix indentation in amzn2/2023 buildspec files Add /out/ to Windows path in pr-build.yml Add line to echo the contents of expected artifact path Uncomment pr-build/agent.spec blocks, simplify sudo dnf update Add an 'automatic yes' to the version update pr-build-amzn2023.yml Edit file artifact variable name in amzn23 spec file
Removed troubleshooting lines that echo patath Comment out custom Go path
Minor edit to renaming in amzn2 spec, and windows artifact path
Edit syntax for goroot setting in amzn2 buildspec Edit gorspath in amzn2 buildspec, move .exe artifact in pr-build Remove the move command for .exe, reorder go path commands AL2 Copy .exe to codebuild dir, remove go path for AL2 arm build Remove path redirecting for .exe file, add list command for /out/ dir
Retry moving .exe to codebuild dir in pr-build Edit path adjustment for .exe~~ in pr-build Cleaning up pr-build.yml
Minor cleanup to minimize changes to files
Add go version logic form pr-build to amzn2 buildspec Remove default codebuild goroot for amzn2 buildspec
Ad variables and renaming logic to handlle AL2/23 in pr-build-amzn2 Add an echo to find where GO_VERSION is being retreived Remove else statement to check the if statement installs correct go version Add logic to retreive AL2023 version Remove quotes for variable definitions Move go and AL23 variables to install phase Remove qtrailing quotation mark for go variable
Edit AL23 package manager update command Add CB image variables for go installation function Define variables within build image detecting function Redefine RPM variable to be flexible enough to only include one in the artifact file section
Changing github username back to aws in preparation for PR
Change name of AL target to match Makefile
Restore order of make targets in pr-build.yml
|
The commit history on this PR #4220 was rather long. I used |
sparrc
previously approved these changes
Jul 1, 2024
singholt
reviewed
Jul 1, 2024
singholt
approved these changes
Jul 2, 2024
sparrc
approved these changes
Jul 2, 2024
Ephylouise
added a commit
to Ephylouise/amazon-ecs-agent
that referenced
this pull request
Jul 2, 2024
* Remove goenv installed version of go for AL buildspecs * Add logic to retrieve AL2023 version * Install go based on AL23 version update * Use the os-release file to detect Amazon Linux version * Add AL23 ref doc and install specific go version
Ephylouise
added a commit
to Ephylouise/amazon-ecs-agent
that referenced
this pull request
Jul 2, 2024
This reverts commit 531612a.
Merged
jiuchoe4
pushed a commit
to saurabhc123/amazon-ecs-agent
that referenced
this pull request
Jul 12, 2024
* Remove goenv installed version of go for AL buildspecs * Add logic to retrieve AL2023 version * Install go based on AL23 version update * Use the os-release file to detect Amazon Linux version * Add AL23 ref doc and install specific go version
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
The following changes were made to ensure that the project can build all necessary artifacts before deprecating a legacy component. It specifically modifies the buildspecs to include build capabilities for the full ECS-init RPM/DEB package as well as a Windows executable file. This will ensure that all required images are created during the Agent build process.
The goal is to have the artifacts built from CodeBuild jobs, which run the project's buildspecs, to be used in testing. Providing updates to the buildspecs and minor adjustments to the Makefile will enable CloudFormation and CodeBuild configurations that produce the full package of images in the development artifacts bucket.
Implementation details
A new make target called amazon-linux-rpm-codebuild was created. It uses similar logic as the amazon-linux-rpm-integrated target, but with minor changes to accommodate the CodeBuild build process. To avoid altering the integrated target, which can be used in Koji builds, the new target was created. The new target is called in the build specifications along with the Windows executable file, which relies on the docker-release target.
A new file was added, called pr-build-amzn.yml to accommodate the dependencies of the Amazon Linux builds. Amazon Linux 2, for example uses package manager yum, while Amazon Linux 2023 uses package manager dnf which are used to install Go, as required versions do not come preinstalled. Combining the Amazon Linux 2023 script with the pr-build.yml was considered, but that would result in a complex artifact build script that would be difficult to maintain and debug if issues arise. To keep the code clean and decoupled, the Amazon Linux RPMs (AL2 amd, AL2 arm, AL23 amd, AL23 arm) are all addressed in the new file.
Files changed:
Building the Amazon Linux RPMs will require 4 new CodeBuild (CB) projects, one for each Amazon Linux variant and architecture combination. This is due to the build process used to create the artifacts stored in the S3 bucket, as the Amazon Linux builds require separate image configurations.
Testing
Testing was performed in a separate AWS account using CodeBuild projects and webhooks to validate the modifications. The existing Agent CodeBuild jobs and the 4 new proposed projects were replicated in the separate account. The result was successful, with the following new artifacts and associated build logs landing in the expected S3 bucket:
Licensing
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.