Generate staging buildspec from Makefiles

[jaxesn]

Issue #, if available:

Description of changes:

The main purpose of this PR is to provide explicit dependency declaration to support auto generating the staging-bundle.yml.

The staging build, which uses that bundle, is a batch codebuild which builds "everything" in the build-tooling repo. The purpose of this is to have a process where everything is built from scratch, which is different than our dev build/release process which just builds the projects which changed.

In some cases we have the build triggers, in our dev build process, setup where if dependent projects change, it will also trigger the project which depend on it, but I suspect this is in complete. For the most this works out to be ok because we only have one version of the projects in this repo and dependencies are pulled using the GIT_TAG from repo. That means that if say distribution changes to a new version and harbor kicks off before the distribution job publishes the new tarball, with the new git_tag in the name, it will wait_for_tar to try and ensure it pulls the correct version. In the worst case this wait will timeout and the harbor build will fail, requiring a manual retrigger which would then use the newest version of the distribution dep.

The staging build/release building everything, using the batch build yml to model the dependencies, is to validate that everything is using the correct version and building in the proper order. Currently the staging-build.yaml has been manually created and maintain and has gotten out of date, for example there is no distribution dependency on the harbor build as their should be. The reason this ends up not being an issue is because the staging build builds to and pulls from the same folder/image tags as the dev build/release. I want to explore changing this to ensure the staging build is running from a "clean" env.

Before introducing the staging-build.yaml we used to use https://github.com/aws/eks-anywhere-build-tooling/blob/main/Makefile#L8 to model these dependencies since we would build everything from one code build job, use make release from the main Makefile. We do not do this anymore which makes this array of projects largely unnecessary and out of date.

We also had a psuedo-dependency model using the FETCH_BINARIES helper in makefiles which really just a way to standardize downloading the tars of other projects, there are really just a few examples of this across the repo.

The first commit in this PR changes FETCH_BINARIES into PROJECT_DEPENDENCIES with the main purpose of being able to use this information to regenerate the staging-bundle.yml ensuring it is always up to date and there is only one real source of truth for which projects depend on other projects.

This has been removed for now, but I may bring it back in another PR The second commit is more of a POC in how with deps modeled in the makefile structure, we could introduce an additional target to make it easier to build all the dependencies project locally if that is desired.

re: #801

/hold

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[jaxesn]

/unhold

[jaxesn]

/hold

Adding presubmit to verify this stays up to date

[ewollesen]

Shoot, these comments have been sitting here for a while, sorry.

[abhay-krishna]

/retest

[jaxesn]

/unhold

Added the presubmit

[eks-distro-bot]

@jaxesn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
tinkerbell-helm-presubmit	7e5c2be	link	true	/test tinkerbell-helm-presubmit

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[abhay-krishna]

/retest

[jaxesn]

/approve

[abhay-krishna]

/lgtm
/approve

[eks-distro-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: abhay-krishna, jaxesn

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [abhay-krishna,jaxesn]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment