This contains the configuration for the Cloud Foundry Buildpacks team Concourse deployment.
- dependency-builds: build binaries for Cloud Foundry buildpacks
- buildpacks: test and release all of the buildpacks
- brats: run BRATS against the master branch of buildpacks
- buildpack-verification: generate static site for buildpack verification
- buildpacks-ci: testing tasks for correct usage, rebuild CI docker images
- cf-release: deployment of latest buildpacks to cf-release develop
Jobs and tasks in the buildpacks-ci repository store state in public-buildpacks-ci-robots. See repository README for details.
./bin/update-pipelinesfly intercept -j $JOB_NAME -t task -n $TASK_NAMEfly intercept -c $RESOURCE_NAME rm -rf /tmp/git-resource-repo-cache-
Check out the
binary-buildsbranch -
Edit the YAML file appropriate for the build (e.g.
ruby-builds.yml) -
Find the version number and package SHA256 of the new binary. For many binaries, the project website provides the SHA256 along with the release (for example, jruby.org/download provides the SHA256 along with each JRuby release). For others (such as Godep), you download the .tar.gz file and run
shasum -a 256 <tar_file>to obtain the SHA256. -
Add any number of versions and their checksums to the array, e.g.
ruby: - version: 2.2.2 sha256: 5ffc0f317e429e6b29d4a98ac521c3ce65481bfd22a8cf845fa02a7b113d9b44
-
git commit -am 'Build ruby 2.2.2' && git push
Build should automatically kick off at
https://buildpacks.ci.cf-app.com/pipelines/binary-builder and silently
upload a binary to the buildpacks.cloudfoundry.org bucket under
dependencies/,
e.g. https://buildpacks.cloudfoundry.org.s3.amazonaws.com/dependencies/ruby/ruby-2.2.2-linux-x64.tgz
Note that the array is a stack, which will be emptied as the build succeeds in packaging successive versions.
When you want to change how a binary gets built, there are two places you may need to make changes. All binaries are built by the dependency-builds pipeline, and you may need to change the task that builds them. For many binaries, the dependency-builds pipeline runs recipes from the binary-builder repo; for those binaries, you will usually need to change the recipe rather than the concourse task.
For the list of currently supported binaries, check out our dependency-builds pipeline.
The concourse task that orchestrates the building is buildpacks-ci/tasks/build-binary-new/builder.rb; many of the recipes are in binary-builder.
To test these changes locally, you can execute the concourse task for it, but point to local changes. For instance:
$ cd buildpacks-ci
$ STACK=cflinuxfs2 fly -t buildpacks e -c tasks/build-binary-new/build.yml -j dependency-builds/build-r-3.4.X -i buildpacks-ci=.
For binaries that use recipes in binary-builder, you can also test in Docker. For instance:
$ docker run -w /binary-builder -v `pwd`:/binary-builder -it cloudfoundry/cflinuxfs2:ruby-2.2.4 ./bin/binary-builder --name=ruby --version=2.2.3 --md5=150a5efc5f5d8a8011f30aa2594a7654
$ ls
ruby-2.2.3-linux-x64.tgz
buildpacks-ci pipelines and tasks refer to many other repositories. These repos are where the buildpack team and others develop buildpacks and related artifacts.
Each officially-supported buildpack has a develop and a master branch.
Active development happens on develop. Despite our best efforts, develop will sometimes be unstable and is not production-ready.
Our release branch is master. This is stable and only updated with new buildpack releases.
- binary-buildpack
- go-buildpack
- nodejs-buildpack
- php-buildpack
- python-buildpack
- ruby-buildpack
- dotnet-core-buildpack
- staticfile-buildpack
- buildpack-packager Builds cached and uncached buildpacks
- machete Buildpack integration testing framework.
- compile-extensions Suite of utility scripts used in buildpacks at runtime
- libbuildpack Library used for writing buildpacks in Golang
- binary-builder Builds binaries against specified rootfs
- brats Buildpack Runtime Acceptance Test Suite, a collection of smoke tests
BOSH releases are used in the assembly of cf-release.
- go-buildpack-release
- ruby-buildpack-release
- python-buildpack-release
- php-buildpack-release
- nodejs-buildpack-release
- staticfile-buildpack-release
- binary-buildpack-release
- java-offline-buildpack-release
- java-buildpack-release
- dotnet-core-buildpack-release
These buildpacks are possible candidates for promotion, or experimental architecture explorations.
- concourse-filter Redacts credentials from Concourse logs
- new_version_resource Concourse resource to track dependency versions by scraping webpages
Some repositories are private for historical or security reasons. We list them for completeness.
- deployments-buildpacks See repository README.
- buildpacks-ci-robots See repository README.