To build an OpenShift release you run the hack/build-release.sh
script on a system with Docker, which
will create a build environment image and then execute a cross platform Go build within it. The build
output will be copied to _output/releases
as a set of tars containing each version. It will also build
the openshift/origin-base
image which is the common parent image for all OpenShift Docker images.
$ hack/build-release.sh
Once the release has been built the official Docker images can be generated with hack/build-images.sh
.
The resulting images can then be pushed to a Docker registry.
$ hack/build-images.sh
Note: To build the base and release images, run:
$ hack/build-base-images.sh
Once a release has been created, it can be pushed:
$ hack/push-release.sh
To cut an official tag release, we generally use the images built by ci.openshift.redhat.com under the openshift3_ami job.
- Create a new git tag
git tag v0.X.X -a -m "v0.X.X" HEAD
- Push the tag to GitHub
git push origin --tags
whereorigin
isgit.luolix.top/openshift/origin.git
- Run the "openshift3_ami" job
- Once the images are pushed to the repository, run
OS_PUSH_TAG="v0.X.X" hack/push-release.sh
. Your tag must match the Git tag. - Upload the binary artifacts generated by that build to GitHub release page
- Send an email to the dev list, including the important changes prior to the release.
We generally cut a release before disruptive changes land.
OpenShift uses three levels of testing - unit tests, integration test, and end-to-end tests (much like Kubernetes).
Unit tests follow standard Go conventions and are intended to test the behavior and output of a single package in isolation. All code is expected to be easily testable with mock interfaces and stubs, and when they are not it usually means that there's a missing interface or abstraction in the code. A unit test should focus on testing that branches and error conditions are properly returned and that the interface and code flows work as described. Unit tests can depend on other packages but should not depend on other components (an API test should not be writing to etcd).
The unit tests for an entire package should not take more than 0.5s to run, and if they do, are probably not really unit tests or need to be rewritten to avoid sleeps or pauses. Coverage on a unit test should be above 70% unless the units are a special case.
See pkg/template/generator
for examples of unit tests. Unit tests should follow Go conventions.
Run the unit tests with:
$ hack/test-go.sh
or an individual package unit test with:
$ hack/test-go.sh pkg/build
To run only a certain regex of tests in a package, use:
$ hack/test-go.sh pkg/build -test.run=SynchronizeBuildRunning
To get verbose output add -v
to the end:
$ hack/test-go.sh pkg/build -test.run=SynchronizeBuildRunning -v
To run all tests with verbose output:
$ hack/test-go.sh "" -v
To turn off or change the coverage mode, which is -cover -covermode=atomic
by default, use:
$ KUBE_COVER="" hack/test-go.sh
To run tests without the go race detector, which is on by default, use:
$ KUBE_RACE="" hack/test-go.sh
A line coverage report is run by default when testing a single package. To create a coverage report for all packages:
$ OUTPUT_COVERAGE=true hack/test-go.sh pkg/build
Integration tests cover multiple components acting together (generally, 2 or 3). These tests should focus on ensuring that naturally related components work correctly. They should not be extensively testing branches or error conditions inside packages (that's what unit tests do), but they should validate that important success and error paths work across layers (especially when errors are being converted from lower level errors). Integration tests should not be testing details of the intercomponent connections - API tests should not test that the JSON serialized to the wire is correctly converted back and forth (unit test responsibility), but they should test that those connections have the expected outcomes. The underlying goal of integration tests is to wire together the most important components in isolation. Integration tests should be as fast as possible in order to enable them to be run repeatedly during testing. Integration tests that take longer than 0.5s are probably trying to test too much together and should be reorganized into separate tests. Integration tests should generally be written so that they are starting from a clean slate, but if that involves costly setup those components should be tested in isolation.
We break integration tests into two categories, those that use Docker and those that do not. In general, high-level components that depend on the behavior of code running inside a Docker container should have at least one or two integration tests that test all the way down to Docker, but those should be part of their own test suite. Testing the API and high level API functions should generally not depend on calling into Docker. They are denoted by special test tags and should be in their own files so we can selectively build them.
All integration tests are located under test/integration/*
. All integration tests must set the
integration
build tag at the top of their source file, and also declare whether they need etcd
with the !no-etcd
build tag and whether they need Docker with the !no-docker
build tag. For
special function sets please create subdirectories like test/integration/deployimages
.
Run the integration tests with:
$ hack/test-integration.sh
The script launches an instance of etcd and then invokes the integration tests. If you need to execute an individual test start etcd and then run:
$ hack/test-go.sh test/integration -tags 'integration no-docker' -test.run=TestBuildClient
There is a CLI integration test suite which covers general non-Docker functionality of the CLI tool working against the API. Run it with:
$ hack/test-cmd.sh
The final test category is end to end tests (e2e) which should verify a long set of flows in the product as a user would see them. Two e2e tests should not overlap more than 10% of function, and are not intended to test error conditions in detail. The project examples should be driven by e2e tests. e2e tests can also test external components working together.
End to end tests should be Go tests with the build tag e2e
in the test/e2e
directory.
Run the end to end tests with:
$ hack/test-end-to-end.sh
OpenShift and Kubernetes use Godep for dependency management.
Godep allows versions of dependent packages to be locked at a specific commit by vendoring them
(checking a copy of them into Godeps/_workspace/
). This means that everything you need for
OpenShift is checked into this repository, and the hack/config-go.sh
script will set your GOPATH
appropriately. To install godep
locally run:
$ go get github.com/tools/godep
If you are not updating packages you should not need godep installed.
There are a few steps involved in rebasing Origin to a new version of Kubernetes. We need to make sure
that not only the Kubernetes packages were updated correctly into Godeps
, but also that all tests are
still running without errors and code changes, refactorings or the inclusion/removal of attributes
were properly reflected in the Origin codebase.
Before you begin, make sure you have both openshift/origin and GoogleCloudPlatform/kubernetes in your $GOPATH:
$ go get github.com/openshift/origin
$ go get github.com/GoogleCloudPlatform/kubernetes
Check out the version of Kubernetes you want to rebase as a branch or tag named stable_proposed
in
GoogleCloudPlatform/kubernetes. For example,
if you are going to rebase the latest master
of Kubernetes:
$ cd $GOPATH/src/github.com/GoogleCloudPlatform/kubernetes
$ git checkout master
$ git pull
$ git checkout -b stable_proposed
If all requirements described in Preparation were correctly attended, you should not have any trouble with rebasing the Kubernetes code using the script that automates this process.
$ cd $GOPATH/src/github.com/openshift/origin
$ hack/rebase-kube.sh
Read over the changes with git status
and make sure it looks reasonable. Check specially the
Godeps/Godeps.json
file to make sure no dependency is unintentionally missing.
Commit using the message bump(github.com/GoogleCloudPlatform/kubernetes):<commit SHA>
, where
<commit SHA>
is the commit id for the Kubernetes version we are including in our Godeps. It can be
found in our Godeps/Godeps.json
in the declaration of any Kubernetes package.
If for any reason you had trouble rebasing using the script, you may need to to do it manually.
After following all requirements described in the Preparation topic, you will need to run
godep restore
from both the Origin and the Kubernetes directories and then godep save ./...
from the Origin directory. Follow these steps:
$ cd $GOPATH/src/github.com/openshift/origin
make clean ; godep restore
will restore the package versions specified in theGodeps/Godeps.json
of Origin to your GOPATH.$ cd $GOPATH/src/github.com/GoogleCloudPlatform/kubernetes
$ git checkout stable_proposed
will checkout the desired version of Kubernetes as branched in Preparation.$ godep restore
will restore the package versions specified in theGodeps/Godeps.json
of Kubernetes to your GOPATH.$ cd $GOPATH/src/github.com/openshift/origin
.$ make clean ; godep save ./...
will save a list of the checked-out dependencies to the fileGodeps/Godeps.json
, and copy their source code intoGodeps/_workspace
.- If in the previous step godep complaints about the checked out revision of a package being different
than the wanted revision, this probably means there are new packages in Kubernetes that we need to add.
Do a
godep save <pkgname>
with the package specified by the error message and then$ godep save ./...
again. - Read over the changes with
git status
and make sure it looks reasonable. Check specially theGodeps/Godeps.json
file to make sure no dependency is unintentionally missing. The whole Godeps directory will be added to version control, including_workspace
. - Commit using the message
bump(github.com/GoogleCloudPlatform/kubernetes):<commit SHA>
, where<commit SHA>
is the commit id for the Kubernetes version we are including in our Godeps. It can be found in ourGodeps/Godeps.json
in the declaration of any Kubernetes package.
If in the process of rebasing manually you found any corner case not attended by the hack/rebase-kube.sh
script, make sure you update it accordingly to help future rebases.
Eventually during the development cycle we introduce changes to dependencies right in the Origin
repository. This is not a largely recommended practice, but it's useful if we need something that,
for example, is in the Kubernetes repository but we are not doing a rebase yet. So, when doing the next
rebase, we need to make sure we get all these changes otherwise they will be overriden by godep save
.
- Check the
Godeps
directory commits history for commits tagged with the UPSTREAM keyword. We will need to cherry-pick all UPSTREAM commits since the last Kubernetes rebase (remember you can find the last rebase commit looking for a message likebump(github.com/GoogleCloudPlatform/kubernetes):...
). - For every commit tagged UPSTREAM, do
git cherry-pick <commit SHA>
. - Notice that eventually the cherry-pick will be empty. This probably means the given change were already merged in Kubernetes and we don't need to specifically add it to our Godeps. Nice!
- Read over the commit history and make sure you have every UPSTREAM commit since the last rebase (except only for the empty ones).
After making sure we have all the dependencies in place and up-to-date, we need to work in the Origin codebase to make sure the compilation is not broken, all tests pass and it's compliant with any refactorings, architectural changes or behavior changes introduced in Kubernetes. Make sure:
make clean ; hack/build-go.sh
compiles without errors and the standalone server starts correctly.hack/test-go.sh
runs without errors.hack/test-cmd.sh
runs without errors.hack/test-integration.sh
runs without errors.hack/test-end-to-end.sh
runs without errors.
See Building a Release above for setting up the environment for the test-end-to-end.sh tests.
It is helpful to look at the Kubernetes commit history to be aware of the major topics. Although it can potentially break or change any part of Origin, the most affected parts are usually:
- https://github.com/openshift/origin/blob/master/pkg/cmd/server/start.go
- https://github.com/openshift/origin/blob/master/pkg/cmd/server/kubernetes/master.go
- https://github.com/openshift/origin/blob/master/pkg/cmd/server/origin/master.go
- https://github.com/openshift/origin/blob/master/pkg/cmd/cli/cmd/factory.go
- https://github.com/openshift/origin/blob/master/pkg/cmd/cli/cli.go
- https://github.com/openshift/origin/blob/master/pkg/api/meta/multimapper.go
Place all your changes in a commit called "Refactor to match changes upstream".
A typical pull request for your Kubernetes rebase will contain:
- One commit for the Godeps bump (
bump(github.com/GoogleCloudPlatform/kubernetes):<commit SHA>
). - Zero, one or more cherry-picked commits tagged UPSTREAM.
- One commit "Refactor to match changes upstream".
To update to a new version of a dependency that's not already included in Kubernetes, checkout the
correct version in your GOPATH and then run godep save <pkgname>
. This should create a new
version of Godeps/Godeps.json
, and update Godeps/_workspace/src
. Create a commit that includes
both of these changes with message bump(<pkgname>): <pkgcommit>
.
If you run into difficulties running OpenShift, start by reading through the troubleshooting guide.