This guide explains how to set up your environment for development on AKS Engine.
We welcome contributions. This project has set up some guidelines in order to ensure that (a) code quality remains high, (b) the project remains consistent, and (c) contributions follow the open source legal requirements. Our intent is not to burden contributors, but to build elegant and high-quality open source code so that our users will benefit.
Make sure you have read and understood the main CONTRIBUTING guide:
https://github.com/Azure/aks-engine/blob/master/CONTRIBUTING.md
The easiest way to start hacking on AKS Engine is to use a Docker-based environment. If you already have Docker installed then you can get started with a few commands.
$ make dev
Or on Windows (ensure Docker is configured for Linux containers on Windows):
powershell ./makedev.ps1
This make target mounts the AKS Engine source directory as a volume into the Docker container, which means you can edit your source code in your favorite editor on your machine, while still being able to compile and test inside of the Docker container. This environment mirrors the environment used in the AKS Engine continuous integration (CI) system.
When make dev
completes, you will be left at a command prompt inside a Docker container.
Run the following commands to pull the latest dependencies and build the aks-engine
tool.
# set up the hack/tools directory for your platform
make -C hack/tools clean install
# install and download build dependencies
make bootstrap
# build the `aks-engine` binary
make build
The build process leaves the compiled aks-engine
binary in the bin
directory. Make sure everything completed successfully by running bin/aks-engine
without any arguments:
$ ./bin/aks-engine
Usage:
aks-engine [flags]
aks-engine [command]
Available Commands:
addpool Add a node pool to an existing AKS Engine-created Kubernetes cluster
completion Generates bash completion scripts
deploy Deploy an Azure Resource Manager template
generate Generate an Azure Resource Manager template
get-logs Collect logs and current cluster nodes configuration.
get-versions Display info about supported Kubernetes versions
help Help about any command
rotate-certs Rotate certificates on an existing AKS Engine-created Kubernetes cluster
scale Scale an existing AKS Engine-created Kubernetes cluster
update Update an existing AKS Engine-created VMSS node pool
upgrade Upgrade an existing AKS Engine-created Kubernetes cluster
version Print the version of aks-engine
Flags:
--debug enable verbose debug logs
-h, --help help for aks-engine
--show-default-model Dump the default API model to stdout
Use "aks-engine [command] --help" for more information about a command.
Here's a quick demo video showing the dev/build/test cycle with this setup.
If the above docker container conveniences don't work for your developer environment, below is per-platform guidance to help you set up your local dev environment manually to build an aks-engine
binary from source.
Building an aks-engine
binary from source has a few requirements for each of the platforms. Download and install the prerequisites for your platform: Windows, Linux, or Mac:
Setup steps:
-
Setup your go workspace. This guide assumes you are using
c:\Users\me\code\go
as your Go workspace:- Type WIN+R to open the run prompt
- Type
rundll32 sysdm.cpl,EditEnvironmentVariables
to open the system variables - Add
c:\go\bin
andc:\Users\me\code\go\bin
to your PATH variables - Click "new" and add new environment variable named
GOPATH
and set the value toc:\Users\me\code\go
-
Build aks-engine:
- Type Windows key-R to open the run prompt
- Type
cmd
to open a command prompt - Type
mkdir %GOPATH%
to create your gopath - Type
cd %GOPATH%
- Type
mkdir -p src\github.com\Azure
to create the gopath to aks-engine - Type
cd src\github.com\Azure
- Type
git clone https://github.com/Azure/aks-engine
to download aks-engine from GitHub - Type
cd aks-engine
- Type
make bootstrap
to get the supporting components - Type
make
to build the project - Run
.\bin\aks-engine.exe
to see the command line parameters
Setup steps:
- Open a command prompt to setup your gopath:
mkdir $HOME/go
- Edit
$HOME/.bash_profile
and add the following lines to setup your go path:export GOPATH=$HOME/go export PATH=$PATH:/usr/local/go/bin:$GOPATH/bin
source $HOME/.bash_profile
Build aks-engine:
- Type
mkdir -p $HOME/go/src/github.com/Azure
to create your gopath - Type
cd $_
to switch directories to that same path - Type
git clone https://github.com/Azure/aks-engine
to download aks-engine from GitHub - Type
cd aks-engine
to change to the source directory - Type
make bootstrap
to install supporting components - Type
make
to build the project - Type
./bin/aks-engine
to see the command line parameters
The code for the AKS Engine project is organized as follows:
- The individual programs are located in
cmd/
. Code inside ofcmd/
is not designed for library re-use. - Shared libraries are stored in
pkg/
. - The
tests/
directory contains a number of utility scripts. Most of these are used by the CI/CD pipeline. - The
docs/
folder is used for documentation and examples.
Go dependencies are managed with
Golang Dep and stored in the
vendor/
directory.
We use Git for our version control system. The master
branch is the
home of the current development candidate. Releases are tagged.
We accept changes to the code via GitHub Pull Requests (PRs). One workflow for doing this is as follows:
- Use
go get
to clone the aks-engine repository:go get github.com/Azure/aks-engine
- Fork that repository into your GitHub account
- Add your repository as a remote for
$GOPATH/github.com/Azure/aks-engine
- Create a new working branch (
git checkout -b feat/my-feature
) and do your work on that branch. - When you are ready for us to review, push your branch to GitHub, and then open a new pull request with us.
Third party dependencies reside locally inside the repository under the vendor/
directory. We use dep to enforce our dependency graph, declared in Gopkg.toml in the project root.
If you wish to introduce a new third party dependency into aks-engine
, please file an issue, and include the canonical VCS path (e.g., github.com/Azure/azure-sdk-for-go
) along with either the desired release string expression to depend on (e.g., ~8.1.0
), or the commit hash to pin to a static commit (e.g., 4cdb38c072b86bf795d2c81de50784d9fdd6eb77
). A project maintainer will then own the effort to update the codebase with that dependency, including relevant updates to Gopkg.toml
and vendor/
.
As a rule we want to distinguish dependency update PRs from feature/bug PRs; we may ask that feature/bug PRs which include updates to vendor/
and/or contain any other dependency-related overhead to be triaged into separate PRs that can be managed independently, pre-requisite dependency changes in one, and features/bugs in another. The objective of enforcing these distinctions is to help focus the PR review process, and to make manageable the difficult task of rationalizing a multitude of parallel PRs in flight, many of which which may carry hard-to-reconcile dependency side-effects when aggressively updated with a fresh dependency graph as part of the PR payload.
We follow the Go coding style standards very closely. Typically, running
go fmt
will make your code beautiful for you.
We also typically follow the conventions recommended by go lint
and
gometalinter
. Run make test-style
to test the style conformance.
Read more:
- Effective Go introduces formatting.
- The Go Wiki has a great article on formatting.
Unit tests may be run locally via make test
.
AKS Engine maintains its own E2E test implementation (see the test/e2e/
source directory) to validate Kubernetes on Azure functionality from AKS Engine source.
A make
target convenience is maintained to easily run these tests:
$ make test-kubernetes
In practice, running E2E tests locally requires lots of environmental context, in order to tell the E2E runner what kind of cluster configuration you want to test, which tests you may want to run or skip, what level of timeout tolerance to permit, and many other runtime-configurable options that express the exact test criteria you intend to validate. A real-world E2E invocation may look this this instead:
$ ORCHESTRATOR_RELEASE=1.22 CLUSTER_DEFINITION=examples/kubernetes.json SUBSCRIPTION_ID=$TEST_AZURE_SUB_ID CLIENT_ID=$TEST_AZURE_SP_ID CLIENT_SECRET=$TEST_AZURE_SP_PW TENANT_ID=$TEST_AZURE_TENANT_ID LOCATION=$TEST_AZURE_REGION CLEANUP_ON_EXIT=false make test-kubernetes
Thorough guidance around effectively running E2E tests to validate source code changes can be found here.
To debug AKS Engine code directly, use the Go extension for Visual Studio Code or use Delve at the command line.
To debug AKS Engine with VS Code, first ensure that you have the Go extension installed. Click the "Extensions" icon in the Activity Bar (on the far left), search for "go", then install the official Microsoft extension titled "Rich Go language support for Visual Studio Code."
Once installed, the Go extension will go get
several helper applications, including Delve for
debugging support. You can read more about VS Code integration with Delve
here.
Open the directory that you checked out the aks-engine
repo to in VS Code.
If you are writing tests (and if you are, we want to give you a hug!), you can debug them directly in Visual Studio code.
Set a breakpoint inside a test, then move your mouse pointer to the top of the function definition. To the right of "run test" appears a link saying "debug test": click it!
To debug changes to AKS Engine source during active development, the default Go debugging configuration in .vscode/launch.json
needs
to be edited. Open that file (or just click the gear-shaped "Open launch.json" icon if you have the
Debug panel open).
Here is an example launch.json
file that contains a configuration for debugging the
aks-engine generate
command using the examples/kubernetes.json
file for its cluster
configuration.
{
"version": "0.2.0",
"configurations": [
{
"name": "aks-engine generate",
"type": "go",
"request": "launch",
"mode": "debug",
"program": "${workspaceRoot}",
"env": {},
"args": [
"generate", "--api-model=${workspaceRoot}/examples/kubernetes.json",
"--set", "masterProfile.dnsPrefix=my-dns-prefix",
"--set", "linuxProfile.ssh.publicKeys[0].keyData=my-public-key-contents",
"--set", "servicePrincipalProfile.clientId=my-service-principal-client-id",
"--set", "servicePrincipalProfile.secret=my-service-principal-secret"
],
"showLog": true
}
]
}
Copy and paste the configuration and change the values in the --set
arguments to reference your
details. You can create multiple configurations in launch.json
to debug aks-engine upgrade
,
scale
, and other commands.
For a more detailed debugging configuration, check out the example launch.json
in this directory. It assumes you have $CLIENT_ID
, $CLIENT_SECRET
, and $AKSE_PUB_KEY
environment
variables set, and will prompt you for other command inputs when starting the debugger.
The .vscode/launch.json
file is ignored by git
, so your local version won't be overwritten when
you push or pull changes.
To start debugging, set a breakpoint in the code of interest. Then choose your configuration in the Debug panel and click the green "Start Debugging" arrow.
When the debugger hits your breakpoint, the variables and call stack will populate in the Debug panel. Now you can step through code and inspect memory at runtime using Visual Studio Code's standard debugging controls.
This just scratches the surface. Please read more about debugging with VS Code.
To debug aks-engine generate
from the command line:
dlv debug github.com/Azure/aks-engine -- generate ~/Documents/azure/kubernetes.json
To test an individual package or a single test:
dlv test github.com/Azure/aks-engine/pkg/engine
dlv test github.com/Azure/aks-engine/pkg/engine -- -test.run ^TestNetworkPolicyDefaults$
AKS Engine employs a Continuous Integration (CI) system that incorporates Azure DevOps, configured to interact with the AKS Engine GitHub project.
The following steps constitute the AKS Engine CI pipeline:
- Contributor opens a Pull Request (PR) against the AKS Engine project
- An AKS Engine team member comments on the PR to trigger an Azure DevOps job that
- applies the changes to the HEAD of the master branch
- runs unit tests and code coverage reports
- generates multiple ARM templates for different deployment scenarios
- simultaneously provisions the clusters based on generated templates in Azure
- The PR is code reviewed by the members of AKS Engine team
- Once the PR is approved and the end-to-end job has passed, the PR can now be merged into the master branch
- Once merged, another job is triggered to verify integrity of the master branch. This job is similar to the PR job.
To make it easier use AKS Engine source code as a library and to go get github.com/Azure/aks-engine
, some
generated Go code is committed to the repository. Your pull request may need to regenerate those
files before it will pass the required make ensure-generated
step.
Always run make build
before you submit a pull request to validate compilation and
generated code hygiene. Run make ensure-generated
yourself to validate that things check out. If there are
discrepencies, make ensure-generated
will output a brief error report.
- Changes under the
parts/
folder require thepkg/engine/templates_generated.go
file to be updated. - Changes under
pkg/i8n/translations
require thepkg/engine/translations_generated.go
file to be updated.