- Overview
- Definitions
- Schemas reference
- Arcade's templates
The Azure DevOps schema originally defined a schema which included phase and queue objects. Azure DevOps modified their schema and replaced the phase object with the job object, and the queue with a pool. This change is not a direct one-to-one change as "queue" was a member of "phase" but now "pool" is not a member of "job". Additionally, a "matrix" is no longer a part of the "queue/pool" object, it is now a member of the "job" (defined as a strategy).
Phase schema - The Azure pipeline schema originally included phase and queue objects. That schema which refers to a "phase" object is called the "phase schema".
Job schema - Azure DevOps replaced the "phase" object with a "job" object and changed the members of that object. Authoring yaml which refer to a job object instead of a phase object is referred to as the "job schema". "Job" and "phase" are not interchangeable objects, they are mutually exclusive.
-
Using Arcade's templates allows you to write "cleaner" pipeline files by conditionally adding tasks depending on the context your build is running in. You can use the same yaml file for both PR and official builds. ie, the templates will add the "install Microbuild plugin" step if you are running in an official build context.
-
Arcade's templates enable sending telemetry so that we can gather information about build reliability and we can use that information to target what areas should be improved to help our customers be impacted by infrastructure less often.
-
Arcade's templates simplify using Helix
-
Arcade's templates simplify enabling dependency flow
-
Azure DevOps no longer supports the phase schema
-
BYOC (scalable agent pools) is not supported by the phase schema
job.yml is the base.yml replacement which uses the "job" schema instead of the "phase" schema.
See the job template.
Example:
In the main pipeline you would reference the template like this...
jobs:
- template: /eng/common/templates/job/job.yml
parameters:
enableMicrobuild: true
enablePublishBuildArtifacts: true
name: Windows_NT_Job
pool: dotnet-external-temp
steps:
- script: echo Hello World!
- template: /eng/common/templates/job/job.yml
parameters:
enableMicrobuild: true
enablePublishBuildArtifacts: true
name: Linux_Job
pool: dnceng-linux-external-temp
steps:
- script: echo Hello World!
Additional example of the phase to job schema transition
The "jobs" template wraps the "job" template, providing all of the functionality of the "job" template, but additionally adding a dependent job; the publish-build-assets job. Previously, repos participating in Arcade's dependency flow model had to explicitly add this job to their yml file and define its dependencies. If you're referencing the jobs.yml
template, you can just specify enablePublishBuildAssets as "true" and that job will be included in your build.
If you are participating in dependency flow (publishing assets), it is recommended that you use the "jobs" template.
See the jobs template.
Example:
In the main pipeline you would reference the template like this...
jobs:
- template: /eng/common/templates/jobs/jobs.yml
parameters:
enableMicrobuild: true
enablePublishBuildArtifacts: true
jobs:
- job: Windows_NT_Job
pool: dotnet-external-temp
steps:
- script: echo Hello World!
- job: Linux_Job
pool: dnceng-linux-external-temp
steps:
- script: echo Hello World!
Additional example of the phase to jobs schema transition
-
The "job" template makes use of variable array syntax. The array syntax is the only way to reference "variable groups" in yaml and so, that is the supported way to list variables in yaml.
Sample array syntax:
variables: # Create a "foo" variable - name: foo value: bar # Reference a variable group - group: myvargroup
If your variables are defined using the object syntax, they will not work with
job.yml
/jobs.yml
.Sample object syntax (not valid when referencing
job.yml
orjobs.yml
):# will not evaluate properly if you're using the job.yml or jobs.yml templates variables: foo: bar
-
There are a couple of parameters which
job.yml
supports thatbase.yml
did not.-
enablePublishBuildArtifacts - Adds the "PublishBuildArtifacts" task to your steps.
-
enablePublishTestResults - Adds the "PublishTestResults" task to your steps.
-
helixRepo - defines your source repo and is used to automatically determine the
_HelixSource
variable -
helixType - define the helix telemetry type, ie. build/product/
-
Transition example:
Original yaml referencing eng\common\templates\phases\base.yml
.
azure-pipelines.yml
variables:
Build.Repository.Clean: true
_HelixType: build/product/
_HelixSource: pr/dotnet/arcade-minimalci-sample/$(Build.SourceBranch)
trigger:
- master
pr:
- master
phases:
- template: /eng/common/templates/phases/base.yml
parameters:
name: Windows_NT
enableTelemetry: true
queue:
name: dotnet-external-temp
parallel: 99
matrix:
debug_configuration:
_BuildConfig: Debug
release_configuration:
_BuildConfig: Release
steps:
- script: eng\common\cibuild.cmd
-configuration $(_BuildConfig)
-prepareMachine
name: Build
displayName: Build
condition: succeeded()
- task: PublishBuildArtifacts@1
displayName: Publish Logs to VSTS
inputs:
PathtoPublish: '$(Build.SourcesDirectory)/artifacts/log/$(_BuildConfig)'
PublishLocation: Container
ArtifactName: $(Agent.Os)_$(Agent.JobName)
continueOnError: true
condition: always()
variables:
_HelixBuildConfig: $(_BuildConfig)
Updated yaml referencing eng\common\templates\job\job.yml
.
azure-pipelines.yml
trigger:
- master
pr:
- master
jobs:
- template: /eng/common/templates/job/job.yml
parameters:
name: Windows_NT
enableTelemetry: true
# Allow job.yml to add the "PublishBuildArtifacts" step - https://github.com/dotnet/arcade/blob/93b39c3209a5929662190c7e85b43b4f2a32bab1/eng/common/templates/job/job.yml#L38
enablePublishBuildArtifacts: true
helixRepo: dotnet/arcade-minimalci-sample
pool:
name: dotnet-external-temp
strategy:
matrix:
debug_configuration:
_BuildConfig: Debug
release_configuration:
_BuildConfig: Release
steps:
- checkout: self
clean: true
- script: eng\common\cibuild.cmd
-configuration $(_BuildConfig)
-prepareMachine
name: Build
displayName: Build
condition: succeeded()
Transition example:
Original yaml referencing eng\common\templates\phases\base.yml
.
azure-pipelines.yml
variables:
Build.Repository.Clean: true
_HelixType: build/product/
_HelixSource: pr/dotnet/arcade-minimalci-sample/$(Build.SourceBranch)
trigger:
- master
pr:
- master
phases:
- template: /eng/common/templates/phases/base.yml
parameters:
name: Windows_NT
enableTelemetry: true
queue:
name: dotnet-external-temp
parallel: 99
matrix:
debug_configuration:
_BuildConfig: Debug
release_configuration:
_BuildConfig: Release
steps:
- script: eng\common\cibuild.cmd
-configuration $(_BuildConfig)
-prepareMachine
name: Build
displayName: Build
condition: succeeded()
- task: PublishBuildArtifacts@1
displayName: Publish Logs to VSTS
inputs:
PathtoPublish: '$(Build.SourcesDirectory)/artifacts/log/$(_BuildConfig)'
PublishLocation: Container
ArtifactName: $(Agent.Os)_$(Agent.JobName)
continueOnError: true
condition: always()
variables:
_HelixBuildConfig: $(_BuildConfig)
Updated yaml referencing eng\common\templates\jobs\jobs.yml
.
azure-pipelines.yml
trigger:
- master
pr:
- master
jobs:
- template: /eng/common/templates/jobs/jobs.yml
parameters:
enableTelemetry: true
enablePublishBuildArtifacts: true
# Add a dependent job which publishes build assets for dependency flow
enablePublishBuildAssets: true
helixRepo: dotnet/arcade-minimalci-sample
# define jobs in the jobs object
jobs:
- job: Windows_NT
pool:
name: dotnet-external-temp
strategy:
matrix:
debug_configuration:
_BuildConfig: Debug
release_configuration:
_BuildConfig: Release
steps:
- checkout: self
clean: true
- script: eng\common\cibuild.cmd
-configuration $(_BuildConfig)
-prepareMachine
name: Build
displayName: Build
condition: succeeded()
Notes:
-
The
jobs.yml
template, unlikejob.yml
, does not have support for handling explicit templates.Example:
jobs: - template: /eng/common/templates/jobs/jobs.yml parameters: jobs: - job: Windows_NT steps: - script: echo Hello world! # This won't work, you can't pass a template as a job to jobs.yml - template: sayhi.yml
If you need to reference a template, then you should use
job.yml
to wrap each individual job rather than usingjobs.yml
to wrap all jobs. -
See https://github.com/dotnet/arcade for an example of a repo that uses the "jobs" template.
The phase schema is no longer supported or documented. Including the schema format here for reference.
- phase: string # name
displayName: string
dependsOn: string | [ string ]
condition: string
continueOnError: true | false
queue: string | queue
server: true | server
variables: { string: string } | [ variable ]
steps: [ script | bash | powershell | checkout | task | templateReference ]
name: string
demands: string | [ string ] # Supported by private pools
timeoutInMinutes: number
cancelTimeoutInMinutes: number
parallel: number
matrix: { string: { string: string } }
The job schema has replaced the phase schema and is publicly documented