Skip to content

Commit

Permalink
Rewrite the entire Azure DevOps build system (#34984)
Browse files Browse the repository at this point in the history
This pull request rewrites the entire Azure DevOps build system.

The guiding principles behind this rewrite are:

- No pipeline definitions should contain steps (or tasks) directly.
- All jobs should be in template files.
- Any set of steps that is reused across multiple jobs must be in
  template files.
- All artifact names can be customized (via a property called
  `artifactStem` on all templates that produce or consume artifacts).
- No compilation happens outside of the "Build" phase, to consolidate
  the production and indexing of PDBs.
- All step and job templates are named with `step` or `job` _first_,
  which disambiguates them in the templates directory.
- Most jobs can be run on different `pool`s, so that we can put
  expensive jobs on expensive build agents and cheap jobs on cheap
  build agents. Some jobs handle pool selection on their own, however.

Our original build pipelines used the `VSBuild` task _all over the
place._ This resulted in PowerToys being built in myriad ways, different
for every pipeline. There was an attempt at standardization early on,
where `ci.yml` consumed jobs and steps templates... but when
`release.yml` was added, all of that went out the window.

It's the same story as Terminal (microsoft/terminal#15808).

The new pipelines are consistent and focus on a small, well-defined set
of jobs:

- `job-build-project`
    - This is the big one!
    - Takes a list of build configurations and platforms.
    - Produces an artifact named `build-PLATFORM-CONFIG` for the entire
      matrix of possibilities.
    - Builds all of the installers.
    - Optionally signs the output (all of the output).
    - Admittedly has a lot going on.
- `job-test-project`
    - Takes **one** build config and **one** platform.
    - Consumes `build-PLATFORM-CONFIG`
    - Selects its own pools (hardcoded) because it knows about
      architectures and must choose the right agent arch.
    - Runs tests (directly on the build agent).
- `job-publish-symbols-using-symbolrequestprod-api`
    - Consumes `**/*.pdb` from all prior build phases.
    - Uploads all PDBs in one artifact to Azure DevOps
    - Uses Microsoft's internal symbol publication REST API to submit
      stripped symbols to MSDL for public consumption.

Finally, this pull request has some additional benefits:

- Symbols are published to the private and public feeds at the same
  time, in the same step. They should be available in the public symbol
  server for public folks to debug against!
- We have all the underpinnings necessary to run tests on ARM64 build
  agents.
    - Right now, `ScreenResolutionUtility` is broken
    - I had to introduce a custom version of `UseDotNet` which would
      install the right architecture (🤦); see microsoft/azure-pipelines-tasks#20300.
- All dotnet and nuget versioning is consolidated into a small set of
  step templates.
- This will provide a great place for us to handle versioning changes
  later, since all versioning happens in one place.
  • Loading branch information
DHowett authored Sep 25, 2024
1 parent 4240a7c commit ad1f204
Show file tree
Hide file tree
Showing 25 changed files with 1,268 additions and 1,261 deletions.
7 changes: 7 additions & 0 deletions .github/actions/spell-check/expect.txt
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ AFX
AGGREGATABLE
AHybrid
AKV
akv
ALarger
ALLAPPS
ALLINPUT
Expand Down Expand Up @@ -728,6 +729,7 @@ ISettings
IShell
isocpp
iss
issecret
ISSEPARATOR
ITask
ith
Expand Down Expand Up @@ -946,6 +948,7 @@ mrw
msc
mscorlib
msdata
MSDL
msedge
MSGFLT
msiexec
Expand Down Expand Up @@ -1138,6 +1141,7 @@ pch
pchast
PCIDLIST
PCWSTR
pdbs
pdisp
pdo
pdto
Expand Down Expand Up @@ -1203,6 +1207,7 @@ ppv
prc
Prefixer
Preinstalled
prependpath
prevhost
previewer
PREVIEWHANDLERFRAMEINFO
Expand Down Expand Up @@ -1565,6 +1570,7 @@ SWC
SWFO
SWP
SWRESTORE
symbolrequestprod
SYMCACHE
SYMED
SYMOPT
Expand Down Expand Up @@ -1640,6 +1646,7 @@ TOUCHEVENTF
TOUCHINPUT
touchpad
tracelogging
trafficmanager
traies
transicc
TRAYMOUSEMESSAGE
Expand Down
41 changes: 0 additions & 41 deletions .pipelines/ci/caching.yml

This file was deleted.

46 changes: 0 additions & 46 deletions .pipelines/ci/ci.yml

This file was deleted.

44 changes: 0 additions & 44 deletions .pipelines/ci/templates/build-powertoys-ci.yml

This file was deleted.

20 changes: 0 additions & 20 deletions .pipelines/ci/templates/build-powertoys-installer.yml

This file was deleted.

38 changes: 0 additions & 38 deletions .pipelines/ci/templates/build-powertoys-precheck.yml

This file was deleted.

Loading

0 comments on commit ad1f204

Please sign in to comment.