Azure pipelines migration #1267

bergmeister · 2019-06-20T14:03:40Z

PR Summary

This is the minimum viable, lift and shift approach to have test running in Azure DevOps.
For better visibility in a PR I decided to create multiple build definitions instead of using multiple jobs in one build definition and another reason is that if one used multiple job in one build definition, one would not be able to see from which platform they came.
Good news: They pass on macOS but there are some test failures to be looked into for Ubuntu. I suggest to only validate against PS 5/6 on Windows and Mac and keep track of the Ubuntu failures until we have fixed them. Somehow there is also a bug in Azure DevOps where the task to upload test results it supposed to always run using succeededOrFailed but in practice it does not run if the previous task fails...

After this I suggest we only keep the PS v4 test in AppVeyor

@rjmholt Can I ask you please to look into the test failures from your compatibility module please? It seems that those tests pass on Ubuntu 16 but not Ubuntu 18, see related PR #1262 where I tried to fix the Ubuntu builds in AppVeyor. Also: I was not sure if we still need the calls to ./PSCompatibilityCollector/build.ps1 in the build yaml script?

PR Checklist

PR has a meaningful title
- Use the present tense and imperative mood when describing your changes
Summarized changes
Change is not breaking
Make sure all .cs, .ps1 and .psm1 files have the correct copyright header
Make sure you've added a new test if existing tests do not effectively test the code changed and/or updated documentation
This PR is ready to merge and is not Work in Progress.
- If the PR is work in progress, please add the prefix WIP: to the beginning of the title and remove the prefix when the PR is ready.

… into CI_NotAppVeyorSpecific

rjmholt · 2019-06-28T18:52:53Z

My current suspicion is that the Ubuntu 18.04 problems are caused by the LANG=C.UTF-8 locale change on that platform. I'll have a look into how dictionaries are built to make sure they're not taking the culture into account in any way.

rjmholt · 2019-06-28T22:09:19Z

Hmmmm just confirmed it wasn't that and haven't been able to reproduce yet on my ubuntu 18.04 VM

rjmholt · 2019-06-28T22:11:44Z

Also: I was not sure if we still need the calls to ./PSCompatibilityCollector/build.ps1 in the build yaml script?

We should certainly be building the module to make sure it builds in CI, but not sure if you mean that there's another call to that build script elsewhere that outmodes it?

… into AzurePipelinesMigration

… it.

# Conflicts: # Tests/Engine/CustomizedRule.tests.ps1

bergmeister · 2019-10-27T12:47:41Z

Close and re-open to see if things have changed and if the new Ubuntu failures on AppVeyor, happen here as well.

rjmholt · 2020-01-03T03:14:11Z

Hmmm, so it is reproducing in AzDO CI as well. After spending a day trying to reproduce this, I stopped trying when @JamesWTruher and I didn't see it in AzDO (only AppVeyor).

Since this seems to be causing an issue in AzDO now, I'll go back to trying to reproduce it. I didn't have much luck replicating the AzDO environment locally...

bergmeister · 2020-01-03T08:29:23Z

Maybe just ignoring the failures similar to the AppVeyor ones would be the best? Interestingly there were new AppVeyor AppVeyor failures just before xmas in AppVeyor but they have disappeared now

rjmholt · 2020-01-03T22:03:01Z

Frustrating. But yes, I don't think these failures should block anything, so we should ignore them for now.

rjmholt · 2020-01-03T22:03:15Z

I'll open a PR to ignore them in AzDO

bergmeister · 2020-01-03T22:09:47Z

I suggest to just add them in this PR here and use an environment variable that is only set in Az-Do

rjmholt · 2020-01-03T22:57:13Z

I suggest to just add them in this PR here and use an environment variable that is only set in Az-Do

Ok with you if I commit to this branch?

bergmeister · 2020-01-03T23:49:29Z

Yes, please

rjmholt · 2020-01-07T18:22:13Z

Getting this:

To https://github.com/bergmeister/PSScriptAnalyzer
 ! [remote rejected] AzurePipelinesIntegration -> AzurePipelinesIntegration (permission denied)
error: failed to push some refs to 'https://github.com/bergmeister/PSScriptAnalyzer'

Going to open a PR instead

rjmholt · 2020-01-07T18:23:56Z

See bergmeister#11

Exclude failing tests on AzDO/Ubuntu

bergmeister · 2020-01-07T18:55:52Z

Don't know why the push did not work, especially since I had the 'Allow edits from maintainers' checkbox ticked. Thanks for the PR of the PR.

bergmeister · 2020-01-07T19:06:05Z

Great, the Ubuntu build fails now because somehow after bootstrap the .net core sdk, it seems to lose it on the PATH env variable. Will merge bootstrap and build tasks into one then

…n Az-Do

… into AzurePipelinesMigration Merge using HEAD, which reverts the last commit... # Conflicts: # build.psm1

bergmeister · 2020-01-07T20:31:33Z

Yippee, it's finally green. Now, I have to do a self-review as well as some things might've changed since then.
For example I created a build definition for each type of hosted agent image as there was a dropdown (this allowed to report them indiviudally in GitHub). Now there is only the previously chosen option left in the dropdown, which means that we could not add a new build definition with a new image. Therefore I think it might be better to put the image into YAML as well now and use only 1 build definition. What do you think?

bergmeister · 2020-01-07T20:34:04Z

.azure-pipelines-ci/templates/test.yaml

+  inputs:
+    testRunner: NUnit
+    testResultsFiles: 'TestResults.xml'
+    condition: succeededOrFailed()


Something that should be said is that although I set it to always run, but when the build or test fails, this task does not run and therefore test results are not uploaded. This seems to be a bug with the Azure DevOps YAML though. We should raise something but I don't think that's a blocking issue

@JamesWTruher FYI

JamesWTruher

LGTM

bergmeister added 20 commits June 16, 2019 12:13

make ci scripts agnostic of CI provider

1516ff5

Merge branch 'master' of https://github.com/PowerShell/PSScriptAnalyzer…

cc6464f

… into CI_NotAppVeyorSpecific

Add basic yaml for testing in azure pipelines

a457ea7

bootstrap template

16d1ae3

fix whitespace. the joy of yaml

8e3ded0

fix path

91c4ca7

refactor

8025294

test template

ec67d89

fix yaml

e691f34

publish test results

a46dbd3

fix indentation

3f150ba

unix yaml

15eacdb

have bootstrap and build in the same task for unix

7eda9ad

ps v5 template

d6bcc20

ps v5 fix

79d3c85

try to use template for pwsh in bootstrap

c55fd1e

fix indent

6678f44

fix template param for psv2

3b4a63c

apply templating everywhere

f0376d8

skip dontet cli init on v5 as well

cb7037e

bergmeister requested a review from JamesWTruher June 20, 2019 17:28

bergmeister added the Area - Build label Jun 22, 2019

Fix custom rules tests that fail on Linux

7d4f0bf

bergmeister requested a review from rjmholt June 22, 2019 07:12

bergmeister added 3 commits July 5, 2019 20:50

Add more assertion details to failure

377be99

Merge branch 'master' of https://github.com/PowerShell/PSScriptAnalyzer…

4cff565

… into AzurePipelinesMigration

Add more assertion message to other failures and fix property name on…

479a682

… it.

bergmeister closed this Aug 22, 2019

bergmeister reopened this Aug 22, 2019

Merge branch 'master' into AzurePipelinesMigration

0289308

# Conflicts: # Tests/Engine/CustomizedRule.tests.ps1

bergmeister closed this Oct 27, 2019

bergmeister reopened this Oct 27, 2019

bergmeister closed this Jan 2, 2020

bergmeister reopened this Jan 2, 2020

Exclude failing tests on AzDO/Ubuntu

385febf

Merge pull request #11 from rjmholt/AzurePipelinesIntegration

ed81374

Exclude failing tests on AzDO/Ubuntu

bergmeister added 4 commits January 7, 2020 19:54

try fix ubuntu build where dotnet --version makes the pipeline stop i…

ccb4e91

…n Az-Do

Merge branch 'master' of https://github.com/PowerShell/PSScriptAnalyzer…

cc706d2

… into AzurePipelinesMigration Merge using HEAD, which reverts the last commit... # Conflicts: # build.psm1

Bootstrap as part of build process

a350326

remove bootstrap.yaml, which is now not needed any more

b44c9a9

bergmeister commented Jan 7, 2020

View reviewed changes

re-trigger sporadic failure

bc53763

JamesWTruher approved these changes Jan 28, 2020

View reviewed changes

bergmeister merged commit 8b2c02d into PowerShell:master Feb 2, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Azure pipelines migration #1267

Azure pipelines migration #1267

bergmeister commented Jun 20, 2019 •

edited

Loading

rjmholt commented Jun 28, 2019

rjmholt commented Jun 28, 2019

rjmholt commented Jun 28, 2019

bergmeister commented Oct 27, 2019

rjmholt commented Jan 3, 2020

bergmeister commented Jan 3, 2020

rjmholt commented Jan 3, 2020

rjmholt commented Jan 3, 2020

bergmeister commented Jan 3, 2020

rjmholt commented Jan 3, 2020

bergmeister commented Jan 3, 2020

rjmholt commented Jan 7, 2020

rjmholt commented Jan 7, 2020

bergmeister commented Jan 7, 2020

bergmeister commented Jan 7, 2020 •

edited

Loading

bergmeister commented Jan 7, 2020

bergmeister Jan 7, 2020 •

edited

Loading

rjmholt Jan 7, 2020

JamesWTruher left a comment

Azure pipelines migration #1267

Azure pipelines migration #1267

Conversation

bergmeister commented Jun 20, 2019 • edited Loading

PR Summary

PR Checklist

rjmholt commented Jun 28, 2019

rjmholt commented Jun 28, 2019

rjmholt commented Jun 28, 2019

bergmeister commented Oct 27, 2019

rjmholt commented Jan 3, 2020

bergmeister commented Jan 3, 2020

rjmholt commented Jan 3, 2020

rjmholt commented Jan 3, 2020

bergmeister commented Jan 3, 2020

rjmholt commented Jan 3, 2020

bergmeister commented Jan 3, 2020

rjmholt commented Jan 7, 2020

rjmholt commented Jan 7, 2020

bergmeister commented Jan 7, 2020

bergmeister commented Jan 7, 2020 • edited Loading

bergmeister commented Jan 7, 2020

bergmeister Jan 7, 2020 • edited Loading

Choose a reason for hiding this comment

rjmholt Jan 7, 2020

Choose a reason for hiding this comment

JamesWTruher left a comment

Choose a reason for hiding this comment

bergmeister commented Jun 20, 2019 •

edited

Loading

bergmeister commented Jan 7, 2020 •

edited

Loading

bergmeister Jan 7, 2020 •

edited

Loading