ci: finish the migration to azure

[pietroalbini]

This moves to azure the last builders as we discussed on Discord last night.

r? @alexcrichton

[Mark-Simulacrum]

Unless you want a review specifically by Alex, r=me

[pietroalbini]

@bors r=Mark-Simulacrum p=1000

[bors]

📌 Commit effffe792da67a2a49f8819bb0fc1c0935044a34 has been approved by Mark-Simulacrum

[pietroalbini]

r? @Mark-Simulacrum

[bors]

⌛ Testing commit effffe792da67a2a49f8819bb0fc1c0935044a34 with merge cf7c4892a8319d8d4f3fb3c8a42e612adc756a79...

[alexcrichton]

Fwiw I don't think we can delete Travis and appveyor yet since bors is still listening for those services to complete, so it may be best to just add one null job on both and we can later change bors after this lands

[pietroalbini]

Uh good point @alexcrichton, added a dummy job on each of them.

@bors r=Mark-Simulacrum retry

[bors]

📌 Commit fffac31 has been approved by Mark-Simulacrum

[bors]

⌛ Testing commit fffac31 with merge 28bffc3f492cccedccc22608bf33fcb0b466e912...

[bors]

💔 Test failed - checks-azure

[pietroalbini]

@bors retry

[bors]

⌛ Testing commit fffac31 with merge a4ebb38...

[bors]

💔 Test failed - checks-azure

[Xanewok]

Now only Miri failed. @RalfJung is it spurious? Can we retry it or does it need an update?

[RalfJung]

This doesn't look spurious. I have never seen this failure before:

normalized stdout:
args


expected stdout:
args


diff of stdout:

 args
 

The actual stdout differed from the expected stdout.
Actual stdout saved to C:\Users\VSSADM~1\AppData\Local\Temp\compiletest9YVbIE\args.stdout
To update references, run this command from build directory:
tests/run-pass/update-references.sh 'C:\Users\VSSADM~1\AppData\Local\Temp\compiletest9YVbIE' 'args.rs'

error: 1 errors occurred comparing output.
status: exit code: 0
command: PATH="D:\a\1\s\build\x86_64-pc-windows-msvc\stage2\bin;D:\a\1\s\build\x86_64-pc-windows-msvc\stage2-tools\x86_64-pc-windows-msvc\release\build\curl-sys-602f6dde8610a19b\out\build;D:\a\1\s\build\x86_64-pc-windows-msvc\stage2-tools\x86_64-pc-windows-msvc\release\build\libnghttp2-sys-7776a7dcba25a9ae\out\i\lib;D:\a\1\s\build\x86_64-pc-windows-msvc\stage2-tools\x86_64-pc-windows-msvc\release\build\libz-sys-b75492441773d9c1\out\build;D:\a\1\s\build\x86_64-pc-windows-msvc\stage2-tools\x86_64-pc-windows-msvc\release\deps;D:\a\1\s\build\x86_64-pc-windows-msvc\stage2-tools\x86_64-pc-windows-msvc\release;D:\a\1\s\build\x86_64-pc-windows-msvc\stage2\lib\rustlib\x86_64-pc-windows-msvc\lib;D:\a\1\s\ninja;C:\Python27amd64;D:\a\1\s\citools\msys64\mingw64\bin;D:\a\1\s\citools\msys64\usr\bin;D:\a\1\s\sccache;C:\agents\2.153.2\externals\git\cmd;C:\hostedtoolcache\windows\Python\3.6.8\x64;C:\hostedtoolcache\windows\Python\3.6.8\x64\Scripts;C:\Program Files\Mercurial;C:\vcpkg;C:\cf-cli;C:\Program Files (x86)\NSIS;C:\Program Files\Mercurial;C:\Program Files\Boost\1.69.0;C:\Users\VssAdministrator\.dotnet\tools;C:\Program Files\dotnet;C:\mysql-5.7.21-winx64\bin;C:\Program Files\Java\zulu-8-azure-jdk_8.38.0.13-8.0.212-win_x64\bin;C:\npm\prefix;C:\Program Files\Rust\.cargo\bin;C:\hostedtoolcache\windows\Ruby\2.5.3\x64\bin;C:\Go1.12.4\bin;C:\Program Files\Git\bin;C:\Program Files\Git\usr\bin;C:\Program Files\Git\mingw64\bin;C:\hostedtoolcache\windows\Python\3.6.8\Scripts;C:\hostedtoolcache\windows\Python\3.6.8;C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin;C:\Program Files\Microsoft MPI\Bin;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0;C:\ProgramData\Chocolatey\bin;C:\Program Files\Docker;C:\Program Files\PowerShell\6;C:\Program Files\dotnet;C:\Program Files\Microsoft SQL Server\130\Tools\Binn;C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\130\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\150\DTS\Binn;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Program Files\Microsoft Service Fabric\bin\Fabric\Fabric.Code;C:\Program Files\Microsoft SDKs\Service Fabric\Tools\ServiceFabricLocalClusterManager;C:\Program Files\Git\cmd;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\tools\php;C:\Program Files (x86)\Subversion\bin;C:\Program Files\nodejs;C:\Program Files\CMake\bin;C:\Strawberry\c\bin;C:\Strawberry\perl\site\bin;C:\Strawberry\perl\bin;C:\Program Files\OpenSSL\bin;C:\Users\VssAdministrator\AppData\Local\Microsoft\WindowsApps" "D:\\a\\1\\s\\build\\x86_64-pc-windows-msvc\\stage2-tools-bin\\miri.exe" "tests/run-pass\\args.rs" "-L" "C:\\Users\\VSSADM~1\\AppData\\Local\\Temp\\compiletest9YVbIE" "--target=x86_64-pc-windows-msvc" "--error-format" "json" "-C" "prefer-dynamic" "-o" "C:\\Users\\VSSADM~1\\AppData\\Local\\Temp\\compiletest9YVbIE\\args.stage-id.exe" "-Dwarnings" "-Dunused" "--edition" "2018" "--sysroot" "D:\\a\\1\\s\\build\\x86_64-pc-windows-msvc\\stage2" "-L" "C:\\Users\\VSSADM~1\\AppData\\Local\\Temp\\compiletest9YVbIE\\args.stage-id.aux" "-A" "unused"
stdout:
------------------------------------------
args

------------------------------------------
stderr:
------------------------------------------

------------------------------------------

test [ui] run-pass\args.rs ... FAILED

And only on Windows? Things work fine on our own AppVeyor CI.

What changed compared to the previous AppVeyor tools builder?

[pietroalbini]

What changed compared to the previous AppVeyor tools builder?

Our configuration scripts are the same between AppVeyor and Azure, but the CI image is of course different. I don't know what are the exact changes though.

[RalfJung]

Looks like somehow ether the encoding of the process arguments into a Windows command string, or the decoding (inside the interpreted program) into a list again, fails now where it worked before. No idea how that can happen; differences in the host platform shouldn't "leak" into the interpreted program.

This is using the same architecture / target triple, right?

You could try updating Miri to this branch. That will definitely fail to land, but it'll tell us a bit more. (EDIT: don't, I didn't realize that all tests with reference output are failing; see the next post.)

Alternatively, if this is an Azure vs AppVeyor thing, it could help to have Azure CI for Miri. We probably eventually want that anyway. But I won't have time to set that up any time soon, I'm afraid.

[RalfJung]

Looking at this again... wait a second. "normalized" and "expected" stdout look the same, and still compiletest complains they are different? And I just now noticed that all tests with reference output are failing in the same way.

Is this a \n vs \r\n thing? Does maybe the Miri git repo get cloned with different settings so that the line endings in the reference file are different?

[mati865]

Clippy had similar CRLF vs LF issue when adding Windows based Travis job (Appveyor was working just fine). Possibly Windows Azure jobs have misconfigured git so it changes line endings when cloning the repo.
The workaround was to enforce line endings with .gitattributes: https://github.com/rust-lang/rust-clippy/blob/fffb2691a3d2d6bbd9521ff35d98301b2cdf4298/.gitattributes

[RalfJung]

Thanks! Let's try a .gitattributes for Miri as well then. rust-lang/miri#816 is just waiting for an AppVeyor slot to land... those slots can be hard to get though. :/

[RalfJung]

The Miri PR made it. :)

So if you include that in here, maybe that helps with the line-ending problems?

[pietroalbini]

@bors r+ p=2000

[bors]

📌 Commit 3159741fe0e5a752ea96f752c32a72ade3afbfbd has been approved by pietroalbini

[RalfJung]

@pietroalbini I think you have conflicts with master now? There was a Miri update there since this branch started.

[pietroalbini]

Thanks for noticing that @RalfJung...

@bors r=Mark-Simulacrum p=1000

[bors]

📌 Commit dc9d2b3 has been approved by Mark-Simulacrum

[bors]

⌛ Testing commit dc9d2b3 with merge 6ea4036...

[bors]

☀️ Test successful - checks-azure, checks-travis, status-appveyor
Approved by: Mark-Simulacrum
Pushing 6ea4036 to master...