[Build] Enable building the early Swift Driver on linux with the prebuilt host toolchain

[finagolfin]

@bnbarham, let's see if this works on the linux CI now. This simply reverts most of your #68091, with the prebuilt Swift toolchain now installed on the linux CI.

[bnbarham]

@swift-ci please build toolchain

[finagolfin]

Alright, the only platform actually affected by this pull worked, linux. I checked the CI log and it's building and using the early swift driver just fine.

MacOS and Windows aren't affected by this pull: the former had a spurious build failure, while the latter's CI appears to be broken right now.

@bnbarham, ready for review and merge.

[bnbarham]

@swift-ci please build toolchain UBI9

[bnbarham]

@swift-ci please build toolchain CentOS 7

[bnbarham]

@swift-ci please build toolchain Ubuntu 22.04

[bnbarham]

@swift-ci please build toolchain Amazon Linux 2

[bnbarham]

@swift-ci please build toolchain Ubuntu 18.04

[bnbarham]

@swift-ci please test

[finagolfin]

Linux CI failed because the early swift driver tests are now being run, maybe for the first time on linux. @artemcm, you added those tests in #36377 a couple years ago, did you ever try them on linux? Natively on Android, I simply disable them with --skip-test-early-swift-driver.

Amazon Linux 2 toolchain build failed at source checkout, unrelated to this pull.

[bnbarham]

@swift-ci please build toolchain Amazon Linux 2

[bnbarham]

Natively on Android, I simply disable them with --skip-test-early-swift-driver.

I'd rather get to the bottom of the failures than just disable them 😅. That really just needs reproducing locally since the tests just all fail with exit code 1 though.

[finagolfin]

I'd rather get to the bottom of the failures than just disable them

Both this CI and my native Android toolchain build run the more extensive tests in the swift-driver repo later in the build, so I figured that's enough.

That really just needs reproducing locally since the tests just all fail with exit code 1 though.

Not all, if you scroll down, there are weird errors like this:

Command Output (stderr):
--
Swift version 5.11-dev (LLVM 1cd18baafcdabfd, Swift e002773d258ee29)
Target: x86_64-unknown-linux-gnu
Handled a.swift
Handled b.swift
Handled c.swift
Handled d.swift
Handled e.swift
Handled f.swift
Handled g.swift
error: autolink-extract command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: error opening input file 'a.o' (The file was not recognized as a valid object file
)

Most likely, it is just the various environment variables not being set right on linux. I figured it was something like that on Android and never looked into it.

[finagolfin]

Looked into those failing tests, it appears they are surfacing a genuine discrepancy between the legacy C++ Driver and the new swift-driver. I'm working on a fix.

[finagolfin]

@artemcm, how about a CI run with swiftlang/swift-driver#1479, that should fix most of these tests.

[artemcm]

swiftlang/swift-driver#1479
@swift-ci test

[finagolfin]

Rebased and added a fix for the single failing test, @bnbarham, need another CI run with the linked swift-driver pull.

[finagolfin]

This test now passes on linux as part of the early swift driver tests, so I enabled it, but it then failed with the old C++ Driver.

The issue is that these two lines don't expect another autolink file to be added on macOS, but that is needed on linux, which outputs this instead:

Job finished: {link: main <= main.o file1.o file2.o ... 1 more}

Omitting the closing brace here and below should get this test to work on both Mac and linux.

[bnbarham]

swiftlang/swift-driver#1479
@swift-ci test

[finagolfin]

@bnbarham, ready for review and merge, after one last Windows CI run and merge of the linked swift-driver pull, that gets these early swift driver tests to work on linux.

[DougGregor]

Oh, this is great. Thank you @finagolfin

[bnbarham]

Thanks @finagolfin! I'll do one last toolchain build and then merge :)

[bnbarham]

swiftlang/swift-driver#1479

@swift-ci please build toolchain Ubuntu 22.04