Lower the platform requirement

[Kyle-Ye]

Description

The current platform requirement makes it hard for other existing package to adopt swift-testing.
Consider lowering the requirement and mark related API with @avaiable mark.

Expected behavior

A package with iOS 13 support can have a dependency on this package.

Actual behavior

A package with iOS 13 support can not have a dependency on this package.

Steps to reproduce

No response

swift-testing version/commit hash

No response

Swift & OS version (output of swift --version && uname -a)

No response

[stmontgomery]

Thanks for raising this topic. First, a reminder: we currently consider this package experimental and are not certain what the final OS deployment versions for this package on Apple platforms will be. That said, there is currently not a strong reason to maintain the current, higher OS version requirements and it would be beneficial to lower them to allow wider experimentation.

How should we decide which specific versions to use, then? This can be tricky to balance: we don't want to go too far in the past, since clients can become dependent on those older versions and, if we want to evolve this codebase in a way that depends on a newer API, we might find that we have to compromise our implementation which becomes a maintenance burden. But we also don't want to require versions which are too new, at least not without a specific motivation, since that can limit adoption.

The swift-testing package relies heavily on Swift Concurrency, so we can't require versions older than that feature supports. Also, regardless of the library deployment targets, when it comes to iOS, watchOS, and tvOS, the tools only support installing and running tests on certain versions. It can also be useful to align OS version requirements with XCTest, so that there are not conflicts for "helper" libraries which vend testing utilities for use in either library. For these reasons, I think the most sensible starting point for us is to use the lowest OS versions which both support Swift Concurrency and running tests via Xcode—specifically:

• macOS 13.0
• iOS 13.0
• macCatalyst 16.0
• tvOS 13.0
• watchOS 7.0

[Kyle-Ye]

Thanks for raising this topic. First, a reminder: we currently consider this package experimental and are not certain what the final OS deployment versions for this package on Apple platforms will be. That said, there is currently not a strong reason to maintain the current, higher OS version requirements and it would be beneficial to lower them to allow wider experimentation.

How should we decide which specific versions to use, then? This can be tricky to balance: we don't want to go too far in the past, since clients can become dependent on those older versions and, if we want to evolve this codebase in a way that depends on a newer API, we might find that we have to compromise our implementation which becomes a maintenance burden. But we also don't want to require versions which are too new, at least not without a specific motivation, since that can limit adoption.

The swift-testing package relies heavily on Swift Concurrency, so we can't require versions older than that feature supports. Also, regardless of the library deployment targets, when it comes to iOS, watchOS, and tvOS, the tools only support installing and running tests on certain versions. It can also be useful to align OS version requirements with XCTest, so that there are not conflicts for "helper" libraries which vend testing utilities for use in either library. For these reasons, I think the most sensible starting point for us is to use the lowest OS versions which both support Swift Concurrency and running tests via Xcode—specifically:

• macOS 13.0
• iOS 13.0
• macCatalyst 16.0
• tvOS 13.0
• watchOS 7.0

We may not be focusing on the same thing. Even if Swift concurrency or any other framework this package currently depended on have a higher requirement. I still suggest using platforms: [.iOS(.v13)].

Example

Considering Apple release a new module call DemoKit on iOS 18.

// module DemoKit
@available(iOS 18, *)
public class Demo { ... }

And a third-party developer developed a library MyDemoKit built upon the newly introduced DemoKit.

// module MyDemoKit + platforms: [.iOS(.v18)]
public final class MyDemo: DemoKit.Demo { ... }

If the library owner choose to write platforms: [.iOS(.v18)], a swift package / xcodeproj which have iOS 15 support can't add MyDemoKit as a dependency while they can add DemoKit as a dependency.

// MyPackage with platforms: [.iOS(.v15)]
// Or MyApp.xcodeproj whose minimum deployments is iOS 15

import DemoKit // ✅
import MyDemoKit // ❌ Compiler error

func test() {
    if #available(iOS 18, *) {
        let d = Demo()
        ...
    }
}

Instead of forcing the downstream to increase the target requirement, we can make it possible by lowering the requirement of MyDemoKit and potentially mark all the symbols as @available(iOS 18, *) when needed.

// module MyDemoKit + platforms: [.iOS(.v15)]
@available(iOS 18, *)
public final class MyDemo: DemoKit.Demo { ... }

By doing so, MyPackage and MyApp.xcodeproj can have a dependency on MyDemoKit just like the build-in DemoKit.

// MyPackage with platforms: [.iOS(.v15)]
// Or MyApp.xcodeproj whose minimum deployments is iOS 15

import DemoKit // ✅
import MyDemoKit // ✅

func test() {
    if #available(iOS 18, *) {
        let d1 = Demo()
        let d2 = MyDemo()
        ...
    }
}

Conclusion

By using the trick of so called "weak linking" in SwiftPM by lowering the requirement. The package can potentially get a broader audience.

Some package are actually iOS 15+ and hope to use swift-testing as their testing framework. But for the sake of their downstream user, they have lowered their requirement and marked all of their symbol with @available(iOS 15, *) in the past.

[Kyle-Ye]

Currently I use the following snippets to workaround (Add "ENABLE_SWIFT_TESTING=1" when testing)

let enableSwiftTesting = ProcessInfo.processInfo.environment["ENABLE_SWIFT_TESTING"] != nil
let platforms: [SupportedPlatform]?
if enableSwiftTesting {
    platforms = [.iOS(.v16), .macOS(.v13), .tvOS(.v16), .watchOS(.v9), .visionOS(.v1)]
} else {
    platforms = [.iOS(.v13), .macOS(.v10_15), .tvOS(.v13), .watchOS(.v6), .visionOS(.v1)]
}
let package = Package(
    name: ...,
    platforms: platforms,
    ...
)

if enableSwiftTesting {
    package.dependencies.append(.package(url: "https://github.com/apple/swift-testing.git", from: "0.1.0"))
    package.targets.append(xx)
}

I'd like to know what's your opinion on this, can we make some progress on the PR I suggested?
cc @stmontgomery

[stmontgomery]

Thanks @Kyle-Ye. After further consideration I agree it will be best if we align our OS deployment targets with each other as you suggest. One specific reason that became more clear to me recently is that if we keep our macOS version higher than iOS, it is easier for build errors to be introduced on iOS since we don't build for it quite as regularly as we do for macOS (at least, not given our current CI).

I discovered that I introduced such a build failure recently in #104 and have a fix incoming for that, which I'll need to land before we merge your PR.

[Kyle-Ye]

By the way, is there any plan for a new release (eg. 0.1.1 or 0.2.0)? Thanks @stmontgomery