Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix running cargo test without nextest #5266

Merged
merged 1 commit into from
May 29, 2024
Merged

Commits on May 29, 2024

  1. Fix running cargo test without nextest

    Reproduction: `cargo test -p apollo-federation -- -q`
    
    ## Background
    
    #5157 introduced a new kind of
    snapshot test for the Rust query planner. Test schemas are specified as sets
    of subgraph schemas, so composing them is needed. Since we don’t have
    composition in Rust yet, the tests rely on JS composition through Rover.
    To avoid a dependency on Rover on CI and for most contributors,
    the composed supergraph are cached in the repository. Rover use is opt-in
    and only required when a cached file is missing or out of date.
    (Composition inputs are hashed.)
    
    The file name is derived (through a macro) from the test function name.
    To detect name conflicts, a static `OnceLock<Mutex<HashSet<&'static str>>>`
    is used to ensure no name is used more than once.
    
    ## The problem
    
    The static hash set relies on all snapshot tests running in the same process.
    This is the case with `cargo test`, but `cargo nextest run` as used on CI
    isolates every test in its own process. This breaks conflict detection
    of cache file names for composed supergraph schemas, since each test only
    "sees" itself in the static hash set.
    
    #5240 introduced a name conflict:
    composition is used in a function called twice with different arguments.
    Because nextest was used both locally and on CI, the conflict went undectected.
    As a result, running `cargo test` on dev fails because the conflict is detected.
    
    ## This PR
    
    This PR fixes this case of cache file name conflict, but conflict detection
    is still broken with nextest.
    
    As a result it’s possible that this kind of conflict could happen again
    and be merged undectected.
    
    ## Non-solutions tried
    
    * Nextest has a notion of [test groups](https://nexte.st/book/test-groups),
      but they don’t appear to let multiple tests run in the same process
    
    * Instead of relying on the runtime side effect of tests, could conflict
      detection rely on enumerating tests at compile time?
      The [`linkme` crate](https://crates.io/crates/linkme) is a building block
      Router used to register Rust plugins. It could be used here in all
      composition inputs can be made const, but `std::any::type_name` used
      in a macro to extract the current function name is not `const fn`
      [yet](rust-lang/rust#63084)
    
    ## Potential solutions
    
    * Remove the cache and accept the dependency on Rover for testing.
      This impacts CI and all contributors.
    
    * Require every `planner!` macro invocation to specify an explicit
      cache file name instead of relying on the function name.
      Then conflict detection can use `linkme`.
    
    * Move conflict detection to a separate test that something something
      parses Rust source files of other tests something
    SimonSapin committed May 29, 2024
    Configuration menu
    Copy the full SHA
    c45f7bc View commit details
    Browse the repository at this point in the history