Consistently use subtyping in method resolution

[oli-obk]

fixes #126062

An earlier version of this PR modified how we compute variance, but the root cause was an inconsistency between the usage of eq and sub, where we assumed that the latter passing implies the former will pass.

r? @compiler-errors

[rustbot]

changes to the core type system

cc @compiler-errors, @lcnr

[compiler-errors]

I don't personally feel comfortable about adding a new error variance. Seems really strange imo.

I think we could easily just force the error variance to Invariant rather than Bivariant, which would prevent these ICEs at risk of slightly more awkward errors, but at the benefit of not having it be its own thing that actually need to be handled in relations.

r? lcnr

[compiler-errors]

More thoughts:

So the underlying issue is the inconsistency we have with the xform implementation, where:

• invariant xform bivariant = invariant
• covariant xform bivariant = bivariant

... and the fact we're using equality during method lookup and subtyping during confirmation, leading to a mismatch in the result of those relations.

This seems like a kind of roundabout way of fixing this, since the real underlying issue is that the way we handle bivariance in the compiler is just kind of awkward.

Not to repeat myself, but I think forcing invariance would just fix things in a simpler way.

[oli-obk]

Not to repeat myself, but I think forcing invariance would just fix things in a simpler way.

done

[compiler-errors]

OK -- will review this then. Thanks for considering my feedback.

If lcnr strongly disagrees about forcing invariance, then it shouldn't be hard to do a follow-up and actually introduce the error variance later.

r? compiler-errors

[compiler-errors]

Does this function ever return Err?

Unless I'm being particularly unobservant, maybe let's rename this fn to check_variances_for_type_defn_and_set_unconstrained_to_invariant

(long name but there's only a few callsites -- happy to workshop the name lol

[compiler-errors]

edit: nvm I guess lol

[oli-obk]

I could return a vec of erroneous indices instead of doing the modification and the checks. Seems cleaner to separate these things

[compiler-errors]

some random thoughts -- pls forgive me for commenting on things that were modified subsequently in a follow-up commit lol

[compiler-errors]

This branch is unreachable rn

[compiler-errors]

nvm

[compiler-errors]

can we use ? like:

check_type_defn()?;
check_variances_for_type_defn()?;

then at the bottom we can Ok(())?

Seems to simplify control flow a bit

[compiler-errors]

nvm this is irrelevant

[compiler-errors]

edit: nvm I guess lol

[compiler-errors]

I wonder if we should just use error_reported to delay the error message below but don't early-return (using emit_unless). I kinda feel bad that we're setting all of the substs to bivariant even if they are constrained otherwise (e.g. to covariance) somewhere else in the definition.

[oli-obk]

Yea, these early aborts were preexisting. I didn't want to touch them in the previously-big PR (I'm assuming they got added to avoid ICEs), but now it's reasonable to investigate.

[lcnr]

More thoughts:

So the underlying issue is the inconsistency we have with the xform implementation, where:

* invariant xform bivariant = invariant

* covariant xform bivariant = bivariant

... and the fact we're using equality during method lookup and subtyping during confirmation, leading to a mismatch in the result of those relations.

We get subtyping to succeed and equate to fail and that causes an ICE because we expect "subtype implies equal"? The underlying issue is just that, isn't it? We have types which are subtypes of each other but not equal. Apart from variance this is also the case with the leak check. This can't currently trigger as we just put the nested obligations in the FnCtxt without trying them, even in the new solver, but if we were to eagerly compute nested obligations, the following test would ICE https://rust.godbolt.org/z/KxGeqevaW.

I also don't see how this PR fixes the underlying issue with bivariance:

trait Proj {
    type Assoc;
}
impl<T> Proj for T {
    type Assoc = T;
}

struct Fail<T: Proj<Assoc = U>, U>(T);

impl Fail<i32, i32> {
    const C: () = ();
}

fn main() {
    Fail::<i32, u32>::C
    // expected: WF error
    // actual: ICE
}

[oli-obk]

We get subtyping to succeed and equate to fail and that causes an ICE because we expect "subtype implies equal"? The underlying issue is just that, isn't it?

yes, but I did not want to touch that behaviour, because I really can't tell the implications. Almost nothing should be affected, because we already generate fresh inference vars during method resolution, so all we're doing is constraining inference vars that have no other constraints.

I did the change (see latest force push), and the only affected test is never type fallback, which checks with my analysis above.

[lcnr]

why this change?

[oli-obk]

Instead of merging the two infer vars, we now create a subtyping relation for them, so they aren't the same. Never type fallback will still fall back to ! for the one type var, but the other type var which is a subtype, will only fall back to (). I haven't debugged this deeply, since never type fallback is cursed and debugging it will probably yield no new information beyond all the edge cases that already are know for never type fallback.

[lcnr]

oh lel, you didn't change the UI test, you just reformatted it

[lcnr]

yes, but I did not want to touch that behaviour, because I really can't tell the implications.

I think non-method calls should use eq instead of sub everywhere 🤔 it seems surprising that Type::<Subtype>::CONST works? though i'd like @compiler-errors input on that

[oli-obk]

#122317 (fully) switched from eq to subtyping

[lcnr]

oh well, that's ugly 🤔

struct B<T>(T);

impl B<fn(&'static ())> {
    fn method(self) {
        println!("hey");
    }
}

fn foo(x: B<for<'a> fn(&'a ())>) {
    x.method();
}

fn main() {
    B::<for<'a> fn(&'a ())>::method(B(|&()| ()));
}

i expected us to only use subtyping when looking up the self type, not when using a path 🤔 that's kinda meh. But yeah, I guess sub is fine here, opening an issue as I would like to try and change that

[matthiaskrgr]

@bors r-
#126480 (comment)

[oli-obk]

@bors r=lcnr

[bors]

📌 Commit 3e6e6b1 has been approved by lcnr

It is now in the queue for this repository.

[bors]

⌛ Testing commit 3e6e6b1 with merge 9b584a6...

[bors]

☀️ Test successful - checks-actions
Approved by: lcnr
Pushing 9b584a6 to master...

[rust-timer]

Finished benchmarking commit (9b584a6): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

Results (primary 5.0%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	5.0%	[5.0%, 5.0%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	5.0%	[5.0%, 5.0%]	1

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 669.445s -> 670.422s (0.15%)
Artifact size: 320.51 MiB -> 319.89 MiB (-0.19%)