JIT: resolve issues with OSR and PGO

[AndyAyersMS]

OSR and PGO don't play nicely together.

See #46882 (comment) and follow on comments.

[AndyAyersMS]

There's a workaround in place (use block profiling with OSR) that's good enough for now.

So, moving to Future.

[AndyAyersMS]

The interaction of OSR and PGO is a bit more involved than I'd been thinking. There are a few sub-problems:

1. The original method's PGO data will likely be incomplete. In particular there may not be any profile data "after" the patchpoint so large portions of the method may appear cold when the OSR method is being jitted.
2. With sparse profiles we may even lose the ability to infer counts in the parts of the original method that were profiled, unless we model the potential impact of the OSR flow.
3. This problem will persist if we ever rejit the method at Tier1, since any execution of the original method that would have accumulated profile data will be redirected to the OSR method.
4. With sparse instrumentation the OSR method's flow graph needs to match the original method, initially, so we can determine the right placement of counts.

It seems plausible that we can fix 2, 3, and 4 by modelling OSR flow in the original method, deferring OSR flow updates until after the profile model has been built (which is happening as part of #59784) and then adding instrumentation to the root part of the OSR method (this is a bit novel, but seems doable, we just use the schema binding we get from reading the PGO data in the OSR method to emit "redundant" probes in the OSR method itself). The downside here is that the OSR method has some extra overhead, but maybe that's acceptable.

But fixing 1 seems hard...unless we have the OSR method return back to the original method , once we've exited the part of the OSR method flowgraph that is post-dominated by the patchpoint. Note there could be many such points, and each one might need a custom "fixup" action to restore the IL state in the original method from the live state in the OSR method; plus we might need to do lifetime extensions to ensure that we don't dead code something in the OSR method that's still needed back in the original.

[AndyAyersMS]

Rough plan of attack.

1. Keep using "dense" profiles for Tier0 methods with patchpoints.
2. Don't use dynamic PGO data to optimize OSR methods (static PGO ok).
3. Add ability to fetch schema and live data from VM. OSR method does this, verifies schema matches, and then instruments the parts of the code it will generate. Thus both the Tier0 and OSR methods will contribute to overall profile production.

So OSR methods won't be competitive with their Tier1 counterparts; this should be OK. But they should be much faster than Tier0 and perhaps that's good enough.

[AndyAyersMS]

Working on the abovem ran into a complication. It's not possible to get the OSR method to produce the exact same schema as the Tier0 method.

They produce the same flow graph initially (since it's based on method IL). But they do different selective importation, and currently block and class instrumentation is gated on this:

runtime/src/coreclr/jit/fgprofile.cpp

Lines 346 to 349 in 1d352fc

	bool ShouldProcess(BasicBlock* block) override
	{
	return ((block->bbFlags & (BBF_INTERNAL | BBF_IMPORTED)) == BBF_IMPORTED);
	}

runtime/src/coreclr/jit/fgprofile.cpp

Lines 1427 to 1430 in 1d352fc

	bool ShouldProcess(BasicBlock* block) override
	{
	return ((block->bbFlags & (BBF_INTERNAL | BBF_IMPORTED)) == BBF_IMPORTED);
	}

We could fix this easily enough for counts by "over allocating" the schema for both Tier0 and OSR (basically pretending like we're going to import the entire method) because count probes don't depend on the IR in the block. The downside here is using more schema/data memory then necessary.

But for class probes the probe structure depends on the IR so if we import differently we can't hope to match class probe schema entries.

Some alternatives to explore:

• update the runtime to recognize when one schema (the one we propose to use for the OSR method) is a subset of another (the Tier0 method), and properly update the offset info for the subset based on the superset. This should not be too hard, provided the runtime doesn't reorder the schema entries that the jit creates at Tier0.
• do something similar, but on the jit side -- fetch the Tier0 schema/data and then verify each place we'd put a probe is already covered.

[AndyAyersMS]

One other issue:

• we may lose some PGO coverage for inlinees as we're not adding probes for them to the OSR method