Facilitate tracing from WASM

[mattrutherford]

Add --wasm-tracing CLI flag to enable tracing from WASM.

This must be used in conjunction with --tracing-targets (and optionally --tracing-receiver).

Add 2 new host functions to allow tracing span enter and exit. The tracing API requires span target and name to be const, so it's not possible to create spans directly with the actual target and name; instead all are created with a single pre-defined identifier, with the span's fields containing the actual target and name, which is then patched in the tracing Subscriber in client.

Each span is stored in the TracingProxy along with it's associated guard until a call to exit is received, at which point it's dropped, triggering a span exit call to the tracing Subscriber.

Each host function call can take around 300ns, although only the second call will be included in the measurement. The total cost of a wasm trace that's incorporated into the measurement is potentially around 0.4µs.

[parity-cla-bot]

It looks like @mattrutherford signed our Contributor License Agreement. 👍

Many thanks,

Parity Technologies CLA Bot

[mattrutherford]

Here are some graphs for the first few minutes of running a dev chain:

[mattrutherford]

Just updated this PR to use a different way to create tracing spans, rather than rely on a brittle mechanism of pre-registered spans, this way reserves a special span name WASM_TRACE_IDENTIFIER to signal to the subscriber in client::tracing that the actual span name and target should instead be extracted from the spans' associated Fields. While still a compromise, I think that it's a more manageable one - just requiring that we reserve 3 identifiers - one for span name and two more for associated fields.

[tomusdrw]

I don't know that much about how tracing works, but looks good to me if it works :)

[mattrutherford]

I don't know that much about how tracing works, but looks good to me if it works :)

It works and while not as pure from the POV of tracing, I think it's a better solution for substrate at the moment.

[bkchr]

Last nitpicks, after that we should be okay.

[bkchr]

Ty for your work! :)

[mattrutherford]

Thanks all for helping get this one done!

Made final check with Polkadot master and should be ready to merge now.

Just reiterating that this runtime will be incompatible with earlier versions of the client due to the new host functions utilised by it. (Client upgrade is critical on runtime upgrade.)

[gavofyork]

hmm... is there no way to make it compatible assuming we're not using the new runtime functions? i can't really merge this any time with our current need to have some degree of release stability and a fairly regular upgrade schedule across two live chains.

[bkchr]

But what is the problem? We just need to make sure we have a node release before applying the runtime upgrade.

[mattrutherford]

Just to also clarify that the client is compatible with earlier runtime versions (currently syncing Polkadot CC1 with client based from this branch).

[gavofyork]

But what is the problem? We just need to make sure we have a node release before applying the runtime upgrade.

The problem is that we have two production networks with several hundreds of validators between them, as well as a tendency to do runtime upgrades from master branch every day or two. As soon as I merge this, then I won't be able to make a runtime ugrade on Polkadot until everybody has upgraded their nodes, same for Kusama. Requiring validators to upgrade on such a short timespan, or hamstringing myself so that I can't push upgrades from master branch is not something that I want to do outside of a real emergency.

[mattrutherford]

@gavofyork Would you agree to splitting this PR into two?

1st stage include the new host functions and related features in client (minus CLI options).

2nd stage at some point in future we can then add the runtime features to make use of them, also adding the CLI parameters to enable it.

This would allow us to give whatever window seems appropriate to allow everyone to upgrade.

[gavofyork]

Perfect.

[gnunicorn]

closing because of inactivity.