Add metrics tracking number of recently loaded crates/versions/platforms

[Nemo157]

These metrics are intended to be useful for evaluating changes for #1004. Knowing how many different crates get accessed over short timespans could help us know things like the potential memory cost of keeping cached indexes around short term.

I'm not sure what the performance impact of these might be in production. The most likely issue would be a large memory usage if the number of different platforms requested per-hour is very high, leading to a lot of small strings kept around to track these.

(based on #1017, so to review the lockfile changes look at the last commit in isolation)

[jyn514]

The most likely issue would be a large memory usage if the number of different platforms requested per-hour is very high, leading to a lot of small strings kept around to track these.

We already run high at about 5 GB in prod. I find it hard to think the number of crate/version/platform pairs would take up anything close to that, do you have a guesstimate for 'lot of small strings'?

[pietroalbini]

I think we can avoid the problem with strings by not storing them at all. My understanding is we don't care about the crate names/versions, so we can just store i32s:

• For crates we could map the crate ID with the instant.
• For versions we could map the release ID with the instant.
• For targets we could map (release ID, interned target name) with the instant.

[jyn514]

Didn't look too closely, but the idea seems right. Might review again later.

[pietroalbini]

Other than Joshua's comments, this looks good to me!

[jyn514]

Looks good to me after a rebase :)

[Nemo157]

Rebased, some minor conflicts but I've identified the only major conflict below, both this and master had started tracking the target when extracting that from the URL but this wants it without trailing /.

[Nemo157]

this target handling was the only major conflict.