feat!: used_extensions calls for both ops and signatures

[aborgna-q]

Adds methods

• OpType::used_extensions(&self) -> Result<ExtensionRegistry, _>
• Signature::used_extensions(&self) -> Result<ExtensionRegistry, _>
  and tests these along with the code merged in feat!: Resolve OpaqueOps and CustomType extensions  #1735.

Moves the code from #1735 into resolution::types_mut, and adds a (quite-similar) non-mutable version in ::types that only collects the extensions without modifying the OpType.

Fixes the resolution not exploring types inside a CustomType type arguments.

drive-by: Implement Display, Serialize, and ::new for ExtensionRegistry.
drive-by: ExtensionSet should take ids by value when inserting.
drive-by: Fix Hugr::resolve_extension_defs not scanning all the ops.

These changes were extracted from #1738.

BREAKING CHANGE: Removed ExtensionRegistry::try_new. Use new instead, and call ExtensionRegistry::validate to validate.
BREAKING CHANGE: ExtensionSet::insert and singleton take extension ids by value instead of cloning internally.

[codecov]

Codecov Report

Attention: Patch coverage is 82.62712% with 82 lines in your changes missing coverage. Please review.

Project coverage is 86.29%. Comparing base (4c1c6ee) to head (2c8ad5e).
Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
hugr-core/src/extension/resolution/types_mut.rs	75.00%	4 Missing and 31 partials ⚠️
hugr-core/src/extension.rs	71.01%	20 Missing ⚠️
hugr-core/src/extension/resolution/types.rs	92.85%	7 Missing ⚠️
hugr-core/src/extension/resolution/ops.rs	68.75%	5 Missing ⚠️
hugr-core/src/ops/custom.rs	72.72%	3 Missing ⚠️
hugr-core/src/extension/resolution/test.rs	92.00%	1 Missing and 1 partial ⚠️
hugr-core/src/hugr/rewrite/inline_dfg.rs	66.66%	0 Missing and 2 partials ⚠️
hugr-core/src/hugr/rewrite/replace.rs	0.00%	2 Missing ⚠️
hugr-core/src/hugr/validate/test.rs	77.77%	0 Missing and 2 partials ⚠️
hugr-core/src/extension/op_def.rs	75.00%	0 Missing and 1 partial ⚠️
... and 3 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1739      +/-   ##
==========================================
+ Coverage   86.28%   86.29%   +0.01%     
==========================================
  Files         177      179       +2     
  Lines       32087    32662     +575     
  Branches    28999    29574     +575     
==========================================
+ Hits        27687    28187     +500     
- Misses       2775     2797      +22     
- Partials     1625     1678      +53     

Flag	Coverage Δ
python	92.42% <ø> (ø)
rust	85.65% <82.62%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[aborgna-q]

Coverage is quite bad. I'll add some extra tests today.

Done

[acl-cqc]

Doesn't the signature of a DataflowBlock include all the elements of the inputs, other_outputs and sum_rows? So can't you just collect_signature_exts on the block's signature, and the same for e.g. TailLoop, Conditional, and so on, like OpaqueOp and most other blocks?

[aborgna-q]

Yes, but that requires allocating a Signature, cloning all row elements, computing the concrete input/output rows, and immediately destructuring them to run this same logic.

[acl-cqc]

Is this true even with Cow of signature? I mean, we're gonna be computing that signature a good few times anyway...

[acl-cqc]

Hmmmm, even when we have Cow<Signature>? I mean, we are gonna be constructing that thing a few times, clearly it's gonna help to cache it in the op. OK don't add that for this PR, but if you're optimizing that anyway and could use that here...

[acl-cqc]

So the other way to do this - is to put the collect_blah_exts methods as impls of a single method in a trait, and then you can add a trait-default method (or a method taking an impl ExtensionCollectionTrait) that does this...that'd get you the two used_extensions methods that you have plus a load more with only having to write this once. (I wasn't going to suggest the trait approach until I saw this...)

[acl-cqc]

This doesn't work if you change OpType::used_extensions to take the extra Option<Node> parameter I suggested elsewhere, though. So feel free to drop.

[aborgna-q]

I opted for keeping all the logic in a single module, as it's short and quite similar between all the ops.

[acl-cqc]

If you changed OpType::used_extensions to take the Option<Node> param as extra, which seems reasonable, then you could call that here and remove all direct calls to collect_op_(type_)extensions. Then hide those (or inline into the single caller OpType::used_extensions).

[aborgna-q]

I was calling the resolve methods explicitly, but it's exactly the same as op.used_exts.
I just changed it to that.

[acl-cqc]

Should this be pub? The nomenclature is a bit confusing here - it feels equivalent to Signature::used_extensions but actually it's only one part of OpType::used_extensions. However the other collect things take two &muts. Should one really use this directly?

[aborgna-q]

It's hidden inside the private types module. But I'll add the (crate) for clarity.

[acl-cqc]

I reckon the pub function here should be OpType::used_extensions.

[aborgna-q]

It's a private module. (Otherwise the docs check would complain)

[acl-cqc]

Ah ok, yes I missed that. Ok I'll hold some fire there then ;)

[hugrbot]

This PR contains breaking changes to the public Rust API.

cargo-semver-checks summary

--- failure inherent_method_missing: pub method removed or renamed ---

Description:
A publicly-visible method or associated fn is no longer available under its prior name. It may have been renamed or removed entirely.
      ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
     impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.37.0/src/lints/inherent_method_missing.ron

Failed in:
ExtensionRegistry::try_new, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-core/src/extension.rs:58

--- failure pub_module_level_const_missing: pub module-level const is missing ---

Description:
A public const is missing, renamed, or changed from const to static.
      ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
     impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.37.0/src/lints/pub_module_level_const_missing.ron

Failed in:
EMPTY_REG in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-core/src/extension.rs:209

[acl-cqc]

Generally - great :). I think all of these comments are now small and local, so looks good and thanks @aborgna-q , let's get it in :)

I think there is a bit of an issue that ideally one would like Hugr::validate to also validate the registry (if not cached in that AtomicBool), but that requires passing an &mut ExtensionRegistry which is gonna require changing everywhere (and annoyingly). Otherwise you could add methods to validate immutably (i.e. without updating the cached bool), I guess....

[acl-cqc]

Most nitty super-nit ever: these two lines after the removal, for consistency with the other operations :-) :-D

[aborgna-q]

remove returns the Option<Arc<Ext>>, so moving it requires declaring local vars..

[acl-cqc]

Suggested change

	/// This is used to avoid re-validating the extensions every time they are
	/// This is used to avoid re-validating the extensions every time the registry is

(I think - if you grab an extension out of the registry, you can use it without validating the registry; it's only when the registry itself is passed to validate a Hugr, say. Of course this is a subtlety of how we interpret "use" of an extension vs "use" of the regstry, so feel free to be more explicit - "every time a Hugr is validated against the registry", perhaps, although that's a bit more of a mouthful)

[acl-cqc]

random driveby, feel free to ignore - could this could be a const fn, to which you could add Extensions yourself? (and then static EMPTY_REG: ExtensionRegistry = ExtensionRegistry::new_empty(); or similar)

[aborgna-q]

ExtensionRegistry now has internal mutability, so it should not be used as a const.
(it does compile, but using it just raises linting errors everywhere, with good reason)

[acl-cqc]

Hmmmm, even when we have Cow<Signature>? I mean, we are gonna be constructing that thing a few times, clearly it's gonna help to cache it in the op. OK don't add that for this PR, but if you're optimizing that anyway and could use that here...

[acl-cqc]

Yes please do! The comment above does not explain (to me) why we are using node_capacity

[acl-cqc]

Many examples here and elsewhere, I wonder if it's worth adding to ExtensionRegistry

   fn validated(mut self) -> Result<Self, ....> {
      self.validate()?;
      self
   }

[aborgna-q]

We'll call ExtRegistry::validate on each Hugr::validate as soon as #1738 gets merged, so this shouldn't be needed.