Implement numeric generics

[jfecher]

Related issue(s)

Provides a workaround for #606 but does not resolve it

Description

Summary of changes

Allow generics to be used in array length types. For example, this allows

fn id<N>(array: [Field; N]) -> [Field; N] {
    array
}

Structs can also be parameterized over array lengths:

struct Foo<N> {
    limbs: [u64; N],
}

Some basic math on constants and globals (using +, -, *, and /) is also supported, although we do not officially support math on non-constants like N above. Supporting arithmetic on variables like N would require non-trivial unification and evaluation support on symbolic expressions.

Test additions / changes

Added test numeric_generics

Checklist

• I have tested the changes locally.
• I have formatted the changes with Prettier and/or cargo fmt with default settings.
• I have linked this PR to the issue(s) that it resolves.
• I have reviewed the changes on GitHub, line by line.
• I have ensured all changes are covered in the description.

Additional context

Some care was taken to avoid naming this feature "const generics" even though it is effectively quite similar to rust's const generics feature. Avoiding this name lets us:

1. Be more clear that we may not support the exact same feature set rust supports here, while also allowing us room in the future to expand or branch off in a different direction.
2. Avoid a confusing name to newcomers. Particularly the const keyword being co-opted for compile-time evaluated expressions rather than just constants is confusing. This feature is also not called comptime generics in noir as we do not currently allow comptime variables within these generics. This is because comptime variables can be passed in as function arguments and allowing for these to be referenced would require something closer to dependent types.

[jfecher]

Note that this PR seems to have found a new bug in either the SSA pass or acir gen - likely with regards to array handling or generic arrays now being allowed within structs. The error is currently occurring while running the new numeric_generics test. I need to do some more investigation on whether we should wait to merge or if the error should be fixed separately in a later PR and we should edit the test in the meantime. At the time of writing, this PR currently does not introduce any changes in noirc_evaluator.

Edit: I think it is a separate bug, perhaps more related to the sum(id(...)) function calls. I've swapped out the test with a slightly modified example from the bug report and it is proving and verifying correctly.

[jfecher]

Fixed the bug in the global_consts test.
It is also worth mentioning this PR officially removes support for the use of local variables within array length types. This was never officially supported to begin with and was only possible via a bug that allowed it if the local was initialized to an integer.

In the future we can look into changing it to officially allow this rule but it impacts name resolution which will be greatly changed once the higher order functions PR is merged so I'd like to avoid these conflicts for now.

[vezenovm]

I think this PR looks good. Just had a coupe minor questions

[jfecher]

Added a commit fixing a bug where the previous representation of TypeExpression did not have a variant for NamedGenerics. This meant it was possible to have code which bound over named generics because it treated them wrongly as rebindable TypeVariables.
This meant the compiler previously accepted

fn bad_id<N, M>(a: [Field; N]) -> [Field; M] { a }

and now it correctly errors.

[jfecher]

Alright, I've added support for parsing numeric generic expressions within the normal generic brackets <> as well. This likely should have been included from the start but I was worried how messy it would be to duplicate the expression parser. The final solution I landed on was adding a boolean parameter to parse_expression_with_precedence to indicate whether it should parse a normal expression or only the restricted syntax valid within type expressions. Other expression functions like term are still partially duplicated, though with most choices removed.

[okuyiga]

Does this PR contemplate support for numeric generics in implementations. I'm working on a bigint library and am particularly interested in support for something like this

struct BigInt<N> {
    limbs : [u32; N],
}

impl BigInt<N> {
    fn mul(self, other : BigInt<N>) -> BigInt<N * 2> {
        // ...
    }
}

Also this

fn loop_example<N>() -> Field {
    let mut sum = 0;
    for i in 0..N {
        sum = sum + i;
    }
    sum
}

[jfecher]

Does this PR contemplate support for numeric generics in implementations?

Unfortunately not, impls currently do not support generics on the impl at all. This is planned though, and once generics on impls are supported numeric generics should come naturally with that. As for your specific examples I'll break them down a bit on what is supported and what is not:

// Structs generic over unsigned integer numerals will be entirely supported as of this PR
struct BigInt<N> {
    limbs : [u32; N],
}

// Generics on impls are not currently supported, but are planned for a later date
impl BigInt<N> {

    // Arithmetic on generic parameters (N here) is currently unsupported officially.
    // This PR provides the beginnings of support for them but offering full support would
    // require full unification support for these which would require a symbolic expression solver.
    // I'd like to experiment with expanding support for this in the future but this feature may be
    // unnecessary if Noir ever gets true array slice types `[T]` with no hidden size parameter.
    fn mul(self, other : BigInt<N>) -> BigInt<N * 2> {
        // ...
    }
}

fn loop_example<N>() -> Field {
    let mut sum = 0;
    
    // Using numeric generics as expressions themselves is also not supported. You can accomplish
    // something similar however by passing in N as a comptime Field parameter instead. If you have an
    // array type parameterized over N you can also extract it into a comptime Field via std::array::len.
    // I may work on making numeric generics directly useable within expressions eventually but it is
    // admittedly not the highest of priorities since there are workarounds present.
    for i in 0..N {
        sum = sum + i;
    }
    sum
}

[guipublic]

It seems that there are a lot of similar code and I am wondering if we could not try to unify somehow comptime and numeric generics in a new type which gets to become evaluated after monomorphisation?
I believe this new type would not be restricted to u64 and could be another integer size or fieldelement (of course it will be u64 is used for array len).
If you think it is a good idea but it would be a significant change, we should do it in a separate PR.
If you think it is a bad idea, we can forget about it.

[jfecher]

Just added a commit removing the experimental feature of allowing arithmetic on generic integers like N+1 in fn grow_array<N>(array: [Field; N]) -> [Field; N+1] { ... }. This requires a full symbolic arithmetic solver if we truly want to support it.
It was originally added in this PR as an experiment and not officially supported, but users may run into it and file bugs when it was not as well supported as they hoped, so it has been removed from this PR for safety.
In the future, I think functions like grow_array would be better written via unsized slice/Vec types as encoding increasingly more information in types would naturally lead to dependent types which is not a goal of noir.

[guipublic]

It's OK for me, I agree that trying to do more has some implications so it's a good first step for now.