better dependent-expression support for range types

[zerbina]

Summary

Fix complex type-variable-dependent expressions not working reliably
for range and array-index types used in routine signatures.
Internally, expressions in both contexts are now first treated as
generic, and using replaceTypesInBody for expressions with
potentially unbound type variables is removed.

Language changes:

• syntax macros and templates (i.e., those where all parameters are
  untyped) used in range- and index-type expression are expanded
  before all further analysis (and thus typed macros/template, static
  expressions, etc.)
• typed macros and templates used in range- and index-type expressions
  depending on type variables are potentially expanded multiple times

Fixes:

• type-variable-dependent range and array-index types in routine
  signatures can now be arbitrarily complex (refer to the added tests
  for an example of what is now possible)

Details

Overview:

• all expressions used with range types and in array-index positions
  now use the generic pre-pass
• type-variable-dependent expressions in range type constructors are
  now always generic expressions (i.e., symbols are looked up, but
  not AST is not typed)
• replaceTypesInBody is renamed to instantiateTypesInBody and
  support for unresolved type variables is removed -- the new
  replaceTypeVarsInBody now has to be used for that case

Semantic analysis has very basic, scattered support for expressions
depending on unresolved type variables, allowing semExprWithType to
be used for typing expressions in range and array type constructors.
However, in more complex cases, this usually results in errors with
unclear descriptions. Until more complete support for typing expressions
depending on unresolved type variables is implemented, it is more
robust to treat them as fully generic.

Analyzing the expressions

Both semRangeAux and semArrayIndex now always run the generic pre-
pass first (semGenericStmt, which also expands syntax macros), which
is required for detecting whether unresolved type variables are used.
When no unresolved type variables exist in the analyzed expression, it
is directly passed on to semExprWithType.

The procedure for querying whether a symbol is that of an unresolved
type variable (isUnresolvedSym), is moved to the ast_query
module.

Looking for unresolved type variables is combined with making sure that
all type variables are referenced in the AST via their symbols
(fixupTypeVars) -- something that is not always already the case. Some
type variables not having their symbol bound would mean that the later
pass for replacing type variables cannot detect them.

As an unintended side-effect of using semGenericStmt, unrelated
symbols are marked as used, which is what causes the different error
message in tgeneric_backtrack_type_inference.nim (the presence of the
sfUsed flag for generic parameters of routines decides whether a
routine is marked with tfUnresolved).

Processing the generic expressions

For processing generic expressions used in the context of static
parameter inference, where unbound type variables are possible, the
replaceTypeVarsInBody procedure is introduced. While using
replaceTypesInBody with allowMetaTypes = true would have sufficed,
this is intended as a step towards removing the "meta-types allowed"
mode from semtypinst.

Implementation-wise, replaceTypeVarsInBody is similar to prepareNode,
with the differences being that replaceTypeVarsInBody:

• does not instantiates or traverses into any types
• only considers symbol nodes referring to type variables

Expression already having a type are not analyzed again when used as call
operands, and thus two compromises are required for the generic-expression
approach to work:

• both prepareNode and replaceTypeVarsInBody have to mirror
  how semSym assigns types to symbol nodes of type variables (fixType
  implements this)
• replaceTypeVarsInBody has to remove from the expression the types
  previously assigned by both semRangeAux and semArrayIndex

replaceTypesInBody is renamed to instantiateTypesInBody. In addition,
the unnecessarily exported replaceTypeVarsN is un-exported.

Misc

• restore the name of the tgenericshardcases test, which was
  erroneously changed to tsharedcases during a test suite cleanup

[zerbina]

The PR message was quite the mess, sorry for that. I've cleaned it up a bit, and hopefully it's easier to read and parse now.

Looking ahead, the new fixupTypeVars, if merged, would be a great help in fixing the issues with when statements inside generic objects (they too use semExprWithType at the moment).

[saem]

The way it works now is quite a bit better. Only some non-blocking suggestions.

Test coverage seems good. I tried to come up with some meaningful range tests to suggest, but nothing particularly novel occurred to me. The best I could come up with is:

proc r(t: typedesc, rng: range[-typeNameLen(t), typeNameLen(t]): int =
  result = rng.type.high - rng.type.low

doAssert r(MyType, 0) == 12

[saem]

Suggested change

	## Computes the correct type to assign to the symbol node of a replaced
	## type variable.
	## Computes the correct type to assign to the symbol of a replaced type
	## variable.

not sure if that's too pendantic, but we're technically working with the symbol and not an nkSym.

[zerbina]

Yep, we're working with the symbol, but the returned type is meant to be assigned to the node, hence the explicit mention.

[saem]

this is really nice.

[saem]

The PR message was quite the mess, sorry for that. I've cleaned it up a bit, and hopefully it's easier to read and parse now.

thanks for the additional edits, in fairness i found the earlier version was already pretty good, but the improvement is appreciated all the same.

Looking ahead, the new fixupTypeVars, if merged, would be a great help in fixing the issues with when statements inside generic objects (they too use semExprWithType at the moment).

considering i'd like to lean on when and generics a whole lot more (reducing macro usage) that's a win all around. :D

---

Awesome PR.

[zerbina]

Thanks a lot for the review, @saem.

Adding a test for range was a good call, as it uncovered unrelated issues with typeRel for range types. They are not hard to fix, but I think it'd be better to do so in a follow-up PR (I've marked the added test as a known issue).

[zerbina]

/merge

[github-actions]

Merge requested by: @zerbina

Contents after the first section break of the PR description has been removed and preserved below:

---

Notes for Reviewers

• the original goal was only to remove the "meta-types allowed" mode from replaceTypesInBody, but the PR effectively turned into a partial reboot of sem: simplify generic parameter symbol handling #644
• the changes are a preparation for multiple other improvements to generic symbol handling, sigmatch, and semtypinst