feat: implement "stateless" modules

[charles-cooper]

resolves #2431, #2670, #3294

partially implements .vyi files #2814 (but does not handle ellipsis for bytestring/dynarray length, that will come in a follow up PR)

partially finishes #2852 (although it does not automatically export events in the ABI)

What I did

How I did it

How to verify it

Commit message

this commit implements support for "stateless" modules in vyper.

this is the first major step in implementing vyper's module system.
it redesigns the language's import system, allows calling internal
functions from imported modules, allows for limited use of types from
imported modules, and introduces support for `.vyi` interface files.

note that the following features are left for future, follow-up work:
- modules with variables (constants, immutables or storage variables)
- full support for imported events (in that they do not get exported
  in the ABI)
- a system for exporting imported functions in the external interface
  of a contract

this commit first and foremost changes how imports are handled in vyper.

previously, an imported file was assumed to be an interface file. some
very limited validation was performed in `InterfaceT.from_ast`, but not
fully typechecked, and no code was generated for it.

now, when a file is imported, it is checked whether it is
  1. a `.vy` file
  2. a `.vyi` file
  3. a `.json` file

the `.json` pathway remains more or less unchanged. the `.vyi` pathway
is new, but it is fairly straightforward and is basically a "simple"
path through the `.vy` pathway which piggy-backs off the `.vy` analysis
to produce an `InterfaceT` object.

the `.vy` pathway now does full typechecking and analysis of the
imported module. some changes were made to support this:
- a new ImportGraph data structure tracks the position in the import
  graph and detects (and bands) import cycles
- InputBundles now implement a `_normalize_path()` method.
  this method normalizes the path so that source IDs are stable no
  matter how a file is accessed in the filesystem (i.e., no matter
  what the search path was at the time `load_file()` was called).
- CompilerInput now has a distinction between `resolved_path` and `path`
  (the original path that was asked for). this allows us to maintain UX
  considerations (showing unresolved paths etc) while still having a
  1:1:1 correspondence between source id, filepath and filesystem.

these changes were needed in order to stabilize notions like "which file
are we looking at?" no matter the way the file was accessed or how it
was imported. this is important so that types imported transitively
can resolve as expected no matter how they are imported - for instance,
`x.SomeType` and `a.x.SomeType` resolving to the same type.

the other changes needed to support code generation and analysis for
imported functions were fairly simple, and mostly involved generalizing
the analysis/code generation to type-based dispatch instead of AST-based
dispatch.

other changes to the language and compiler API include:
- import restrictions are more relaxed - `import x` is allowed now
  (previously, `import x as x` was required)
- change function labels in IR
  function labels are changed to disambiguate functions of the same name
  (but whose parent module are different). this was done by computing a
  unique function_id for every function and using that function_id when
  constructing its IR identifier.
- add compile_from_file_input which accepts a FileInput instead of a
  string. this is now the new preferred entry point into the compiler.
  its usage simplifies several internal APIs which expect to have
  `source_id` and `path` in addition to the raw source code.
- change compile_code api from contract_name= to contract_path=

additional changes to internal APIs and passes include:
- remove `remove_unused_statements()`
  the "unused statements" are now important to keep around for imports!
  in general, it is planned to remove both the AST expansion and
  constant folding passes as copying around the AST results in both
  performance and correctness problems
- abstract out a common exception rewriting pattern.
  instead of raising `exception.with_annotation(node)` -- just catch-all
  in the parent implementation and then don't have to worry about it at
  the exception site.
- rename "type" metadata key on most top-level declarators to more
  specific names (e.g. "func_type", "getter_type", etc).
- remove dead package pytest-rerunfailures
  use of `--reruns` was removed in c913b2db0881a6
- refactor: move `parse_*` functions, remove vyper.ast.annotation
  move `parse_*` functions into new module vyper.ast.parse and merge in
  vyper.ast.annotation
- rename the old `GlobalContext` class to `ModuleT`
- refactor: move InterfaceT into `vyper/semantics/types/module.py`
  it makes more sense here since it is closely coupled with `ModuleT`.

Description for the changelog

Cute Animal Picture

[codecov-commenter]

Codecov Report

Attention: 97 lines in your changes are missing coverage. Please review.

Comparison is base (7c74aa2) 83.82% compared to head (b0c6684) 83.96%.
Report is 3 commits behind head on master.

Files	Patch %	Lines
vyper/semantics/types/function.py	79.83%	13 Missing and 11 partials ⚠️
vyper/semantics/types/module.py	91.32%	10 Missing and 7 partials ⚠️
vyper/ast/parse.py	71.11%	11 Missing and 2 partials ⚠️
vyper/semantics/analysis/module.py	94.00%	7 Missing and 2 partials ⚠️
vyper/codegen/expr.py	78.94%	2 Missing and 2 partials ⚠️
vyper/compiler/input_bundle.py	92.15%	3 Missing and 1 partial ⚠️
vyper/semantics/analysis/base.py	84.00%	3 Missing and 1 partial ⚠️
vyper/semantics/analysis/utils.py	60.00%	3 Missing and 1 partial ⚠️
vyper/semantics/types/subscriptable.py	73.33%	3 Missing and 1 partial ⚠️
vyper/semantics/analysis/local.py	78.57%	1 Missing and 2 partials ⚠️
... and 7 more

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3663      +/-   ##
==========================================
+ Coverage   83.82%   83.96%   +0.14%     
==========================================
  Files          91       92       +1     
  Lines       12778    13044     +266     
  Branches     2861     2928      +67     
==========================================
+ Hits        10711    10953     +242     
- Misses       1643     1658      +15     
- Partials      424      433       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[charles-cooper]

constant and immutable values which are part of the library module are currently not exposed as members when importing the module. The current scope of this PR is to implement pure/view libraries iiuc. Those values are not part of any state but are part of the runtime bytecode. Thoughts?

i had to cut the scope somewhere and decided to leave those for later since they use the same mechanism as will be used for stateful modules

[pcaversaccio]

Another 2 comments after testing:

1. My first comment is about unused imports and devex (it's related to my issue Warning about unused function parameters and imports #3272). There is no compilation warning issued currently if I import libraries that are not used. I think it would be a good idea to do that for unused imports in order to improve the quality and cleanliness of the code. This must not be part of that PR, but we should not forget about it.
2. I'm wondering whether we should allow nested module calls:

# Lib1.vy

@internal
@view
def blocknumber() -> uint256:
    return block.number + 1

# Lib2.vy

import Lib1

@internal
@view
def blocknumber() -> uint256:
    return Lib1.blocknumber()

# Main.vy

import Lib2

@external
@view
def blocknumber() -> uint256:
    return Lib2.Lib1.blocknumber()

Thoughts?

[pcaversaccio]

Another 2 feedback:

• We should add more tests using Vyper built-in functions.
• The current implementation allows for stateful changes, if you apply some tricks:

# Lib

@internal
@payable
def call(target: address) -> Bytes[32]:
    response: Bytes[32] = raw_call(self, method_id("hello()"), max_outsize=32, value=msg.value)
    return response

# Main.vy

import Lib

var: uint256


@external
@payable
def hello():
    self.var = msg.value

@external
@payable
def call(target: address) -> Bytes[32]:
    return Lib.call(target)

This is more a discussion about the scope of the PR. If it's pure and view libraries, we should disallow raw_call's IMO.

[charles-cooper]

This is more a discussion about the scope of the PR. If it's pure and view libraries, we should disallow raw_call's IMO.

no, "stateless" refers more to the fact that importing variables (of any kind) is not supported, not the mutability properties of the imported functions.

[charles-cooper]

My first comment is about unused imports and devex (it's related to my issue #3272). There is no compilation warning issued currently if I import libraries that are not used. I think it would be a good idea to do that for unused imports in order to improve the quality and cleanliness of the code. This must not be part of that PR, but we should not forget about it.

yes this is probably useful, but i'm declaring it out-of-scope for this PR

[charles-cooper]

I'm wondering whether we should allow nested module calls:

fixed in 64bc3a5

[pcaversaccio]

I'm approving with the following caveat: I have tested some of the logic but not all, and I also haven't reviewed every single line of code. Under normal circumstances, I wouldn't approve the PR, BUT this PR is an important foundational layer for upcoming PRs that need to be worked on. Furthermore, we have to fully audit/review the stateless and stateful modules implementation, and the sooner we get there (hopefully in January) the better. For anyone going through this PR in the future: Please be aware that there might be bugs, but those bugs will be fixed in upcoming PRs. We don't compromise on any security here, but we have to first get the features done in a proper way, and ensure the complete security based on a feature-complete (in that case library modules) state of the compiler.

[fubuloubu]

Still some code quality issues to address, but otherwise looks fine

Happy with @pcaversaccio's manual testing

[charles-cooper]

@pcaversaccio re. builtins see bea05ca