Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use whole-program code-generation for all backends #712

Merged
merged 15 commits into from
May 20, 2023

Commits on May 19, 2023

  1. Configuration menu
    Copy the full SHA
    ccb2b75 View commit details
    Browse the repository at this point in the history
  2. make vmbackend compile again

    The `ModuleList` used by the collector pass is now no longer re-used in
    the VM backend, but instead translated to a custom representation (see
    `produceModules`).
    
    In addition, the list with the top-level AST is now discarded after it's
    passed to code generation, which should reduce memory pressure, and thus
    peak memory usage, quite a bit.
    zerbina committed May 19, 2023
    Configuration menu
    Copy the full SHA
    d1401b8 View commit details
    Browse the repository at this point in the history
  3. containers: extend the APIs

    Extend the set of routines for interacting with `Store` and `SeqMap`.
    zerbina committed May 19, 2023
    Configuration menu
    Copy the full SHA
    274a730 View commit details
    Browse the repository at this point in the history
  4. collectors: simpler module list management

    Store the modules in a `SeqMap` with their position as the key. This
    conveys intention better and leaves it to the consumer to use a more
    fitting structure/representation.
    
    For the internal representation, `vmbackend` uses a `Store` for the
    list of modules, which makes the intention more clear and querying the
    list a bit more ergonomic.
    zerbina committed May 19, 2023
    Configuration menu
    Copy the full SHA
    eb9198f View commit details
    Browse the repository at this point in the history
  5. add a C code-generation orchestrator

    Much like how it works for the VM backend, the C backend now also uses
    an orchestrator (the new `cbackend` module) that invokes the code
    generator.
    
    The `passes` integration (`myOpen`, `myProcess`, and `myClose`) is
    removed from `cgen`. Setting up the extra header backend module (used
    for C header generation) is now the responsibility of the orchestrator.
    
    Similar to `vmbackend`, the new `cbackend` also operates on the
    semantically analyzed AST of the whole program.
    zerbina committed May 19, 2023
    Configuration menu
    Copy the full SHA
    c01f53f View commit details
    Browse the repository at this point in the history
  6. collectors: track a module's IdGenerator

    The upcoming C code-generation orchestrator needs access to the
    `IdGenerator` for each module. They're now stored with `FullModule`,
    allowing the orchestrator to retrieve them later.
    
    `vmbackend` is adjusted to make use of the module's `IdGenerator`, which
    is more precise than using the `ModuleGraph`'s one. Ideally, the backend
    should not introduce any new symbol and type instances (which is what
    the `IdGenerator`s are required for), but `transf` currently
    necessitates that.
    zerbina committed May 19, 2023
    Configuration menu
    Copy the full SHA
    a847b9e View commit details
    Browse the repository at this point in the history
  7. cgen: remove handling of forwarded procedures

    Since code generation now only takes place *after* the whole program was
    semantically analyzed, forwarded procedures and unresolved borrows no
    longer reach there.
    zerbina committed May 19, 2023
    Configuration menu
    Copy the full SHA
    4e54bcf View commit details
    Browse the repository at this point in the history
  8. jsgen: remove inSystem

    Disabling stack-traces for the system module and all modules it imports
    differs from what the C code-generator does (stack-traces are only
    disabled for the `system` module there, but not for the ones it
    imports).
    
    The required access to `PGlobals` via `g.backend` is also a small
    problem for upcoming `jsgen` refactoring.
    zerbina committed May 19, 2023
    Configuration menu
    Copy the full SHA
    ae1d9a5 View commit details
    Browse the repository at this point in the history
  9. add a JS code-generation orchestrator

    Very similar to the introduction of an orchestrator for the C code-
    generator, but for the JS code-generator.
    
    The `passes` integration is removed form `jsgen`, and writing the module
    to disk and generating the source map moved to the orchestrator.
    zerbina committed May 19, 2023
    Configuration menu
    Copy the full SHA
    73466ce View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    a967e36 View commit details
    Browse the repository at this point in the history
  11. collectors: don't drop declarative nodes

    The pass should only collect the statements into a list, and not
    introduce its own decision making.
    zerbina committed May 19, 2023
    Configuration menu
    Copy the full SHA
    bb74161 View commit details
    Browse the repository at this point in the history
  12. pass the ModuleList to the generateCode routines

    Instead of extracting the `ModuleList` as part of `generateCode`, it is
    now passed in as a `sink` parameter. This:
    * makes it easier to move away from storing the `ModuleList` as part of
      the `ModuleGraph`
    * moves the mutation to the callsite
    * allows for implementing the memory consumption optimization employed
      by `vmbackend` (which is eventually going to be used for the other
      backends too) in a much cleaner way
    zerbina committed May 19, 2023
    Configuration menu
    Copy the full SHA
    0cdfe7b View commit details
    Browse the repository at this point in the history
  13. tests: disable parts of the toverflw.nim test

    With code generation now always happening after all semantic analysis is
    done, option changes applied via the `push` and `pop` pragmas no longer
    apply to top-level statements.
    
    As a temporary solution, the feature could be made to work by processing
    `nkPragma` nodes in the orchestrators, but the planned upcoming changes
    would render this approach unusable again.
    zerbina committed May 19, 2023
    Configuration menu
    Copy the full SHA
    4c317cf View commit details
    Browse the repository at this point in the history

Commits on May 20, 2023

  1. address review comments

    Thank you for the review, @saem .
    zerbina committed May 20, 2023
    Configuration menu
    Copy the full SHA
    2d8e350 View commit details
    Browse the repository at this point in the history
  2. cbackend: clarify the "close module" comment

    The comment was misleading, as a backend module is not closed for
    writing after the call to `finalCodegenActions`.
    zerbina committed May 20, 2023
    Configuration menu
    Copy the full SHA
    6757a31 View commit details
    Browse the repository at this point in the history