-
Notifications
You must be signed in to change notification settings - Fork 39
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
use whole-program code-generation for all backends (#712)
## Summary Change the C and JavaScript backends to use whole-program code- generation in the same way that the VM and IC backends do. In short, this means that code generation is now only run after *all* modules part of the program were semantically analyzed. This brings the architecture of the C and JS backends closer to that of the VM backend, and is the first step towards unifying the backend processing. The final goal is to have all backends process code in the same way, with as much as possible of the pre-processing currently performed by the code generators being moved to a shared, backend-agnostic layer. ## Details Generalize the `ModuleGraph` pass that the VM backend used for gathering the AST of each alive module, and move it into its own module. Compared to the original implementation, the generalized pass: * doesn't drop declarative nodes (the VM backend doesn't care about them, but the C and JS backends do) * remembers the `IdGenerator` associated with each module * doesn't introduce its own module order (that's left to the backend) * uses the more descriptive `SeqMap` instead of a raw `seq` `vmbackend` is adjusted to work with the generalized collection pass: the module list produced by the pass is translated into the structure that the rest of the VM backend still expects, but a `Store` with a dedicated ID type is now used instead of a raw `seq` (preparing for future improvements). Using the same naming scheme as `vmbackend`, the modules `cbackend` and `jsbackend` are introduced -- they implement the code-generation orchestrators for the C and JavaScript backends, respectively. The `passes` integration is removed from `cgen` and `jsgen`, as invoking the code generators is now the responsibility of the orchestrators. The new orchestrators take the module list produced by the collector pass and generate the code for it (with `jsbackend` also writing the output to disk already). They're very basic at the moment, but in the future will take on similar responsibilities as `vmbackend` does for the VM backend (e.g., dead-code elimination, running `transf`, etc.). Since forwarded procedures don't reach the code generators anymore, the special handling for them is removed. ### Known Issues Changes to options performed by `.push` having an effect on top-level code relied on semantic analysis and code generation happening in a pipelined manner. Since this is no longer the case, disabling or enabling checks for top- level code stops working for now, but the plan is to bring this feature back in the future.
- Loading branch information
Showing
11 changed files
with
397 additions
and
239 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| ## The code-generation orchestrator for the C backend. It generates the C code | ||
| ## for the semantically analysed AST of the whole progam by invoking ``cgen``. | ||
| ## | ||
| ## The general direction is to move more logic out of the code generator (such | ||
| ## as figuring out the set of alive procedures) and into the orchestrator, | ||
| ## leaving only the core of code generation to ``cgen``. | ||
|
|
||
| import | ||
| compiler/ast/[ | ||
| ast | ||
| ], | ||
| compiler/backend/[ | ||
| cgen, | ||
| cgendata, | ||
| collectors, | ||
| extccomp | ||
| ], | ||
| compiler/front/[ | ||
| options | ||
| ], | ||
| compiler/modules/[ | ||
| modulegraphs | ||
| ], | ||
| compiler/utils/[ | ||
| containers, | ||
| pathutils | ||
| ] | ||
|
|
||
| from compiler/sem/passes import skipCodegen | ||
|
|
||
| proc generateCode*(graph: ModuleGraph, mlist: sink ModuleList) = | ||
| ## Entry point for C code-generation. Only the C code is generated -- nothing | ||
| ## is written to disk yet. | ||
| let | ||
| config = graph.config | ||
|
|
||
| var g = newModuleList(graph) | ||
|
|
||
| # first create a module list entry for each input module. This has to happen | ||
| # *before* the code generator is invoked. | ||
| for key, val in mlist.modules.pairs: | ||
| let m = newModule(g, val.sym, config) | ||
| m.idgen = val.idgen | ||
|
|
||
| # setup the module for the generated header, if required: | ||
| if optGenIndex in config.globalOptions: | ||
| let f = if config.headerFile.len > 0: AbsoluteFile config.headerFile | ||
| else: config.projectFull | ||
| g.generatedHeader = rawNewModule(g, mlist.modules[config.projectMainIdx2].sym, | ||
| changeFileExt(completeCfilePath(config, f), hExt)) | ||
| incl g.generatedHeader.flags, isHeaderFile | ||
|
|
||
| # the main part: invoke the code generator for all top-level code | ||
| for index in mlist.modulesClosed.items: | ||
| let | ||
| m {.cursor.} = mlist.modules[index] | ||
| bmod = g.modules[index.int] | ||
|
|
||
| # pass all top-level code to the code generator: | ||
| for it in m.stmts.items: | ||
| if not skipCodegen(bmod.config, it): | ||
| genTopLevelStmt(bmod, it) | ||
|
|
||
| # wrap up the main part of code generation for the module. Note that this | ||
| # doesn't mean that they're closed for writing; invoking the code generator | ||
| # for other modules' code can still add new code to this module's sections | ||
| finalCodegenActions(graph, g.modules[index.int], newNode(nkStmtList)) | ||
|
|
||
| # the callsite still expects `graph.backend` to point to the ``BModuleList`` | ||
| # so that ``cgenWriteModules`` can query it | ||
| # XXX: this is the wrong approach -- the code generator must not be | ||
| # responsible for writing the generated C translation units to disk. | ||
| graph.backend = g |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,82 @@ | ||
| ## Implements the "collect" pass. This pass gathers the full AST for each | ||
| ## module into a single structure, which is then meant to be consumed by the | ||
| ## code-generation orchestrators (``cbackend``, ``jsbackend``, etc.). | ||
| ## | ||
| ## This is somewhat similar to the rodfile-based IC backend, but instead of | ||
| ## reading the modules' content from the rodfiles, it's collected via the pass | ||
| ## interface. | ||
|
|
||
| import | ||
| compiler/ast/[ | ||
| ast, | ||
| ast_idgen, | ||
| ast_types, | ||
| lineinfos | ||
| ], | ||
| compiler/modules/[ | ||
| modulegraphs | ||
| ], | ||
| compiler/sem/[ | ||
| passes | ||
| ], | ||
| compiler/utils/[ | ||
| containers | ||
| ] | ||
|
|
||
| type | ||
| FullModule* = object | ||
| stmts*: seq[PNode] ## top level statements in the order they were parsed | ||
| sym*: PSym ## module symbol | ||
| idgen*: IdGenerator | ||
|
|
||
| ModuleListRef* = ref ModuleList | ||
| ModuleList* = object of RootObj | ||
| modules*: SeqMap[FileIndex, FullModule] | ||
| modulesClosed*: seq[FileIndex] | ||
| ## stores the modules in the order they were closed. The first closed | ||
| ## module comes first, then the next, etc. | ||
|
|
||
| ModuleRef = ref object of TPassContext | ||
| ## The pass context for the VM backend. Represents a reference to a | ||
| ## module in the module list | ||
| list: ModuleListRef | ||
| index: FileIndex | ||
|
|
||
| func isFilled*(m: FullModule): bool = | ||
| # required so that ``FullModule`` is usable as the item type of a ``SeqMap`` | ||
| m.sym != nil | ||
|
|
||
| proc takeModuleList*(graph: ModuleGraph): ModuleList = | ||
| ## Moves the ``ModuleList`` set up by the collector pass out of the | ||
| ## `graph.backend` field and returns it. | ||
| result = move ModuleListRef(graph.backend)[] | ||
| graph.backend = nil | ||
|
|
||
| # Below is the `passes` interface implementation | ||
|
|
||
| proc myOpen(graph: ModuleGraph, module: PSym, idgen: IdGenerator): PPassContext = | ||
| if graph.backend == nil: | ||
| graph.backend = ModuleListRef() | ||
|
|
||
| let | ||
| mlist = ModuleListRef(graph.backend) | ||
| pos = module.position.FileIndex | ||
|
|
||
| # add an empty entry for the module: | ||
| mlist.modules[pos] = FullModule(sym: module, idgen: idgen) | ||
|
|
||
| result = ModuleRef(list: mlist, index: pos) | ||
|
|
||
| proc myProcess(b: PPassContext, n: PNode): PNode = | ||
| result = n | ||
| let m = ModuleRef(b) | ||
|
|
||
| m.list.modules[m.index].stmts.add(n) | ||
|
|
||
| proc myClose(graph: ModuleGraph; b: PPassContext, n: PNode): PNode = | ||
| result = myProcess(b, n) | ||
|
|
||
| let m = ModuleRef(b) | ||
| m.list.modulesClosed.add(m.index) | ||
|
|
||
| const collectPass* = makePass(myOpen, myProcess, myClose) |
Oops, something went wrong.