Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[major] Add public modules #130

Merged
merged 1 commit into from
Nov 1, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
115 changes: 80 additions & 35 deletions abi.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,30 +36,68 @@ may evolve as new FIRRTL constructs are added.

## On Modules

### The Circuit and Top Level
### The Circuit

The top level module, as specified by the circuit name, shall be present as a
System Verilog module of the same name. This FIRRTL module is considered a
"public" module and subject to the lowering constraints for public modules.
The circuit is a container of public modules. The circuit, by itself, does not
have any ABI.

### External Modules

An external module may be presumed to exist following the lowering constraints
for public modules. The module shall exist with a verilog name matching the
defname value or, lacking that, the module name.
for public modules. The module shall exist with a Verilog name matching the
defname value or, lacking that, the FIRRTL module name. The ports of an
external module shall adhere to one of the port lowering ABIs.

### Public Modules
No assumption is made of the filename of an implementation of an external
module. It is the user's job to include files in such a way in their tools to
resolve the name.

Any module considered a "public" module shall be implemented in Verilog in a
consistent way. Any public module shall exist as a Verilog module of the same
name.
### Public Modules

Each public module with definitions (e.g. not external modules) shall be placed
in a file with the same name. No assumption is made of the filename of an
implementation of an external module, it is the user's job to include files in
such a way in their tools to resolve the name.
All public modules shall be implemented in Verilog in a consistent way. All
public modules shall exist as Verilog modules of the same name. Each public
module shall be placed in a file with a format as follows where `module` is the
name of the public module:

### Port Lowering ABIv1
``` ebnf
filename = module , ".sv" ;
```

Each public module in a circuit shall produce a filelist that contains the
filename of the file containing the public module and any necessary files that
define all public or private module files instantiated under it. Files that
define external modules are not included. This filelist shall have a name as
follows where `module` is the name of the public module:

``` ebnf
filelist_filename = "filelist_" , module , ".f" ;
seldridge marked this conversation as resolved.
Show resolved Hide resolved
```

The filelist contents are a newline-delimited list of filenames.

The ports of public modules shall be lowered using one of the Port Lowering ABIs.

No information about the instantiations of a public module may be used to
compile a public module---a public module is compiled as if it is never
instantiated. However, it is legal to compile modules which instantiate public
modules with full knowledge of the public module internals.

### Private Modules

Private modules have no defined ABI. If the compilation of private modules
produces files, then those files shall be included when necessary in public
module filelists.

FIRRTL compilers shall mangle the names of private modules that are not removed
by compilation. The mangling scheme is implementation defined. This is done to
avoid name collisions with the private modules produced by other compilations.

### Port Lowering ABIs

There are two supported port lowering ABIs. These ABIs are applicable to public
modules or external modules only.

#### Port Lowering ABIv1

Ports are generally lowered to netlist types, except where Verilog's type system
prevents it.
Expand All @@ -68,8 +106,8 @@ Ports of integer types shall be lowered to netlist ports (`wire`{.verilog}) as a
packed vector of equivalent size. For example, consider the following FIRRTL:

```FIRRTL
circuit Top :
module Top :
circuit :
public module Top :
output out: UInt<16>
input b: UInt<32>
```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it intended non-main modules should be something a separate FIRRTL file can make use of via, say, extmodule?

Since there's no ability to specify the circuit name for an extmodule (the circuit the module is "imported" from?) ABI relying on circuit name across boundaries are unable to match up. In particular I mean the ref ABI in the text below. This is an issue an practice and would be good to fix as part of this.

Since we're requiring the public module name to be the verilog name , and only really "link" at Verilog level, it seems any namespacing provided by including the circuit name is not useful in practice as there would be a naming conflict of Verilog modules already (although I do like the idea, esp leaving door open for such).

While this ABI is clear about a single unit ("what is required/can be expected re:Verilog output"), it doesn't really say how a FIRRTL compiler is supposed to make use of this on the instantiation-side from a separate FIRRTL (or anything else wishing to make use of a FIRRTL-ABI-built "circuit"), so maybe there's some room here-- in addition to how they're used ("module name" needs to be "defname" in practice) it's also unspecified how they're gathered/included such that the macros are available reliably (left to build flow)... (not unreasonably so, bear with me please)...

One could imagine requiring telling the build flow/driver/linker+compiler that module X is from circuit Y or something (since build flow knows what circuits it built anyway, even if individual builds either don't have that information or we'd like to be able to change the definitions being used (so don't want to bake circuit name in the code)?), leaving circuit name in the filenames for co-located multiple implementations of a thing but macros are just ref_<module>_ as within any single "linked" design the module name must be unique.

Similar but maybe simpler possibility is to just drop the circuit portion of the ref ABI (ref_<module> prefix for files/macros), or if wishing backwards-compat with things built using ABI in current document, then treat all modules as being within their own circuit ref_<module>_<module>.

I'll think some more, maybe we should discuss a bit. Thanks.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes a lot of sense. Thanks for writing this up. It took me a couple of days to process. 😅 😓

There's two things here:

  1. To properly handle linking across a boundary, only the module name should be used. This implies that some extmodule Foo should match up to some public module Foo in another compilation or it's Verilog/VHDL equivalent. This shouldn't specify the circuit name as the circuit where this comes from is irrelevant. It's up to the user to figure this out and link it properly.
  2. There is a namespacing problem with private modules. At minimum, these should have the circuit name included. At best, these should use some name mangling which cannot be linked from another FIRRTL compilation. E.g., Verilog allows for $ in module names, but the FIRRTL spec disallows this. We could use this fact to generate a unique private module name which cannot conflict with something else like <circuit>$<module>. It's not pretty, but would work. The complication here is that if the spec isn't silent about this, then it imposes an implicit private module ABI which could be used to link things silently (except that the private ABI can't be accessed from FIRRTL due to the use of $?).

WDYT?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only problem with using the bare module name without the circuit for this is that conflicts are high. clang/gcc avoid this on the C++ side with more complicated name mangling. We could do something like that that includes the names of the ports and their types in a mangled way. This would be super ugly... Perhaps its best to use the module name, but expect/encourage people to use names that are prefixed intelligently and/or use very descriptive names.

Copy link
Member Author

@seldridge seldridge Sep 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added language that makes things more "public module-centric" in ce8aa58. Primarily, this means that the circuit name has no real effect anymore. The only place I see that being used in in the implementation-defined convention for private modules.

E.g., consider the following circuit:

circuit Foo:
  module Baz:

  public module Bar:
    inst baz of Baz

This must produce a Verilog module called Bar in a file Bar.sv. Any additional stuff (ref files, bind files) are all now created using the name Bar and ignoring Foo. This then actually makes this portable to any other compilation referring to Bar as an extmodule.

If the compiler produces a Verilog module for FIRRTL module Baz, it should mangle its name to avoid a collision with another Verilog module. Some examples/ideas:

  • Foo_Baz (simple, to the point)
  • _Foo_Baz
  • Foo$Baz (has the benefit of being legal in FIRRTL, but not in Verilog)

The primary benefit of using the circuit name here is that it is trivial to change without an effect on the rest of the design now. This then fills one of, though not all of, the rolls of the NestedPrefixModulesAnnotation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sounds great!! I don't know how to "manage" the versioning details here -- I'd kinda thought the ABI was mostly versioned orthogonal to the FIRRTL spec, so this change should be ABI v3 and v4 or something?

Whatever seems best, I'm not sure how important that is presently, or if I'm missing something.

Anyway I think everything you've changed and your solutions/answers look great and excited for this direction. Thanks! 🚀

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, looks like ABI is versioned with FIRRTL spec ? And v1/v2 are parts of that or something. So probably nevermind 👍.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm glad this makes sense. Thanks for checking the details.

Yes, the ABI document is co-versioned with the FIRRTL spec. However, the actual ABI can have different "versions" within it. ABI v1/v2 is really "lowered aggregates" and "preserved aggregates". This gets a little complicated as maybe we should then do the same thing for the ref and group ABIs given that this patch makes a change to them. I think we can dodge this and change it with the co-versioned version.

I.e., if you are compiling FIRRTL less than version 3 then you should be generating the files with a certain format. If you are compiling FIRRTL version 4 or greater, then you should generate the new format. I would like to avoid this complexity in CIRCT and do a hard switch to version 4. If we have to, we can add in support for version 3 if there is a use case. However, I'm not planning to add it.

Expand All @@ -86,16 +124,16 @@ module Top(
Ports of aggregate type shall be scalarized according to the "Aggregate Type
Lowering" description in the FIRRTL spec.

Ports of ref type shall be lowered to a Verilog macro of the form `` `define
ref_<circuit name>_<module name>_<portname> <internal path from module>`` in a
file with name `ref_<circuit name>_<module name>.sv`. References to aggregates
will be lowered to a series of references to ground types. This ABI does not
specify whether the original aggregate referent is scalarized or not.
Ports of ref type on public modules shall, for each public module, be lowered to
a Verilog macro of the form `` `define ref_<module name>_<portname> <internal
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we should drop the circuit name. More generally, this would require extmodules to be in a separate circuit and we would need to be able to instantiate modules in other circuits.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we have the circuit name -- and no way to declare the circuit extmodules are in -- how does this work? That's why I think dropping circuit name is helpful, but mostly I don't understand what you're saying ("this would require" -- which are you saying requires that--including circuit or not?).

Are you saying you think we should put extmodules in separate circuit (so that we know what circuit-name the ABI will put their macro under? (and compatibility limits use of one compilation unit to "linking" against definitions of that module that use the "expected" circuit name, not any definition?)), or that we don't want to do that?

If so, how does one use a public module (that isn't top) with probes from another FIRRTL circuit? What does the extmodule look like and how do we find the right ABI bits to use? Thanks!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or maybe an alternate path is to drop the name from the circuit? Does it really provide any value anymore?

Basically, the question is does the circuit name matter for anything anywhere? If it does, it seems we should embed it in the ABI so as to avoid name conflicts when one circuit uses names from another. This obviously doesn't happen now, but it seems like it should. Multiple public modules provides most of what you need to have libraries in a namespace. If it doesn't matter, then sure, drop it all over.

The "would require" thing is referring to specifying the circuit extmodules are in, if circuit is an ABI-affecting namespace.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's drop it. (Do we go a step further and drop circuits entirely? Probably no, but it's an interesting consideration.) If we don't then it's a hint to the compilation on how to prefix or mangle. However, this is weird as it isn't exactly specified in the ABI.

If we instead remove the name then a prefix can be later added as an optional attribute of the circuit and done in a more exact ABI way.

Other questions/design points: can a circuit be nested or should we design with this in mind? This would make circuit more like a namespace and would probably require dropping the top level circuit. This would require circuit names.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Circuit names are dropped in 1b7f50b. This also removes the redundancy between an annotation without a target and an annotation targeting the circuit.

path from module>`` in a file with name `ref_<module name>.sv`. References to
aggregates will be lowered to a series of references to ground types. This ABI
does not specify whether the original aggregate referent is scalarized or not.

All other port types shall lower according ot the type lowering in
section ["On Types"](#On-Types).

### Port Lowering ABIv2
#### Port Lowering ABIv2

Ports are lowered per the v1 ABI above, except for aggregate types.

Expand Down Expand Up @@ -137,30 +175,37 @@ used by a nested group will create output ports because SystemVerilog disallows
the use of a bind instantiation underneath the scope of another bind
instantiation. The names of additional ports are implementation defined.

For each optional group, one binding file will be produced for each group or
nested group. The circuit, group name, and any nested groups are used to derive
a predictable filename of the binding file. The file format uses the format
below:
For each optional group and public module, one binding file will be produced for
each group or nested group. The public module, group name, and any nested
groups are used to derive a predictable filename of the binding file. The file
format uses the format below where `module` is the name of the public module,
`root` is the name of the root-level group and `nested` is the name of zero or
more nested groups:

``` ebnf
filename = "groups_" , circuit , "_", root , { "_" , nested } , ".sv" ;
filename = "groups_" , module , "_", root , { "_" , nested } , ".sv" ;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comment as before, groups_circuit_module seems safer.

```

As an example, consider the following circuit with three optional groups:

``` firrtl
circuit Foo:
circuit:
declgroup Group1, bind:
declgroup Group2, bind:
declgroup Group3, bind:
public module Bar:
public module Baz:
```

When compiled to Verilog, this will produce three bind files:
When compiled to Verilog, this will produce six bind files:

```
groups_Foo_Group1.sv
groups_Foo_Group1_Group2.sv
groups_Foo_Group1_Group2_Group3_.sv
groups_Bar_Group1.sv
groups_Bar_Group1_Group2.sv
groups_Bar_Group1_Group2_Group3_.sv
groups_Baz_Group1.sv
groups_Baz_Group1_Group2.sv
groups_Baz_Group1_Group2_Group3_.sv
```

The contents of each binding files must have the effect of including all code
Expand All @@ -178,7 +223,7 @@ module `Bar`. Both `Foo` and `Bar` contain groups. To make the example
simpler, no constant propagation is done:

``` firrtl
circuit Foo:
circuit:
declgroup Group1, bind:
declgroup Group2, bind:

Expand All @@ -197,7 +242,7 @@ circuit Foo:
node notNotA = not(notA)
define _notNotA = probe(notNotA)

module Foo:
public module Foo:
inst bar of Bar

group Group1:
Expand Down
1 change: 1 addition & 0 deletions include/firrtl.xml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
<item>invalidate</item>
<item>intrinsic</item>
<item>propassign</item>
<item>public</item>
</list>
<list name="types">
<item>UInt</item>
Expand Down
5 changes: 5 additions & 0 deletions revision-history.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,12 @@ revisionHistory:
# additions to the specification should append entries here.
thisVersion:
spec:
- Add public modules. Change circuits to support any number of public
modules which may have disjoint instance hierarchies.
abi:
- Add ABI for public modules and filelist output.
seldridge marked this conversation as resolved.
Show resolved Hide resolved
- Changed ABI for group and ref generated files. These now use the public
module and not the circuit.
# Information about the old versions. This should be static.
oldVersions:
- version: 3.2.0
Expand Down
Loading