Multiple instantiations in one design

[wwwweb]

Hi there,

is there a recommended way to get multiple independent instantiations of neorv32 in one FPGA design, running different software from their independent IMEM ROMs?

The code base seems to support only one imem instantiation, i.e. having a neorv32_application_image.vhd with a constant application_init_image.
When modifying the constant name/package name dependent on the application, the issue propagates to the imem architecture in neorv32_imem.default.vhd to select the right initialization constant. If selecting this by generic, I cannot just give a constant name as generic (at least I do not know a VHDL way to do this). So actually, I have to propagate the actual constant as generic to neorv32_top and include the according neorv32_application_image on the same hierarchy level as the instance of neorv32_top to use it for generic mapping.

These modifications work but obviously I have to patch through the hierarchy. Is there a better way that I do not see?

[stnolting]

Hey there!

The code base seems to support only one imem instantiation, i.e. having a neorv32_application_image.vhd with a constant application_init_image.

Unfortunately, this is correct. The only way to boot several cores with different code is to make the cores fetch their code from "somewhere" (external flash, a global on-chip memory, etc.; see data sheet section Boot Configuration). I do not think there is a straightforward "out of the box" solution for this yet and we already had a discussion about this a long time ago in #35.

Maybe we could change the VHDL code of the processor-internal IMEM (and the bootloader ROM) to add some simple support for this kind of use cases:

• propagate the HW_THREAD_ID generic down to the IMEM module; this generic should be used to define the unique core ID ("hart IDs) within a multi-core system
• replace the default rtl/core/mem/neorv32_imem.default.vhd by a custom one; this does not impact the code base as the files from rtl/mem are not part of the actual processor core sources (which are located in rtl/core)
• the custom IMEM could use textio to read a plain ASCII HEX file that contains the code for the current core; the file name would be constructed based on the HW_THREAD_ID generic; for example imem_init_core_0.hex for core 0
• the HEX files can be generated via he default application makefile using make hex

I am not sure about this, but I think managing just some HEX files would be way simpler than doing library/package hacks... 🤔

[stnolting]

I just tested the initialization of the IMEM from a HEX file using the following function (works!):

  impure function init_imem_hex(file_name : string; num_words : natural) return mem32_t is
    file     text_file   : text open read_mode is file_name;
    variable text_line_v : line;
    variable mem_v       : mem32_t(0 to num_words-1);
    variable index_v     : natural;
    variable char_v      : character;
    variable data_v      : std_ulogic_vector(31 downto 0);
  begin
    mem_v := (others => (others => '0'));
    index_v := 0;
    while not endfile(text_file) loop
      readline(text_file, text_line_v);
      -- construct one 32-bit word --
      data_v := (others => '0');
      for i in 7 downto 0 loop -- 32-bit = 8 hex chars
        read(text_line_v, char_v); -- get one hex char
        case char_v is
          when '0'       => data_v(i*4+3 downto i*4) := x"0";
          when '1'       => data_v(i*4+3 downto i*4) := x"1";
          when '2'       => data_v(i*4+3 downto i*4) := x"2";
          when '3'       => data_v(i*4+3 downto i*4) := x"3";
          when '4'       => data_v(i*4+3 downto i*4) := x"4";
          when '5'       => data_v(i*4+3 downto i*4) := x"5";
          when '6'       => data_v(i*4+3 downto i*4) := x"6";
          when '7'       => data_v(i*4+3 downto i*4) := x"7";
          when '8'       => data_v(i*4+3 downto i*4) := x"8";
          when '9'       => data_v(i*4+3 downto i*4) := x"9";
          when 'a' | 'A' => data_v(i*4+3 downto i*4) := x"a";
          when 'b' | 'B' => data_v(i*4+3 downto i*4) := x"b";
          when 'c' | 'C' => data_v(i*4+3 downto i*4) := x"c";
          when 'd' | 'D' => data_v(i*4+3 downto i*4) := x"d";
          when 'e' | 'E' => data_v(i*4+3 downto i*4) := x"e";
          when 'f' | 'F' => data_v(i*4+3 downto i*4) := x"f";
          when others    => data_v(i*4+3 downto i*4) := x"0";
        end case;
      end loop; -- i
      -- store according byte to memory image --
      mem_v(index_v) := data_v;
      index_v := index_v + 1;
    end loop; -- not end of file
    return mem_v;
  end function init_imem_hex;

The you can initialize the IMEM's mem_rom signal:

constant mem_rom : mem32_t(0 to IMEM_SIZE/4-1) := init_imem_hex("/mnt/n/Projects/neorv32/sw/example/hello_world/neorv32_raw_exe.hex", IMEM_SIZE/4);

[wwwweb]

Cool, thanks for your effort. I will test this in the next days.

There seem to be also some other promising approaches ( see the discussion I started here in German https://www.mikrocontroller.net/topic/541850 ).

Especially the separation by different libraries as in the initial post or using configurations as proposed later in the thread above may have the advantage that the sources and workflow don't need to be changed much* (and additionally even a different IMEM architecture may be used per neorv32 instance**).

Thanks for your great work!

---

*Independent from this discussion, it may be worth considering to rename the "library neorv32; use neorv32..." into "library work; use work..." in the core source files. So the user can decide into which library to compile or compile into multiple libraries.

**Imagine using an IMEM architecture with instantiated BRAM primitives for synthesis that is not initialized but can be updated on synthesis/par result on bitstream creation for dedicated instances [to avoid complete HDL resynth/par of large HDL designs on neorv32 software change during development] while additionally using more comfortable default inferred RAM architecture with the initialization we have now for other instances [with final or simpler software that doesnt need updates] or for instances used in testbenches only.

[stnolting]

There seem to be also some other promising approaches ( see the discussion I started here in German https://www.mikrocontroller.net/topic/541850 ).

Interesting thread! Thanks for the hint.

[...] using configurations as proposed later in the thread above [...]

I have heard about configurations but I have never really used them is a design. Furthermore, I am not sure if synthesis tools actually support them. 🤔

Does GHDL support configurations? I need to check that...

Independent from this discussion, it may be worth considering to rename the "library neorv32; use neorv32..." into "library work; use work..." in the core source files. So the user can decide into which library to compile or compile into multiple libraries.

The neorv32 library is a historic result. 😄 In the early days of this project all the module's architecture name were lacking the neorv32_ prefix and had common names like "ram", "alu" or "control". I wanted to avoid "name collisions" and so I put everything into a specific, core-only library. Changing this would require some work, but maybe it's worth it - I'll think about this again.

Imagine using an IMEM architecture with instantiated BRAM primitives for synthesis that is not initialized but can be updated on synthesis/par result on bitstream creation for dedicated instances [to avoid complete HDL resynth/par of large HDL designs on neorv32 software change during development] [...]

We had a discussion about this some day ago in #390.

Just ignore the rtl/mem folder when adding the processor sources to the project and replace the neorv32_imem architecture by a custom one. I think this is a reasonable solution. What do you think?

[wwwweb]

I have heard about configurations but I have never really used them is a design. Furthermore, I am not sure if synthesis tools actually support them. 🤔

Does GHDL support configurations? I need to check that...

Not sure about GHDL. I also never used configurations so far, but tried now with Modelsim and Vivado. Modelsim supports different notations, Xilinx Vivado only supports one type of notations it seems. What works with both is (have to remind from my head as I don't have the source here atm, so may contain syntax errors):

-- configuration of default architecture
configuration neorv32_imem_config_default of neorv32_top is
  for neorv32_top_rtl
    for neorv32_int_mem_inst_true
      for neorv32_int_mem_inst : neorv32_imem
        use entity work.neorv32_imem(neorv32_imem_rtl);
      end for;
    end for;
  end for;
end neorv32_imem_config_default;

-- configuration of user architecture
configuration neorv32_imem_config_some_user_arch of neorv32_top is
  for neorv32_top_rtl
    for neorv32_int_mem_inst_true
      for neorv32_int_mem_inst : neorv32_imem
        use entity work.neorv32_imem(neorv32_imem_xilinx_primitives);
      end for;
    end for;
  end for;
end neorv32_imem_config_some_user_arch;

Then for multiple instantiations:

library neorv32;
  use neorv32.neorv32_package.neorv32_top;
  use neorv32.neorv32_imem_config_default;
  use neorv32.neorv32_imem_config_some_user_arch;

entity design_top is
...
end design_top;

architecture foo of design_top is
...

-- config
for neorv32_top_inst_0 : neorv32_top
  use configuration neorv32.neorv32_imem_config_default;

for neorv32_top_inst_1 : neorv32_top
  use configuration neorv32.neorv32_imem_config_some_user_arch;

begin
  
  -- instances
  neorv32_top_inst_0 : neorv32_top
    generic map (
    ...
    )
    port map (
    ...
    );
    
  neorv32_top_inst_1 : neorv32_top
    generic map (
    ...
    )
    port map (
    ...
    );
    
end foo;   

A new architecture neorv32_imem_xilinx_primitives of neorv32_imem has to be added of course.

So besides the sim/syn architecture use case I described in the last post, this could also be used for different softwares of the 2 instances. The image generation script just has to generate

• a package neorv32_application_image_suffix, defining a constant application_init_image_suffix
• an architecture neorv32_imem_rtl_suffix of neorv32_imem that includes this package and uses the constant application_init_image_suffix for ROM instantiation
• an according configuration neorv32_imem_config_suffix

with a different suffix or similar for each software image. So these three items have to be compiled for each instance and the according configuration has to be assigned to each instance.

The good thing is, that it is backwards compatible, as the configuration can be just left out, if only one core is instantiated (the tools just take the only default architecture that is available).

The bad thing is, that it is one of these library/package hacks you mentioned 😄 . Plus, it may not be supported by all toolchains.

Independent from this discussion, it may be worth considering to rename the "library neorv32; use neorv32..." into "library work; use work..." in the core source files. So the user can decide into which library to compile or compile into multiple libraries.

The neorv32 library is a historic result. 😄 In the early days of this project all the module's architecture name were lacking the neorv32_ prefix and had common names like "ram", "alu" or "control". I wanted to avoid "name collisions" and so I put everything into a specific, core-only library. Changing this would require some work, but maybe it's worth it - I'll think about this again.

Not sure if we mean the same thing here. I think it is completely okay to have one library for the whole core. But, if you replace the

library neorv32;
  use neorv32.sth;

by the VHDL library placeholder "work"

library work;
  use work.sth;

in all of the core source files, the actual library to which it is compiled is defined at compile time. So the same file can be compiled into multiple libraries (in theory, explained later).

So for the multi-instance use case, this means the two instances are just separated by different libraries, compiling the sources two times with different application images (modelsim compile example):

#first instance
vcom -2008 -work neorv32_0 "$NEORV32_CORE_PATH/neorv32_package.vhd"
vcom -2008 -work neorv32_0 "$NEORV32_CORE_PATH/neorv32_cpu.vhd"
vcom -2008 -work neorv32_0 "$NEORV32_CORE_PATH/neorv32_imem.entity.vhd"
... #all the other neorv32 core sources except for application image
vcom -2008 -work neorv32_0 "$NEORV32_CORE_PATH/neorv32_top.vhd"
vcom -2008 -work neorv32_0 "$NEORV32_CORE_PATH/mem/neorv32_imem.default.vhd"
#software 0
vcom -2008 -work neorv32_0 "$USER_PATH_SW_0/neorv32_application_image.vhd"

#second instance
vcom -2008 -work neorv32_1 "$NEORV32_CORE_PATH/neorv32_package.vhd"
vcom -2008 -work neorv32_1 "$NEORV32_CORE_PATH/neorv32_cpu.vhd"
vcom -2008 -work neorv32_1 "$NEORV32_CORE_PATH/neorv32_imem.entity.vhd"
... #all the other neorv32 core sources except for application image
vcom -2008 -work neorv32_1 "$NEORV32_CORE_PATH/neorv32_top.vhd"
vcom -2008 -work neorv32_1 "$NEORV32_CORE_PATH/mem/neorv32_imem.default.vhd"
#software 1
vcom -2008 -work neorv32_1 "$USER_PATH_SW_1/neorv32_application_image.vhd"

Then for multiple instantiations:

library neorv32_0;
library neorv32_1;

entity design_top is
...
end design_top;

architecture foo of design_top is
...

begin
  
  -- instances
  neorv32_top_inst_0 : entity neorv32_0.neorv32_top
    generic map (
    ...
    )
    port map (
    ...
    );
    
  neorv32_top_inst_1 : entity neorv32_1.neorv32_top
    generic map (
    ...
    )
    port map (
    ...
    );
    
end foo;   

The good thing is, this works without using configurations or the need to change source code (except for the library "work" change). It is backwards compatible as you still can compile a single core into library neorv32 and it is still one core-only library.

The bad thing is, the multi-instance use case by libraries works in Modelsim, but Vivado does not support compiling the same file into multiple libraries (it just ignores the second read of the file).

But, besides the multi-instance use case, the opportunity to compile the single core into a library of own choice (instead of fixed neorv32) may be nice and comes at basically no cost. This was what I meant with my comment.

We had a discussion about this some day ago in #390.

Just ignore the rtl/mem folder when adding the processor sources to the project and replace the neorv32_imem architecture by a custom one. I think this is a reasonable solution. What do you think?

Actually, I just have done this now for a specific Xilinx architecture, using the configuration approach from above (so the rtl/core/mem is not ignored but another architecture is added and selected).

Conclusion:

• multi-instance with configurations is working with Modelsim/Xilinx Vivado at least
• multi-instance with library separation is working with Modelsim but not Xilinx Vivado
• multi-instance with your init_imem_hex works (didn't test myself yet)

[umarcor]

IMHO using configurations for changing the initial content of the ROMs is overkill. That should be done with just a generic containing the path (relative or absolute) to a text/bin/hex file (ref #35). It can optionally accept some other generics to be hardcoded as registries.

Configurations are useful in order to have multiple instances, each with a different architecture, and the architecture defined at "runtime" (actually, during elaboraiton). I don't think we have that use case here, yet. If we want to have multiple instances, we are probably going to use the same architecture (because it's the same device, so we need the same memory description code), and we can probably handle all the params through generics. Configuration might be used to rework the build system itself (i.e. picking different mem VHDL sources), but the issue is that some tools might fail when analysing descriptions for other tools, despite those sources not being used. Therefore, I think we want to keep analysing the relevant sources only.

With regard to using work in the sources of neorv32, instead of using an specific lib name, I believe that could help. Then, it would be possible to copy the sources to two directories and build each of them into a different library, to then instantiate the two top modules in the final design. That would allow to work around the limitations of some tools to analyse the same source into two logical libs at the same time. Nevertheless, I believe this is not necessary in order to handle the memory initialisation; it would be useful if the user wanted the multiple instantiations to have different internal architectures (have different peripherals/features which cannot be selected/customised through the currently available top-level generics).