Can you store an instance's information in an HRef object?

[jacobdbrown4]

HRef objects are meant to reference netlist objects (such as instances, ports, wires) by giving the hierarchical sequence to the object. This enables one to ensure they are accessing the correct object when modifying a netlist.

If one wanted to store information about an hinstance, a suggested and more correct way would be to actually store the information in the instance object itself, not the hinstance. Doing this is pretty simple. The following is an example of getting hinstances, using '.item' to access the instance object, assigning them lengths and widths (as an example of storing extra information), and then re-accessing the information later.

    import spydrnet as sdn
    from random import randint
    
    
    netlist = sdn.load_example_netlist_by_name("hierarchical_luts")
    
    #assign random length and width values to all of the instances
    for hinstance in netlist.get_hinstances(recursive=True):
        length = randint(0,10)
        width = randint(0,10)
        instance = hinstance.item
        instance['length'] = length
        instance['width'] = width
    
    for hinstance in netlist.get_hinstances(recursive=True):
        print(hinstance.name,'  Length: ',hinstance.item['length'],'  Width: ',hinstance.item['width'])

Hopefully this helps!

[ganeshgore]

Hello Jacob, thank you for your reply.

The method you proposed above still saves parameters for each instance (and not for each hsintance).

Parameters like Height and Width happens to be the same for instance or any hierarchical instance of that instance, but think about the parameter like X and Y position. Every hinstance of that instance will have a different number with respect to the top module.

[ganeshgore]

I was thinking more about it, mainly to categorize different parameters expected on each hObject, but I couldn't think of anything other than X_Pos and Y_Pos.
Also, I noticed that although X_Pos and Y_Pos parameters are different for each hObject, they can be derived by traversing the hierarchy (just like a path) instead of storing it in the hRef object; the attached figure illustrates the idea x1, x2, x3, y1, y2, y3 are all instance parameters. In some cases, the parameter value may be inherited from the parent object (like power_supply_domain).
But providing a way to traverse parameters like this from href should address this problem.

[ganeshgore]

I guess I managed to add this functionality in the exiting structure itself

My idea here is while creating the HRef object, I am extending the HRef class with a few methods from its item, which allows you to traves back in the hierarchy to get such properties

This is how I extract methods to copy in HRef object
https://github.com/ganeshgore/spydrnet/blob/35be3e1a01c712c7fbec147649670f4dcafc4ca6/spydrnet/util/hierarchical_reference.py#L9-L13

This is how it is extended
https://github.com/ganeshgore/spydrnet/blob/35be3e1a01c712c7fbec147649670f4dcafc4ca6/spydrnet/util/hierarchical_reference.py#L230

and this is the body of the copied method which can perform different functions based on who called it (its own object or HRef)
https://github.com/ganeshgore/spydrnet/blob/35be3e1a01c712c7fbec147649670f4dcafc4ca6/spydrnet/ir/instance.py#L52

Let me know your comments, I will clean up and create PR for this as well

[andrewmkeller]

There is more going on with this issue than we have considered. As @jacobdbrown4 mentions above, the current convention for storing properties is to store them with the instance or definition of the netlist. As @ganeshgore mentions, this is problematic for giving different hierarchical instances of the same instance different property values.

In netlist viewer like Vivado, only the hierarchical view of the netlist is accessible to the user, all of the underlying data structures are hidden from the user. I am not sure how Vivado handles the case of where different instances of the same hierarchical instances are placed.

In our use of SpyDrNet in the past, we have made a copy of all non-leaf definitions so that each definition is only instanced once. This uniquification has allowed us to apply techniques like TMR and DWC, changing property values, on specific hierarchical instances without affecting other instances.

What @ganeshgore is proposing may also be accomplished by adding a dictionary to the slots collection of the href class, but there are some complications that must be considered, like what happens when the path changes, how would these properties get stored out to disk (EDIF or verilog would require uniquification of hierarchical instances), etc. HRef was originally designed to be a lite unique reference to the actual netlist data.

I am not sure what adding heavy data objects to the href will do to the design approach in SpyDrNet. All hrefs are unique, and all exist virtually whether they are instanced in memory or not. Outdated hrefs can exist that are no longer valid. Garbage collection will remove any href that is not reference explicitly somewhere in your code. What happens now when properties are stored with hrefs? We can't let them be garbage collected because we will lose the associated properties.

We could consider maintaining a complete collection of hrefs that follow the netlist, or just work with hierarchical instances. But the issues mentioned above are still there. A principle that SpyDrNet tries to follow is "cause the least amount of change as needed." If we went all hierarchical instances, then we may lose some of the structure on the netlist brought in (unique copies of non-leaf definitions vs multiple instances of the same non-leaf definition).

There is definitely a need for what is being described, and I like the ideas that are being discussed. What are some additional ideas of how we can approach this?

[andrewmkeller]

Is the uniquify function available in the spydrnet? or is is only in spydrnet-shrec right now?

[ganeshgore]

Hello @andrewmkeller Thanks for your review

I thought about the type of properties we HRef objects can have, and I came up with the following categories.

1] Instance Specific: This property is like name, width, and height; irrespective of what is HRef of this instance, it will be the same
2] Travesible: This is like a full-path or X and Y location from the top module, which can be extracted by traversing back on demand. (Which is what I tried to implement in this PR)
3] Independent/Unique: This property is uniquely based on the full path. I am not entirely sure how we can implement this without creating a flattened graph of the entire design.

.... how Vivado handles the case of where different instances of the same hierarchical instances are placed. ...

• I have not worked with VIVADO TCL interface much, but in physical design from Synopsys and Cadence, they work on the hierarchical object than definitions and instances. Maybe internally, they still maintain a just hierarchy, like SpyDrNet project is doing now.

.... we have made a copy of all non-leaf definitions ....

• That will change the hierarchy, or we need to explicitly keep track of duplicated definitions (which, on second thought, is trying to flatten the netlist)

"... accomplished by adding a dictionary to the slots collection of the Href class ... "

• I thought about it, but the Href object is common for any IR representation (Definition, Instance, Wire ...), in which case additional properties on HRef are dependent on the item it is representing.

... like what happens when the path changes...
Property types 1] and 2] are independent of path changes; for type 3] properties, one way I thought of is using existing callback functionality and creating a plugin to maintain the hashtable of all unique properties. Making this hashtable store only unique values that will help limit the memory requirement
It will be very similar to the default_namespace plugin, which checks for duplicate naming.

"... We can't let them be garbage collected because we will lose the associated properties. ..."
I am guessing it will not affect type 1] and 2] properties as removing outdated HRef, does not lose data. However, in Type 3], the hashtable in the plugin will be permanent, and it will scale based on how much unique data is stored.

Let me know if I am missing out on something

[andrewmkeller]

Is the uniquify function available in the spydrnet? or is is only in spydrnet-shrec right now?

The uniquify function is available in spydrnet. See https://github.com/byuccl/spydrnet/blob/master/spydrnet/uniquify.py.

This operation on the netlist makes a copy of all non-leaf definitions until each non-leaf definition is only ever instanced once in the netlist at most. This operation is different than flattening the netlist. It preserves the hierarchy of the netlist while also making it so that each instance of a definition is unique. There may be multiple copies of the same non-leaf definition, but each copy is only ever instanced once. This function is documented https://byuccl.github.io/spydrnet/docs/stable/reference/uniquify.html, but the documentation could be improved.

• Improve doc string documentation of the uniquify function and pull doc string into sphinx documentation.

The function is easy function to use:

from spydrnet.uniquify import uniquify
...
# Make sure non-leaf definition instances refer to a unique definition
uniquify(netlist)

It looks like we don't have any examples that use this function in the spydrnet repository. We should add some examples.

• Add spydrnet uniquify examples to the docstrings and pull the doc strings into sphinx documentation.

Below is an example of uniquify.

Before

- Leaf Definition:
  - Name: LUT
- Non-Leaf Definition:
  - Name: SubCell
  - Instance:
    - Name: MyLUT
    - Reference: LUT
    - Property: INIT=0xE
- Top Definition:
  - Name: Top
  - Instance:
    - Name: MySubCell1
    - Reference: SubCell
  - Instance:
    - Name: MySubCell2
    - Reference: SubCell

After

- Leaf Definition:
  - Name: LUT
- Non-Leaf Definition:
  - Name: SubCell1
  - Instance:
    - Name: MyLUT
    - Reference: LUT
    - Property: INIT=0xE
- Non-Leaf Definition:
  - Name: SubCell2
  - Instance:
    - Name: MyLUT
    - Reference: LUT
    - Property: INIT=0x5
- Top Definition:
  - Name: Top
  - Instance:
    - Name: MySubCell1
    - Reference: SubCell1
  - Instance:
    - Name: MySubCell2
    - Reference: SubCell2

Hierarchy is preserved, but now the properties of the LUT instances can be changed without affecting multiple areas of the netlist.

If you run uniqueify on your netlist first, then you can change the properties of href.item without changing the properties of different instances of the same item (since the item referenced is now only instanced once).

[andrewmkeller]

In response to #155 (comment) from @ganeshgore.

While we don't know exactly how top EDAs associate properties with specific hierarchical instances, it is clear that they sometimes use a type of href in constraint files to store property values out to disk. For example, when storing locations of cell instances, they might have a file that contains them like this:

set_property LOC X55Y11 "top/MySubCell1/MyLUT"

In this way, the property can be stored without necessarily storing it with the netlist. This approach can also help when you want to store a single property to the lots of items in the netlist.

I would l like to look at what happens to the netlist with the example in the scenario above #155 (comment).

[jacobdbrown4]

@ganeshgore @andrewmkeller is this issue still relevant or can I close it?