VHDL parser from andres manelli's fork for entity parsing support

[michael-etzkorn]

This mirrors the pull request here kevinpt#14 using the master branch of https://github.com/andresmanelli/hdlparse . I've briefly tested this with capstone.vhd from https://github.com/stnolting/captouch . It only seems to pick up the out_filter_t type, but at least it does some sort of parsing of the entity.

>> import hdlparse.vhdl_parser as vhdl
>> vhdl_ex = vhdl.VhdlExtractor()
>> vhdl_objs = vhdl_ex.extract_objects("captouch.vhd")
>> vhdl_objs
[VhdlEntity('captouch'), VhdlType('out_filter_t', 'array_type')]
>> import pprint
>> pprint.pprint(vhdl_objs[0].__dict__) 
{'desc': [],
 'generics': [VhdlParameter('F_CLOCK', 'in', 'integer'),
              VhdlParameter('NUM_PADS', 'in', 'integer'),
              VhdlParameter('SENSITIVITY', 'in', 'integer')],
 'kind': 'entity',
 'name': 'captouch',
 'ports': [VhdlParameter('clk_i', 'in', 'std_ulogic'),
           VhdlParameter('rstn_i', 'in', 'std_ulogic'),
           VhdlParameter('rstn_sync_i', 'in', 'std_ulogic'),
           VhdlParameter('ready_o', 'out', 'std_ulogic'),
           VhdlParameter('touch_o', 'out', 'std_ulogic_vector(NUM_PADS-1 downto 0)'),
           VhdlParameter('pad_io', 'inout', 'std_logic_vector(NUM_PADS-1 downto 0)')],
 'sections': {}}

Previous behavior: vhdl_objs would return as an empty list.

I didn't go through and reapply @andresmanelli's changes. Instead, I just patched the branch with his vhdl_parser.py so @vvvverre's format changes would have to be redone.

[andresmanelli]

If you apply this change:

diff --git a/hdlparse/vhdl_parser.py b/hdlparse/vhdl_parser.py
index af9789d..05e424c 100644
--- a/hdlparse/vhdl_parser.py
+++ b/hdlparse/vhdl_parser.py
@@ -94,6 +94,7 @@ vhdl_tokens = {
   'architecture': [
     (r'end\s+\w+\s*;', 'end_arch', '#pop'),
     (r'/\*', 'block_comment', 'block_comment'),
+    (r'type\s+(\w+)\s*is', 'type', 'type_decl'),
     (r'--.*\n', None),
   ],
   'generic_list': [

you get:

In [1]: import hdlparse.vhdl_parser as vhdl
In [2]: vhdl_ex = vhdl.VhdlExtractor()
In [3]: vhdl_objs = vhdl_ex.extract_objects("captouch.vhd")
In [4]: vhdl_objs
Out[4]:
[VhdlEntity('captouch'),
 VhdlType('ctrl_state_t', 'enum_type'),
 VhdlType('ctrl_t', 'record_type'),
 VhdlType('out_filter_t', 'array_type')]

It's strange that you got one type at all, because the type parser was declared at the root and package level. I added the same line at the architecture level .

[michael-etzkorn]

Hi Andres, I've pushed your fix for type parsing to this branch. Hopefully this weekend I can reintroduce vvvverre's style changes so the style is consistent between the two parsers.

I'm also in favor of parsing the logic type for systemverilog so I'll introduce a PR for that soon (even if this python parser isn't intended to be a full blown SV parser. It'll be nice to have some of the basics.)

[andresmanelli]

I'm not sure if that should be included in another file ? (another parser for SV). But other than that this PR LGTM.

I'll approve, waiting for another reviewer before merge.

[Paebbels]

@umarcor this PR looks ready to be merged. Any objections?

[Paebbels]

Please check my review comments.

[michael-etzkorn]

Unfortunately, reformatting the code with PyCharm seems to make the git diff (and merging) difficult, so I've reintroduced these changes after @andresmanelli's modifications. Diffing this and main, the only differences now seem to be @andresmanelli vhdl_parser changes.

9d8
<
37c36
<         (r'end\s+package', None, '#pop'),
---
>         (r'end\s+\w+\s*;', None, '#pop'),
43c42
<         (r'end\s+package\s+body', None, '#pop'),
---
>         (r'end\s+\w+\s*;', None, '#pop'),
89c88,90
<         (r'end\s+entity\s*;', 'end_entity', '#pop'),
---
>         (r'generic\s*\(', None, 'generic_list'),
>         (r'port\s*\(', None, 'port_list'),
>         (r'end\s+\w+\s*;', 'end_entity', '#pop'),
94c95
<         (r'end\s+architecture\s*;', 'end_arch', '#pop'),
---
>         (r'end\s+\w+\s*;', 'end_arch', '#pop'),
95a97
>         (r'type\s+(\w+)\s*is', 'type', 'type_decl'),
107c109
<         (r'\s*(\w+)\s*', 'generic_param_type'),
---
>         (r'\s*(\w+)[ \t\r\f\v]*', 'generic_param_type'),
109a112,114
>         (r'\)\s*;\s*--(.*)\n', 'line_comment', '#pop:2'),
>         (r'\n\s*\)\s*;\s*--(.*)\n', 'generic_list_comment', '#pop:2'),
>         (r'\n\s*', None),
122c127
<         (r'--.*\n', None),
---
>         (r'--(.*)\n', 'line_comment'),
126c131
<         (r'\s*(in|out|inout|buffer)\s+(\w+)\s*', 'port_param_type'),
---
>         (r'\s*(in|out|inout|buffer)\s+(\w+)[ \t\r\f\v]*', 'port_param_type'),
128a134,137
>         (r'--(.*)\n', 'line_comment'),
>         (r'\)\s*;\s*--(.*)\n', 'line_comment', '#pop:2'),
>         (r'\n\s*\)\s*;\s*--(.*)\n', 'port_list_comment', '#pop:2'),
>         (r'\n\s*', None),
132d140
<         (r'--.*\n', None),
135a144
>         (r'\s*([\w\+\-\*/\s]+)(\s+(?:down)?to)\s+([\w\+\-\*/\s]+)', 'array_range_val'),
139a149
>         (r'\s*([\w\+\-\*/\s]+)(\s+(?:down)?to)\s+([\w\+\-\*/\s]+)', 'array_range_val'),
172a183
>       param_desc (str): Description of the parameter
175c186
<     def __init__(self, name, mode=None, data_type=None, default_value=None, desc=None):
---
>     def __init__(self, name, mode=None, data_type=None, default_value=None, desc=None, param_desc=None):
180a192
>         self.param_desc = None
184c196
<             param = '{} : {} {}'.format(self.name, self.mode, self.data_type)
---
>             param = '{} : {} {}'.format(self.name, self.mode, self.data_type.name + self.data_type.arange)
186c198
<             param = '{} : {}'.format(self.name, self.data_type)
---
>             param = '{} : {}'.format(self.name, self.data_type.name + self.data_type.arange)
188a201,202
>         if self.param_desc is not None:
>             param = '{} --{}'.format(param, self.param_desc)
192c206,229
<         return "VhdlParameter('{}', '{}', '{}')".format(self.name, self.mode, self.data_type)
---
>         return "VhdlParameter('{}', '{}', '{}')".format(self.name, self.mode,
>                                                         self.data_type.name + self.data_type.arange)
>
>
> class VhdlParameterType(object):
>     """Parameter type definition
>
>     Args:
>       name (str): Name of the type
>       direction(str): "to" or "downto"
>       r_bound (str): A simple expression based on digits or variable names
>       l_bound (str): A simple expression based on digits or variable names
>       arange (str): Original array range string
>     """
>
>     def __init__(self, name, direction="", r_bound="", l_bound="", arange=""):
>         self.name = name
>         self.direction = direction.strip()
>         self.r_bound = r_bound.strip()
>         self.l_bound = l_bound.strip()
>         self.arange = arange
>
>     def __repr__(self):
>         return "VhdlParameterType('{}','{}')".format(self.name, self.arange)
309a347,372
> class VhdlEntity(VhdlObject):
>     """Entity declaration
>     Args:
>       name (str): Name of the entity
>       ports (list of VhdlParameter): Port parameters to the entity
>       generics (list of VhdlParameter): Generic parameters to the entity
>       sections (list of str): Metacomment sections
>       desc (str, optional): Description from object metacomments
>     """
>
>     def __init__(self, name, ports, generics=None, sections=None, desc=None):
>         VhdlObject.__init__(self, name, desc)
>         self.kind = 'entity'
>         self.generics = generics if generics is not None else []
>         self.ports = ports
>         self.sections = sections if sections is not None else {}
>
>     def __repr__(self):
>         return "VhdlEntity('{}')".format(self.name)
>
>     def dump(self):
>         print('VHDL entity: {}'.format(self.name))
>         for p in self.ports:
>             print('\t{} ({}), {} ({})'.format(p.name, type(p.name), p.data_type, type(p.data_type)))
>
>
376c439
<     last_item = None
---
>     last_items = []
384c447
<             if last_item is None:
---
>             if not last_items:
387c450,451
<                 last_item.desc = realigned
---
>                 for i in last_items:
>                     i.desc = realigned
443a508,516
>         elif action == 'entity':
>             kind = 'entity'
>             name = groups[0]
>             generics = []
>             ports = []
>             param_items = []
>             sections = []
>             port_param_index = 0
>
458c531
<
---
>             last_items = []
460c533,536
<                 generics.append(VhdlParameter(i, 'in', ptype))
---
>                 p = VhdlParameter(i, 'in', VhdlParameterType(ptype))
>                 generics.append(p)
>                 last_items.append(p)
>
462c538,541
<             last_item = generics[-1]
---
>
>         elif action == 'generic_param_default':
>             for i in last_items:
>                 i.default_value = groups[0]
470a550
>             last_items = []
472c552,554
<                 ports.append(VhdlParameter(i, mode, ptype))
---
>                 p = VhdlParameter(i, mode, VhdlParameterType(ptype))
>                 ports.append(p)
>                 last_items.append(p)
475c557,560
<             last_item = ports[-1]
---
>
>         elif action == 'port_param_default':
>             for i in last_items:
>                 i.default_value = groups[0]
480a566,568
>         elif action == 'array_range_val':
>             l_bound, direction, r_bound = groups
>
483a572
>             last_items = []
485c574,576
<                 ports.append(VhdlParameter(i, mode, ptype + arange))
---
>                 p = VhdlParameter(i, mode, VhdlParameterType(ptype, direction, r_bound, l_bound, arange))
>                 ports.append(p)
>                 last_items.append(p)
488c579,584
<             last_item = ports[-1]
---
>
>         elif action == 'end_entity':
>             vobj = VhdlEntity(name, ports, generics, dict(sections), metacomments)
>             objects.append(vobj)
>             last_items = []
>             metacomments = []
493c589
<             last_item = None
---
>             last_items = []
506c602
<                 'array_type', 'file_type', 'access_type', 'record_type', 'range_type', 'enum_type', 'incomplete_type'):
---
>         'array_type', 'file_type', 'access_type', 'record_type', 'range_type', 'enum_type', 'incomplete_type'):
526a623,627
>         elif action == 'line_comment':
>             for i in last_items:
>                 if not i.param_desc:
>                     i.param_desc = groups[0]
>
594c695,696
<         self.array_types = {'std_ulogic_vector', 'std_logic_vector', 'signed', 'unsigned', 'bit_vector'}
---
>         self.array_types = set(('std_ulogic_vector', 'std_logic_vector',
>                                 'signed', 'unsigned', 'bit_vector'))
723c825
<     code = """
---
>     code = '''
731,733c833,837
<       a,b,c : in std_ulogic;
<       f,g,h : inout bit
<     );
---
>       a,b,c : in std_ulogic;    -- no default value
>       f,g,h : inout bit := '1'; -- bit ports
>       v : in std_logic_vector(lBound -1 downto 0) -- array range
>     ); -- port list comment
>
737c841
<   """
---
>   '''

[Paebbels]

@michael-etzkorn when you're ready for a new review, please use the button on the right navigation bar to retrigger a review.

[Paebbels]

Look now OK. All review items are either resolved or forwarded to issues like #5.

---

/cc @KatCe, @michael-etzkorn, @andresmanelli, @umarcor