Special commands

[brevius]

Special commands results with error:

[ERROR] 20200716 10:00:20 | hyperglass.configuration:153 | _validate_config → 1 validation error for Router
commands
  value is not a valid dict (type=type_error.dict)
[ERROR] 20200716 10:00:20 | hyperglass.exceptions:28 | __init__ → [WARNING] 

Field: commands
  Error: value is not a valid dict

Traceback (most recent call last):
  File "/usr/local/bin/hyperglass", line 11, in <module>
    sys.exit(CLI())
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/hyperglass/cli/commands.py", line 69, in build_frontend
    return build_ui()
  File "/usr/local/lib/python3.6/dist-packages/hyperglass/cli/util.py", line 170, in build_ui
    from hyperglass.configuration import params, frontend_params, CONFIG_PATH
  File "/usr/local/lib/python3.6/dist-packages/hyperglass/configuration/__init__.py", line 183, in <module>
    config=_user_devices.get("routers", []), importer=_routers.Routers._import,
  File "/usr/local/lib/python3.6/dist-packages/hyperglass/configuration/__init__.py", line 154, in _validate_config
    raise ConfigInvalid(err.errors()) from None
hyperglass.exceptions.ConfigInvalid: 

Field: commands
  Error: value is not a valid dict

I added juniper_raw entry in commands.yaml to already exisiting cutomization commands like:

juniper:
  ipv4_default:
[...]
juniper_raw:
  ipv4_default:
[...]
    bgp_community: 'show route protocol bgp table inet.0 community large:{target} detail | match "entry|comm|via|AS.path|Age|Preference|Inactive"'
  ipv6_default:
[...]

After that the error is generated durig UI building. In the config I provided definitions for all commands (as documentation suggests).

[thatmattlove]

While the syntax in your example above looks correct, the error seems to indicate a possible YAML syntax error, or something along those lines. Can you share the entire commands.yaml? If debugging is enabled, there should also be a line of debug output right before this error that says "Unvalidated configuration from /etc/hyperglass/commands.yaml:" followed by the imported YAML as a Python dictionary. Can you share either or both?

[linuxgemini]

I can also confirm this (the flags -b -d are because of #56):

root@hell:~# hyperglass start -b -d
[INFO] 20200718 22:28:39 | hyperglass.configuration:43 | <module> → Configuration directory: /etc/hyperglass
[DEBUG] 20200718 22:28:39 | hyperglass.log:46 | set_log_level → Debugging enabled
[DEBUG] 20200718 22:28:39 | hyperglass.configuration:165 | <module> → Unvalidated configuration from /etc/hyperglass/hyperglass.yaml: {'debug': True, 'developer_mode': False, 'primary_asn': '208563', 'org_name': 'linuxgemini', 'site_title': 'AS208563 hyperglass', 'site_description': 'a hyperglass instance for AS208563', 'request_timeout': 30, 'listen_address': '172.16.1.10', 'listen_port': 59788, 'cors_origins': ['https://hyperg.linuxgemini.space'], 'cache': {'database': 7, 'host': '172.16.1.4', 'port': 6379, 'show_text': True, 'timeout': 120}, 'queries': {'bgp_route': {'enable': True, 'display_name': 'show route table master6 all for'}, 'bgp_community': {'enable': False}, 'bgp_aspath': {'enable': True, 'display_name': 'show ip route all where bgp_path.last', 'pattern': {'asdot': '^(\\d+\\.\\d+)$', 'asplain': '^\\d+$', 'mode': 'asplain'}}, 'ping': {'enable': True, 'display_name': 'ping6'}, 'traceroute': {'enable': True, 'display_name': 'traceroute6'}}, 'structured': {'communities': {'mode': 'deny', 'items': []}, 'rpki': {'mode': 'external'}}}
[DEBUG] 20200718 22:28:39 | hyperglass.configuration:174 | <module> → Unvalidated commands from /etc/hyperglass/commands.yaml: {'special_commands': {'ipv4_default': {'bgp_aspath': 'show route all where bgp_path.last = {target}', 'bgp_community': 'show route all where {target} ~ bgp_community', 'bgp_route': 'show route all where {target} ~ net', 'ping': 'ping -4 -c 5 -I {source} {target}', 'traceroute': 'traceroute -4 -w 1 -q 1 -s {source} {target}'}, 'ipv6_default': {'bgp_aspath': 'show route all where bgp_path.last = {target}', 'bgp_community': 'show route all where {target} ~ bgp_community', 'bgp_route': 'show route all where {target} ~ net', 'ping': 'ping -6 -c 5 -I {source} {target}', 'traceroute': 'traceroute -6 -w 1 -q 1 -s {source} {target}'}, 'ipv4_vpn': {'bgp_aspath': 'show route all where bgp_path.last = {target}', 'bgp_community': 'show route all where {target} ~ bgp_community', 'bgp_route': 'show route all where {target} ~ net', 'ping': 'ping -4 -c 5 -I {source} {target}', 'traceroute': 'traceroute -4 -w 1 -q 1 -s {source} {target}'}, 'ipv6_vpn': {'bgp_aspath': 'show route all where bgp_path.last = {target}', 'bgp_community': 'show route all where {target} ~ bgp_community', 'bgp_route': 'show route all where {target} ~ net', 'ping': 'ping -6 -c 5 -I {source} {target}', 'traceroute': 'traceroute -6 -w 1 -q 1 -s {source} {target}'}}}
[DEBUG] 20200718 22:28:39 | hyperglass.configuration:181 | <module> → Unvalidated devices from /etc/hyperglass/devices.yaml: {'credentials': [{'credential': {'username': 'upstream-main', 'password': 'SNIPPED'}}], 'networks': [{'network': {'name': 'primary', 'display_name': 'Production'}}], 'vrfs': [{'name': 'default', 'display_name': 'Global', 'ipv4': {'access_list': [{'network': '10.0.0.0/8', 'action': 'deny'}, {'network': '192.168.0.0/16', 'action': 'deny'}, {'network': '172.16.0.0/12', 'action': 'deny'}, {'network': '0.0.0.0/0', 'action': 'permit', 'ge': 8, 'le': 24}]}, 'ipv6': {'access_list': [{'network': '::/0', 'action': 'permit', 'ge': 12, 'le': 48}]}}], 'routers': [{'name': 'upstream-main', 'commands': 'special_commands', 'address': '172.16.1.11', 'network': {'name': 'primary', 'display_name': 'Production'}, 'credential': {'username': 'upstream-main', 'password': 'SNIPPED'}, 'display_name': 'upstream-main', 'port': 18443, 'nos': 'bird', 'vrfs': [{'name': 'default', 'display_name': 'Global', 'ipv4': None, 'ipv6': {'source_address': '2a0d:1a40:7800::', 'access_list': [{'network': '::/0', 'action': 'permit', 'ge': 12, 'le': 48}]}}]}]}
[ERROR] 20200718 22:28:39 | hyperglass.configuration:153 | _validate_config → 1 validation error for Router
commands
  value is not a valid dict (type=type_error.dict)
[ERROR] 20200718 22:28:39 | hyperglass.exceptions:28 | __init__ → [WARNING]

Field: commands
  Error: value is not a valid dict

Traceback (most recent call last):
  File "/usr/local/bin/hyperglass", line 11, in <module>
    sys.exit(CLI())
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/hyperglass/cli/commands.py", line 97, in start
    from hyperglass.main import start
  File "/usr/local/lib/python3.6/dist-packages/hyperglass/main.py", line 22, in <module>
    from hyperglass.configuration import (  # isort:skip
  File "/usr/local/lib/python3.6/dist-packages/hyperglass/configuration/__init__.py", line 183, in <module>
    config=_user_devices.get("routers", []), importer=_routers.Routers._import,
  File "/usr/local/lib/python3.6/dist-packages/hyperglass/configuration/__init__.py", line 154, in _validate_config
    raise ConfigInvalid(err.errors()) from None
hyperglass.exceptions.ConfigInvalid:

Field: commands
  Error: value is not a valid dict

root@hell:~#

As like my previous posted issue, config is available on https://files.linuxgemini.space/hyperglass/ (you may need to refresh the file preview to refresh the cache)