Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add napalm/netmiko extras support to plugin config #45

Merged
merged 58 commits into from
Jan 31, 2022
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
2c5c8ed
add extra for connections
jeffkala Dec 29, 2021
38faad8
add debugs
jeffkala Dec 30, 2021
a52e0bb
cleanups
jeffkala Dec 30, 2021
cccefb7
add defaults to plugin config for example
jeffkala Dec 30, 2021
9acbcfe
cleanups add to readme, fix spelling
jeffkala Dec 30, 2021
a92d2cd
fix incorrect default values
jeffkala Jan 6, 2022
d4a2016
rework to support generic connection_options
jeffkala Jan 7, 2022
72c723e
rerun black
jeffkala Jan 7, 2022
043f1a9
update readme with new dictionary paths
jeffkala Jan 7, 2022
da6d7ea
Apply suggestions from code review
jeffkala Jan 10, 2022
c4b3f0c
run black
jeffkala Jan 10, 2022
82d828e
fix python rendering in README to format with black
jeffkala Jan 10, 2022
688464c
refactor to pull conn options from config context
jeffkala Jan 21, 2022
525238e
fix dict key call
jeffkala Jan 21, 2022
8c9985b
update settings
jeffkala Jan 24, 2022
ad4682a
update to merge conn options from global and config context
jeffkala Jan 24, 2022
2f82423
support global and config context
jeffkala Jan 24, 2022
fef0ca1
fix set key path add debug
jeffkala Jan 24, 2022
26f1b40
add debugs
jeffkala Jan 24, 2022
1a351ba
update merge of dicts
jeffkala Jan 24, 2022
09dd870
fix dict loop
jeffkala Jan 24, 2022
c705c5b
fix dict loop
jeffkala Jan 24, 2022
d81581f
tsing fix to new function to merge conn options
jeffkala Jan 24, 2022
73ab965
tsing fix to new function to merge conn options
jeffkala Jan 24, 2022
ee9580f
add extra for connections
jeffkala Dec 29, 2021
3b92fa5
add debugs
jeffkala Dec 30, 2021
2b24c2d
cleanups
jeffkala Dec 30, 2021
9e33b32
cleanups add to readme, fix spelling
jeffkala Dec 30, 2021
52de2ee
rework to support generic connection_options
jeffkala Jan 7, 2022
f096ddc
rerun black
jeffkala Jan 7, 2022
762bed0
update readme with new dictionary paths
jeffkala Jan 7, 2022
28bb19d
Apply suggestions from code review
jeffkala Jan 10, 2022
65c8dbb
run black
jeffkala Jan 10, 2022
39d0276
fix python rendering in README to format with black
jeffkala Jan 10, 2022
88be5e5
refactor to pull conn options from config context
jeffkala Jan 21, 2022
17c32d2
fix dict key call
jeffkala Jan 21, 2022
405c315
update settings
jeffkala Jan 24, 2022
a74eba9
update to merge conn options from global and config context
jeffkala Jan 24, 2022
bf5992b
support global and config context
jeffkala Jan 24, 2022
8921067
fix set key path add debug
jeffkala Jan 24, 2022
070f74e
add debugs
jeffkala Jan 24, 2022
29ddb36
update merge of dicts
jeffkala Jan 24, 2022
bd9e046
fix dict loop
jeffkala Jan 24, 2022
e07d056
fix dict loop
jeffkala Jan 24, 2022
aa8c8ee
tsing fix to new function to merge conn options
jeffkala Jan 24, 2022
a192902
tsing fix to new function to merge conn options
jeffkala Jan 24, 2022
e54513a
rebase and update readme
jeffkala Jan 24, 2022
ad6d01d
rebase and update readme
jeffkala Jan 24, 2022
094b02b
remove debugs
jeffkala Jan 24, 2022
bcb22cb
revamp conn options merge
jeffkala Jan 26, 2022
515e963
cleanup and run black
jeffkala Jan 26, 2022
630e0fb
add a debug
jeffkala Jan 27, 2022
40e1925
add a debug
jeffkala Jan 27, 2022
8668ded
add missing return statement
jeffkala Jan 27, 2022
4b8be06
fix the dictionary building process
jeffkala Jan 27, 2022
6cc8290
update readme cleanup code and run black
jeffkala Jan 27, 2022
5ee00ef
add key table for config contexts
jeffkala Jan 31, 2022
f9d55fc
add key table for config contexts with secrets access type
jeffkala Jan 31, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 27 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,18 @@ PLUGINS = ["nautobot_plugin_nornir"]

PLUGINS_CONFIG = {
"nautobot_plugin_nornir": {
"connection_options": {
jeffkala marked this conversation as resolved.
Show resolved Hide resolved
"napalm": {
"extras": {
"optional_args": {"global_delay_factor": 1},
},
},
"netmiko": {
"extras": {
"global_delay_factor": 1,
},
},
},
"nornir_settings": {
"credentials": "nautobot_plugin_nornir.plugins.credentials.env_vars.CredentialsEnvVars",
"runner": {
Expand All @@ -59,6 +71,18 @@ Alternatively you can use the `CredentialsSettingsVars` class to set the usernam
PLUGINS_CONFIG = {
"nautobot_plugin_nornir": {
"nornir_settings": {
"connection_options": {
"napalm": {
"extras": {
"optional_args": {"global_delay_factor": 1},
},
},
"netmiko": {
"extras": {
"global_delay_factor": 1,
},
},
},
"credentials": "nautobot_plugin_nornir.plugins.credentials.settings_vars.CredentialsSettingsVars",
"runner": {
"plugin": "threaded",
Expand All @@ -83,6 +107,7 @@ The plugin behavior can be controlled with the following list of settings.
| username | ntc | N/A | The username when leveraging the `CredentialsSettingsVars` credential provider. |
| password | password123 | N/A | The password when leveraging the `CredentialsSettingsVars` credential provider. |
| secret | password123 | N/A | The secret password when leveraging the `CredentialsSettingsVars` credential provider, **placeholder only, not currently functioning**. |
| connection_options | {"netmiko": {"extras": "global_delay_factor": 1}}} | N/A | Connection Options for the given plugin provider in use. |

Finally, as root, restart Nautobot and the Nautobot worker.

Expand All @@ -108,7 +133,7 @@ The `dispatcher_mapping` configuration option can be set to extend or map the pl
The above example demonstrates the following use cases.

* Creating a custom only local dispatcher
* Mapping a user defined and preffered platform slug name to expected driver (e.g. ios -> cisco_ios)
* Mapping a user defined and preferred platform slug name to expected driver (e.g. ios -> cisco_ios)
* Overloading platform slug keys, by mapping ios and ios_xe to the same class
* Leveraging the existing "default" Netmiko driver

Expand All @@ -117,7 +142,7 @@ allow users to define their own mappings as described above.

# Inventory

The Nautobot ORM inventory is rather static in nature at this point. The user has the ability to define the `default` data. The native capabilites
The Nautobot ORM inventory is rather static in nature at this point. The user has the ability to define the `default` data. The native capabilities
include.

* Providing an object called within the `obj` key that is a Nautobot `Device` object instance.
Expand Down
13 changes: 12 additions & 1 deletion development/nautobot_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
from nautobot.core.settings import * # noqa: F403
from nautobot.core.settings_funcs import parse_redis_connection


#
# Misc. settings
#
Expand Down Expand Up @@ -132,6 +131,18 @@
# Each key in the dictionary is the name of an installed plugin and its value is a dictionary of settings.
PLUGINS_CONFIG = {
"nautobot_plugin_nornir": {
"connection_options": {
"napalm": {
"extras": {
"optional_args": {"global_delay_factor": 1},
},
},
"netmiko": {
"extras": {
"global_delay_factor": 1,
},
},
},
"nornir_settings": {
"credentials": "nautobot_plugin_nornir.plugins.credentials.env_vars.CredentialsEnvVars",
"runner": {
Expand Down
6 changes: 6 additions & 0 deletions nautobot_plugin_nornir/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,9 @@
NORNIR_SETTINGS = PLUGIN_CFG.get("nornir_settings", _NORNIR_SETTINGS)

DEFAULT_DRIVERS_MAPPING = PLUGIN_CFG.get("dispatcher_mapping", _DEFAULT_DRIVERS_MAPPING)

CONNECTION_SECRETS_PATHS = {
"netmiko": "netmiko.extras.secret",
"napalm": "napalm.extras.optional_args.secret",
"scrapli": "scrapli.extras.auth_secondary",
}
66 changes: 38 additions & 28 deletions nautobot_plugin_nornir/plugins/inventory/nautobot_orm.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,21 +5,21 @@

from django.db.models import QuerySet
from django.utils.module_loading import import_string
from nornir.core.inventory import (
Inventory,
ConnectionOptions,
Defaults,
Groups,
Host,
Hosts,
Group,
ParentGroups,
)

from nautobot.dcim.models import Device
from nautobot_plugin_nornir.constants import CONNECTION_SECRETS_PATHS, PLUGIN_CFG
from nornir.core.inventory import ConnectionOptions, Defaults, Group, Groups, Host, Hosts, Inventory, ParentGroups
from nornir_nautobot.exceptions import NornirNautobotException


def _set_dict_key_path(dictionary, key_path, value):
*keys, last_key = key_path.split(".")
pointer = dictionary
for key in keys:

pointer = pointer.setdefault(key, {})
pointer[last_key] = value

itdependsnetworks marked this conversation as resolved.
Show resolved Hide resolved

def _set_host(data: Dict[str, Any], name: str, groups, host, defaults) -> Host:
connection_option = {}
for key, value in data.get("connection_options", {}).items():
Expand Down Expand Up @@ -145,14 +145,7 @@ def create_host(self, device, cred, params: Dict):
Returns:
dict: Nornir Host dictionnary
"""
host = {
"data": {
"connection_options": {
"netmiko": {"extras": {}},
"napalm": {"extras": {"optional_args": {}}},
},
},
}
host = {"data": {}}
if "use_fqdn" in params and params.get("use_fqdn"):
host["hostname"] = f"{device.name}.{params.get('fqdn')}"
else:
Expand All @@ -165,9 +158,6 @@ def create_host(self, device, cred, params: Dict):
if not device.platform:
raise NornirNautobotException(f"Platform missing from device {device.name}")
host["platform"] = device.platform.slug
if device.platform.napalm_driver:
host["data"]["connection_options"]["napalm"]["platform"] = device.platform.napalm_driver

host["data"]["id"] = device.id
host["data"]["type"] = device.device_type.slug
host["data"]["site"] = device.site.slug
Expand All @@ -183,14 +173,34 @@ def create_host(self, device, cred, params: Dict):
# require password for now
host["password"] = password

# Use secret if specified.
# Netmiko
host["data"]["connection_options"]["netmiko"]["extras"]["secret"] = secret

# Napalm
host["data"]["connection_options"]["napalm"]["extras"]["optional_args"]["secret"] = secret
if PLUGIN_CFG.get("connection_options"):
for nornir_provider, nornir_options in PLUGIN_CFG["connection_options"].items():
if nornir_options.get("connection_secret_path"):
secret_path = nornir_options.pop("connection_secret_path")
elif CONNECTION_SECRETS_PATHS.get(nornir_provider):
secret_path = CONNECTION_SECRETS_PATHS["nornir_provider"]
else:
continue
_set_dict_key_path(PLUGIN_CFG["connection_options"], secret_path, secret)
else:
# Supporting, but not documenting, and will be deprecated in nautobot-plugin-nornir 2.X
host["data"]["connection_options"] = {
"netmiko": {"extras": {}},
"napalm": {"extras": {"optional_args": {}}},
}
# Use secret if specified.
# Netmiko
host["data"]["connection_options"]["netmiko"]["extras"]["secret"] = secret

# Napalm
host["data"]["connection_options"]["napalm"]["extras"]["optional_args"]["secret"] = secret

host["groups"] = self.get_host_groups(device=device)

if device.platform.napalm_driver:
if not host["data"]["connection_options"].get("napalm"):
host["data"]["connection_options"]["napalm"] = {}
host["data"]["connection_options"]["napalm"]["platform"] = device.platform.napalm_driver
jeffkala marked this conversation as resolved.
Show resolved Hide resolved
return host

@staticmethod
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
from unittest import skip

from django.test import TestCase
from nautobot.dcim.models import Device, Manufacturer, Site, DeviceType, DeviceRole
from nautobot.dcim.models import Device, DeviceRole, DeviceType, Manufacturer, Site
from nautobot_plugin_nornir.plugins.inventory.nautobot_orm import NautobotORMInventory


Expand Down