Merge pull request #17658 from netbox-community/develop

Release v4.1.3

.github/ISSUE_TEMPLATE/01-feature_request.yaml

@@ -14,7 +14,7 @@ body:
     attributes:
       label: NetBox version
       description: What version of NetBox are you currently running?
-      placeholder: v4.1.2
+      placeholder: v4.1.3
     validations:
       required: true
   - type: dropdown

.github/ISSUE_TEMPLATE/02-bug_report.yaml

@@ -26,7 +26,7 @@ body:
     attributes:
       label: NetBox Version
       description: What version of NetBox are you currently running?
-      placeholder: v4.1.2
+      placeholder: v4.1.3
     validations:
       required: true
   - type: dropdown

README.md

@@ -7,7 +7,11 @@
   <a href="https://github.com/netbox-community/netbox/stargazers"><img src="https://img.shields.io/github/stars/netbox-community/netbox?style=flat" alt="GitHub stars" /></a>
   <a href="https://explore.transifex.com/netbox-community/netbox/"><img src="https://img.shields.io/badge/languages-15-blue" alt="Languages supported" /></a>
   <a href="https://github.com/netbox-community/netbox/actions/workflows/ci.yml"><img src="https://github.com/netbox-community/netbox/workflows/CI/badge.svg?branch=master" alt="CI status" /></a>
-  <p></p>
+  <p>
+    <strong><a href="https://github.com/netbox-community/netbox/">NetBox Community</a></strong> |
+    <strong><a href="https://netboxlabs.com/netbox-cloud/">NetBox Cloud</a></strong> |
+    <strong><a href="https://netboxlabs.com/netbox-enterprise/">NetBox Enterprise</a></strong>
+  </p>
 </div>
 
 NetBox exists to empower network engineers. Since its release in 2016, it has become the go-to solution for modeling and documenting network infrastructure for thousands of organizations worldwide. As a successor to legacy IPAM and DCIM applications, NetBox provides a cohesive, extensive, and accessible data model for all things networked. By providing a single robust user interface and programmable APIs for everything from cable maps to device configurations, NetBox serves as the central source of truth for the modern network.
@@ -81,11 +85,6 @@ NetBox automatically logs the creation, modification, and deletion of all manage
 * The [official documentation](https://docs.netbox.dev) offers a comprehensive introduction.
 * Check out [our wiki](https://github.com/netbox-community/netbox/wiki/Community-Contributions) for even more projects to get the most out of NetBox!
 
-<p align="center">
-  <a href="https://netboxlabs.com/netbox-cloud/"><img src="docs/media/misc/netbox_cloud.png" alt="NetBox Cloud" /></a><br />
-  Looking for a managed solution? Check out <strong><a href="https://netboxlabs.com/netbox-cloud/">NetBox Cloud</a></strong> or <strong><a href="https://netboxlabs.com/netbox-enterprise/">NetBox Enterprise</a></strong>!
-</p>
-
 ## Get Involved
 
 * Follow [@NetBoxOfficial](https://twitter.com/NetBoxOfficial) on Twitter!

docs/features/synchronized-data.md

@@ -13,6 +13,9 @@ To enable remote data synchronization, the NetBox administrator first designates
 !!! info
     Data backends which connect to external sources typically require the installation of one or more supporting Python libraries. The Git backend requires the [`dulwich`](https://www.dulwich.io/) package, and the S3 backend requires the [`boto3`](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html) package. These must be installed within NetBox's environment to enable these backends.
 
+!!! info
+    If you are configuring Git and have `HTTP_PROXIES` configured to use the SOCKS protocol, you will also need to install the [`python_socks`](https://pypi.org/project/python-socks/) Python library.
+
 Each type of remote source has its own configuration parameters. For instance, a git source will ask the user to specify a branch and authentication credentials. Once the source has been created, a synchronization job is run to automatically replicate remote files in the local database.
 
 The following NetBox models can be associated with replicated data files:

docs/models/ipam/asn.md

@@ -1,14 +1,14 @@
 # ASNs
 
-An Autonomous System Number (ASN) is a numeric identifier used in the BGP protocol to identify which [autonomous system](https://en.wikipedia.org/wiki/Autonomous_system_%28Internet%29) a particular prefix is originating and transiting through. NetBox support both 32- and 64- ASNs.
+An Autonomous System Number (ASN) is a numeric identifier used in the Border Gateway Protocol (BGP) to identify which [autonomous system](https://en.wikipedia.org/wiki/Autonomous_system_%28Internet%29) a particular prefix is originating from or transiting through. NetBox supports both 16- and 32-bit ASNs.
 
 ASNs must be globally unique within NetBox, and may be allocated from within a [defined range](./asnrange.md). Each ASN may be assigned to multiple [sites](../dcim/site.md).
 
 ## Fields
 
 ### AS Number
 
-The 32- or 64-bit AS number.
+The 16- or 32-bit AS number.
 
 ### RIR
 

docs/release-notes/version-4.1.md

@@ -1,5 +1,17 @@
 # NetBox v4.1
 
+## v4.1.3 (2024-10-02)
+
+### Enhancements
+
+* [#17639](https://github.com/netbox-community/netbox/issues/17639) - Add SOCKS support to proxy settings for Git remote data sources
+
+### Bug Fixes
+
+* [#17558](https://github.com/netbox-community/netbox/issues/17558) - Raise validation error when attempting to remove a custom field choice in use
+
+---
+
 ## v4.1.2 (2024-09-26)
 
 ### Enhancements

netbox/core/data_backends.py

@@ -8,10 +8,13 @@
 
 from django import forms
 from django.conf import settings
+from django.core.exceptions import ImproperlyConfigured
 from django.utils.translation import gettext as _
 
 from netbox.data_backends import DataBackend
 from netbox.utils import register_data_backend
+from utilities.constants import HTTP_PROXY_SUPPORTED_SCHEMAS, HTTP_PROXY_SUPPORTED_SOCK_SCHEMAS
+from utilities.socks import ProxyPoolManager
 from .exceptions import SyncError
 
 __all__ = (
@@ -67,11 +70,18 @@ def init_config(self):
 
         # Initialize backend config
         config = ConfigDict()
+        self.use_socks = False
 
         # Apply HTTP proxy (if configured)
-        if settings.HTTP_PROXIES and self.url_scheme in ('http', 'https'):
-            if proxy := settings.HTTP_PROXIES.get(self.url_scheme):
-                config.set("http", "proxy", proxy)
+        if settings.HTTP_PROXIES:
+            if proxy := settings.HTTP_PROXIES.get(self.url_scheme, None):
+                if urlparse(proxy).scheme not in HTTP_PROXY_SUPPORTED_SCHEMAS:
+                    raise ImproperlyConfigured(f"Unsupported Git DataSource proxy scheme: {urlparse(proxy).scheme}")
+
+                if self.url_scheme in ('http', 'https'):
+                    config.set("http", "proxy", proxy)
+                    if urlparse(proxy).scheme in HTTP_PROXY_SUPPORTED_SOCK_SCHEMAS:
+                        self.use_socks = True
 
         return config
 
@@ -87,6 +97,10 @@ def fetch(self):
             "errstream": porcelain.NoneStream(),
         }
 
+        # check if using socks for proxy - if so need to use custom pool_manager
+        if self.use_socks:
+            clone_args['pool_manager'] = ProxyPoolManager(settings.HTTP_PROXIES.get(self.url_scheme))
+
         if self.url_scheme in ('http', 'https'):
             if self.params.get('username'):
                 clone_args.update(

netbox/extras/models/customfields.py

@@ -785,6 +785,12 @@ class Meta:
     def __str__(self):
         return self.name
 
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+
+        # Cache the initial set of choices for comparison under clean()
+        self._original_extra_choices = self.__dict__.get('extra_choices')
+
     def get_absolute_url(self):
         return reverse('extras:customfieldchoiceset', args=[self.pk])
 
@@ -818,6 +824,32 @@ def clean(self):
         if not self.base_choices and not self.extra_choices:
             raise ValidationError(_("Must define base or extra choices."))
 
+        # Check whether any choices have been removed. If so, check whether any of the removed
+        # choices are still set in custom field data for any object.
+        original_choices = set([
+            c[0] for c in self._original_extra_choices
+        ]) if self._original_extra_choices else set()
+        current_choices = set([
+            c[0] for c in self.extra_choices
+        ]) if self.extra_choices else set()
+        if removed_choices := original_choices - current_choices:
+            for custom_field in self.choices_for.all():
+                for object_type in custom_field.object_types.all():
+                    model = object_type.model_class()
+                    for choice in removed_choices:
+                        # Form the query based on the type of custom field
+                        if custom_field.type == CustomFieldTypeChoices.TYPE_MULTISELECT:
+                            query_args = {f"custom_field_data__{custom_field.name}__contains": choice}
+                        else:
+                            query_args = {f"custom_field_data__{custom_field.name}": choice}
+                        # Raise a ValidationError if there are any objects which still reference the removed choice
+                        if model.objects.filter(models.Q(**query_args)).exists():
+                            raise ValidationError(
+                                _(
+                                    "Cannot remove choice {choice} as there are {model} objects which reference it."
+                                ).format(choice=choice, model=object_type)
+                            )
+
     def save(self, *args, **kwargs):
 
         # Sort choices if alphabetical ordering is enforced

netbox/extras/tests/test_customfields.py

@@ -343,6 +343,74 @@ def test_multiselect_field(self):
         instance.refresh_from_db()
         self.assertIsNone(instance.custom_field_data.get(cf.name))
 
+    def test_remove_selected_choice(self):
+        """
+        Removing a ChoiceSet choice that is referenced by an object should raise
+        a ValidationError exception.
+        """
+        CHOICES = (
+            ('a', 'Option A'),
+            ('b', 'Option B'),
+            ('c', 'Option C'),
+            ('d', 'Option D'),
+        )
+
+        # Create a set of custom field choices
+        choice_set = CustomFieldChoiceSet.objects.create(
+            name='Custom Field Choice Set 1',
+            extra_choices=CHOICES
+        )
+
+        # Create a select custom field
+        cf = CustomField.objects.create(
+            name='select_field',
+            type=CustomFieldTypeChoices.TYPE_SELECT,
+            required=False,
+            choice_set=choice_set
+        )
+        cf.object_types.set([self.object_type])
+
+        # Create a multi-select custom field
+        cf_multiselect = CustomField.objects.create(
+            name='multiselect_field',
+            type=CustomFieldTypeChoices.TYPE_MULTISELECT,
+            required=False,
+            choice_set=choice_set
+        )
+        cf_multiselect.object_types.set([self.object_type])
+
+        # Assign a choice for both custom fields on an object
+        instance = Site.objects.first()
+        instance.custom_field_data[cf.name] = 'a'
+        instance.custom_field_data[cf_multiselect.name] = ['b', 'c']
+        instance.save()
+
+        # Attempting to delete a selected choice should fail
+        with self.assertRaises(ValidationError):
+            choice_set.extra_choices = (
+                ('b', 'Option B'),
+                ('c', 'Option C'),
+                ('d', 'Option D'),
+            )
+            choice_set.full_clean()
+
+        # Attempting to delete either of the multi-select choices should fail
+        with self.assertRaises(ValidationError):
+            choice_set.extra_choices = (
+                ('a', 'Option A'),
+                ('b', 'Option B'),
+                ('d', 'Option D'),
+            )
+            choice_set.full_clean()
+
+        # Removing a non-selected choice should succeed
+        choice_set.extra_choices = (
+            ('a', 'Option A'),
+            ('b', 'Option B'),
+            ('c', 'Option C'),
+        )
+        choice_set.full_clean()
+
     def test_object_field(self):
         value = VLAN.objects.create(name='VLAN 1', vid=1).pk
 

netbox/release.yaml

@@ -1,3 +1,3 @@
-version: "4.1.2"
+version: "4.1.3"
 edition: "Community"
-published: "2024-09-26"
+published: "2024-10-02"