From 44de0e8c8c2a8e9c911c35d0199d45f9cca4299b Mon Sep 17 00:00:00 2001 From: Ananya Muddukrishna Date: Wed, 14 Oct 2020 17:34:04 +0200 Subject: [PATCH 01/17] First draft --- articles/unique_network_flows.md | 150 +++++++++++++++++++++++++++++++ 1 file changed, 150 insertions(+) create mode 100644 articles/unique_network_flows.md diff --git a/articles/unique_network_flows.md b/articles/unique_network_flows.md new file mode 100644 index 000000000..6d72ec775 --- /dev/null +++ b/articles/unique_network_flows.md @@ -0,0 +1,150 @@ +--- +layout: default +title: Unique Network Flow Identifiers +permalink: articles/content_filtering.html +abstract: Enable unique network flow identifiers for publishers and subscribers in communicating nodes +author: '[Ananya Muddukrishna](https://github.com/anamud)' +published: true +--- + +{:toc} + +# {{ page.title }} + +
+{{ page.abstract }} +
+ +Original Author: {{ page.author }} + +# Unique Network Flow Identifiers + +For performance, ROS2 applications require careful selection of QoS for publishers and subscribers. Although networks offer various QoS options, ROS2 publishers and subscribers are unable to use them due to non-unique flow identifiers. As a result, the QoS provided by networks to ROS2 publishers and subscribers is vanilla and uniform. This ultimately degrades the performance potential of ROS2 applications and wastes networking infrastructure investments. + +We propose unique network flow identifiers for ROS2 publishers and subscribers. Our proposal is easy to use, convenient to implement, and minimal. Plus, it respects network-agnostic and non-DDS-middleware-friendly concerns in ROS2. + +In this document, we first describe essential background concepts. After that we precisely state the problem and propose a solution to the problem. Existing solutions are compared in the end. + +## Background + +IP networking [1] is the pre-dominant inter-networking technology used today. Ethernet, WiFi, 4G/5G telecommunication all rely on IP networking. + +Streams of IP packets from a given source to destination are called *flows*. Applications can uniquely identify certain flows and explicitly specify what QoS is required from the network for those flows. + +### Flow Identifers + +The *5-tuple* is a traditional unique identifier for flows. The 5-tuple consists of five parameters: source IP address, source port, destination IP address, destination port, and the transport protocol (example, TCP/UDP). + +IPv6 specifies a *3-tuple* for uniquely identifying flows. The IPv6 3-tuple consists of the source IP address, destination IP address, and the Flow Label. The Flow Label [1] is a 20-bit field in the IPv6 header. It is typically set by the source of the flow. The default flow label is zero. + +### Explicit QoS Specification + +We briefly discuss two relevant explicit QoS specification methods for applications -- Differentiated Services and 5G network 5QI. + +- Differentiated Services (DS) [2] is a widely-used QoS architecture for IP networks. The required DS-based QoS is set by the application in the 6-bit DS Code Point (DSCP) sub-field of the 8-bit DS field in the IP packet header. For example, DSCP set to 0x2E specifies expedited forwarding as the required QoS. Expedited forwarding is typically used for real-time data such as voice and video. + + ROS2 lacks an API to specify DS-based QoS for publishers and subscribers. The DSCP value in their flows is therefore set to 0x00. This specifies default forwarding as the required QoS from the network. However, DDS provides the Transport Priority QoS policy to specify DS-based QoS. + + A frustrating problem with DS-based QoS is that intermediate routers can reset or alter the DSCP value of flows. One workaround is to carefully configure intermediate routers to retain DSCP markings from incoming to outgoing flows. + +- 5G network 5QI: The Network Exposure Function (NEF) [3] in the 5G core network provides robust and secure API for QoS specification. This API enables applications to programmatically specify required QoS by associating 5G Quality Indicators (5QIs) to flow identifers. Twenty-six standard 5QIs are identified in the latest release-16 by 3GPP [3:Table 5.7.4-1]. We exemplify a few of them below. The variation in service characteristics of the example 5QIs emphasizes the importance of careful 5QI selection. + +| 5QI | Resource | Priority | Packet Delay Budget (ms) | Packet Error Rate | Example Services | +| ----------- | ------------------------------------------ | -------- | ------------------------ | ----------------- | ------------------------------------------------------------------------------------------------------------------------| +| 3 | Guaranteed bitrate (GBR) | 30 | 50 | 10^-3 | Real Time Gaming; V2X messages; Electricity distribution – medium voltage; Process automation monitoring. | +| 4 | GBR | 50 | 300 | 10^-6 | Non-Conversational Video (Buffered Streaming) | +| 7 | Non GBR (NGBR) | 70 | 100 | 10^-3 | Voice Video (Live Streaming); Interactive Gaming | +| 9 (default) | NGBR | 90 | 300 | 10^-6 | Video (Buffered Streaming); TCP-based traffic (e.g., www, e-mail, chat, ftp, p2p file sharing, progressive video, etc.) | +| 82 | Delay critical guaranteed bitrate (DC GBR) | 19 | 10 | 10^-4 | Discrete Automation | +| 85 | DC GBR | 21 | 5 | 10^-5 | Electricity distribution - high voltage; V2X messages (Remote Driving) | + +The 5G network also has the ability to sensibly infer 5QI QoS from DS-based QoS markings in flows. + +## Problem + +All publishers and subscribers in communicating nodes have the same flow identifers (5-tuple or 3-tuple). This disables explicit network QoS differentiation for publishers and subscribers in communicating nodes. In other words, publishers and subscribers in communicating nodes can only be assigned the same network QoS. + +We believe the problem occurs by design. For performance reasons, RMW implementations are likely to associate IP address, port, and transport protocol to nodes and not to individual publishers/subscribers. This is true for all the tier-1 RMW implementations today (Eloquent Elusor at the time of writing). None of the tier-1 RMW implementations set the IPv6 flow label to differentiate flows of publisher/subscribers. + +### Example + +We use the following example to highlight the problem. + +Consider a distributed robotics application with nodes N1 and N2. N1 is active on device D1 and N2 on device D2. D1 and D2 are connected by an IP network, say a 5G network. + +N1 contains publishers P1 and P2. P1 publishes video data whereas P2 publishes battery status data. + +N2 contains subscribers S1 and S2. S1 receives video from P1 and performs real-time object detection. S2 receives battery data from P2 and performs non-real-time battery management. + +The link P1-S1 requires low-latency QoS from the 5G network, say a maximum delay of 5ms. P2-S2 requires default latency QoS i.e., 300ms. Then, by construction, since the flow identifiers of P1-S1 and P2-S2 links are the same, they cannot be assigned the required QoS by the network. Both P1-S1 and P2-S2 can either be assigned QoS with 5ms delay or with 300ms delay. The former case represents a waste of network resources, the latter case degrades performance. + +## Proposed Solution + +Our proposal to solve the problem is to make the flow identifiers of publishers and subscribers in communicating nodes unique using existing IP and transport header parameters. + +### Network Flow: a new QoS policy + +We construct a new QoS policy called *Network Flow* as a support structure to enable unique identification of flows. + +The Network Flow QoS policy is parameterized by just one parameter -- a 32-bit unsigned integer called `flow_id`. The default `flow_id` is zero. + +Each publisher or subscriber that requires a specific QoS from the network is required to set the `flow_id` parameter to value unique within the node. The `flow_id` can be computed internally in the node or supplied by an external component. + +Eventually the `flow_id` is communicated to the RMW implementation where it is converted to a unique flow identifier. + +The example C++ snippet below shows a node creating a publisher (`pub1`) and a subscriber (`sub1`) with unique `flow_id` values. Subscriber `sub2` is created with the default Network Flow QoS policy (`flow_id` = 0). + +```cpp +auto pub1_qos = rclcpp::QoS(rclcpp::KeepLast(10)).network_flow(0xf4688c52); +pub1 = create_publisher("pub1_topic", pub1_qos); + +auto sub1_qos = rclcpp::QoS(rclcpp::KeepLast(1)).network_flow(0x88181c45); +sub1 = create_subscription("sub1_topic", sub1_qos, std::bind(&new_sub1_message, this, _1)); + +auto sub2_qos = rclcpp::QoS(rclcpp::KeepLast(1)); +sub2 = create_subscription("sub2_topic", sub2_qos, std::bind(&new_sub2_message, this, _1)); +``` + +A programmer-friendly alternative is to parameterize the Network Flow QoS policy with just a boolean parameter called `unique_flow`. Setting `unique_flow` to `true` instructs the RMW implementation to generate a unique flow identifier for the publisher/subscriber associated with the Network Flow QoS policy. By default, `unique_flow` is set to `false`. + +The RMW implementation has several options to convert `flow_id` to a unique flow identifier. + +If the node is communicating using IPv6, then the lower 20-bits of `flow_id` can be transferred to the Flow Label field. This creates a unique 3-tuple. + +Else, if the node is communicating via IPv4, then there are two alternatives. One is copy the `flow_id` to the Options field. Another is to copy the lower 6-bits of the `flow_id` to the DSCP field. Both alternatives enable the network to infer a custom 6-tuple (traditional 5-tuple plus Options/DSCP) that uniquely identifies flows. + +The parameter `unique_flow` can be similarly handled with a minor difference -- the RMW implementation generates a `flow_id` internally first. + +Both DDS and non-DDS RMW implementations can trivially set fields in IP headers using existing socket API on all ROS2 platforms (Linux, Windows, MacOS). Generating the `flow_id` internally is a matter of implementing a simple hashing functions parameterized by exiting unique identifiers for publishers/subscribers within a node. + +### Advantages + +Our proposal has the following advantages: + +- Easy to use: Application developers are only required to decide if unique flow identifiers for publishers/subscribers are necessary. If yes, they can supply the unique identifiers themselves, or obtain it from external components, or delegate unique identifier generation to the RMW implementation. +- Light-weight implementation: Both non-DDS and DDS RMW can implement the required support conveniently with negligible impact on performance. +- Network-agnostic: No particular network is targeted, respecting ROS2 design preferences. +- Minimum change: Introducing the *choice* of unique flow identifiers into the application layer is the minimum framework change required to obtain bespoke QoS from machine-machine network technologies such as 5G. + +### Limitations + +If the RMW implementation decides to create a custom 6-tuple using the IPv4 DSCP field, the flows of only up to 64 (2^6) publishers and subscribers within a node can be uniquely identified. In addition, network administration processes should be notified that the DSCP field is re-purposed as an identifier to prevent misinterpretation. + +## Alternative Solutions + +We list a few alternative solutions to the problem that are limited and dissatisfactory. + +1. Dedicated nodes: Publishers and subscribers that require special network QoS can be isolated to dedicated nodes. Such isolation indeed makes their 5-tuple flow identifier unique. However, this breaks the functionality-based node architecture of the application and degrades performance since nodes are heavy-weight structures. In the worst case, a dedicated node per publisher or subscriber is required. + +2. Side-loaded DDS Transport Priority QoS policies: Conceptually, the custom 6-tuple can be constructed by side-loading unique `flow_id` values into the Transport Priority QoS policy of the DDS RMW implementation. In practice, however, this is difficult to implement for two reasons. First, it expects DDS RMW side-loading competence from application programmers which is inconvenient. Second, side-loading support varies across DDS RMW implementations. To the best of our knowledge, none of the tier-1 DDS implementations for ROS2 today (Eloquent) support side-loading Transport Priority QoS policies for *select few* publishers and subscribers in a node due to lack of fine-grained interfaces. A glaring limitation is that this alternative ignores non-DDS RMW. + +## References + +[1] [Internet Protocol (IETF RFC-791)](https://tools.ietf.org/html/rfc791) + +[2] [IPv6 Flow Label Specification (IETF RFC-6437)](https://tools.ietf.org/html/rfc6437) + +[3] [Differentiated Services (IETF RFC-2474)](https://tools.ietf.org/html/rfc2474) + +[4] [5G System Architecture Specification (3GPP TS 23.501)](https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3144) + From 9e8c1568d6e22d32b2a210048079d8fbbc1c6b44 Mon Sep 17 00:00:00 2001 From: Ananya Muddukrishna Date: Wed, 14 Oct 2020 18:12:21 +0200 Subject: [PATCH 02/17] Update with better argumentation --- articles/unique_network_flows.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/articles/unique_network_flows.md b/articles/unique_network_flows.md index 6d72ec775..2235222c9 100644 --- a/articles/unique_network_flows.md +++ b/articles/unique_network_flows.md @@ -35,7 +35,7 @@ Streams of IP packets from a given source to destination are called *flows*. App The *5-tuple* is a traditional unique identifier for flows. The 5-tuple consists of five parameters: source IP address, source port, destination IP address, destination port, and the transport protocol (example, TCP/UDP). -IPv6 specifies a *3-tuple* for uniquely identifying flows. The IPv6 3-tuple consists of the source IP address, destination IP address, and the Flow Label. The Flow Label [1] is a 20-bit field in the IPv6 header. It is typically set by the source of the flow. The default flow label is zero. +IPv6 specifies a *3-tuple* for uniquely identifying flows. The IPv6 3-tuple consists of the source IP address, destination IP address, and the Flow Label. The Flow Label [1] is a 20-bit field in the IPv6 header. It is typically set by the source of the flow. The default Flow Label is zero. ### Explicit QoS Specification @@ -51,7 +51,7 @@ We briefly discuss two relevant explicit QoS specification methods for applicati | 5QI | Resource | Priority | Packet Delay Budget (ms) | Packet Error Rate | Example Services | | ----------- | ------------------------------------------ | -------- | ------------------------ | ----------------- | ------------------------------------------------------------------------------------------------------------------------| -| 3 | Guaranteed bitrate (GBR) | 30 | 50 | 10^-3 | Real Time Gaming; V2X messages; Electricity distribution – medium voltage; Process automation monitoring. | +| 3 | Guaranteed bitrate (GBR) | 30 | 50 | 10^-3 | Real Time Gaming; V2X messages; Electricity distribution – medium voltage; Process automation monitoring | | 4 | GBR | 50 | 300 | 10^-6 | Non-Conversational Video (Buffered Streaming) | | 7 | Non GBR (NGBR) | 70 | 100 | 10^-3 | Voice Video (Live Streaming); Interactive Gaming | | 9 (default) | NGBR | 90 | 300 | 10^-6 | Video (Buffered Streaming); TCP-based traffic (e.g., www, e-mail, chat, ftp, p2p file sharing, progressive video, etc.) | @@ -70,7 +70,7 @@ We believe the problem occurs by design. For performance reasons, RMW implementa We use the following example to highlight the problem. -Consider a distributed robotics application with nodes N1 and N2. N1 is active on device D1 and N2 on device D2. D1 and D2 are connected by an IP network, say a 5G network. +Consider a distributed robotics application with communicating nodes N1 and N2. N1 is active on device D1 and N2 on device D2. D1 and D2 are connected by an IP network, say a 5G network. N1 contains publishers P1 and P2. P1 publishes video data whereas P2 publishes battery status data. @@ -80,7 +80,7 @@ The link P1-S1 requires low-latency QoS from the 5G network, say a maximum delay ## Proposed Solution -Our proposal to solve the problem is to make the flow identifiers of publishers and subscribers in communicating nodes unique using existing IP and transport header parameters. +Our proposal to solve the problem is to make the flow identifiers of publishers and subscribers in communicating nodes unique using existing IP and transport header fields. ### Network Flow: a new QoS policy @@ -115,15 +115,15 @@ Else, if the node is communicating via IPv4, then there are two alternatives. On The parameter `unique_flow` can be similarly handled with a minor difference -- the RMW implementation generates a `flow_id` internally first. -Both DDS and non-DDS RMW implementations can trivially set fields in IP headers using existing socket API on all ROS2 platforms (Linux, Windows, MacOS). Generating the `flow_id` internally is a matter of implementing a simple hashing functions parameterized by exiting unique identifiers for publishers/subscribers within a node. +Both DDS and non-DDS RMW implementations can trivially set fields in IP headers using existing socket API on all ROS2 platforms (Linux, Windows, MacOS). Generating the `flow_id` internally is a matter of implementing a simple hashing functions parameterized by exiting unique identifiers for publishers/subscribers within a node. A similar hashing scheme can be used by application programmers to set `flow_id` in the Network Flow QoS policy. ### Advantages Our proposal has the following advantages: -- Easy to use: Application developers are only required to decide if unique flow identifiers for publishers/subscribers are necessary. If yes, they can supply the unique identifiers themselves, or obtain it from external components, or delegate unique identifier generation to the RMW implementation. +- Easy to use: Application developers are only required to decide if unique flow identifiers for publishers/subscribers are necessary. If yes, they can easily generate the unique identifiers themselves, or obtain it from external components, or delegate unique identifier generation to the RMW implementation. - Light-weight implementation: Both non-DDS and DDS RMW can implement the required support conveniently with negligible impact on performance. -- Network-agnostic: No particular network is targeted, respecting ROS2 design preferences. +- Network-agnostic: No particular network is preferred, respecting ROS2 design preferences. - Minimum change: Introducing the *choice* of unique flow identifiers into the application layer is the minimum framework change required to obtain bespoke QoS from machine-machine network technologies such as 5G. ### Limitations @@ -136,7 +136,9 @@ We list a few alternative solutions to the problem that are limited and dissatis 1. Dedicated nodes: Publishers and subscribers that require special network QoS can be isolated to dedicated nodes. Such isolation indeed makes their 5-tuple flow identifier unique. However, this breaks the functionality-based node architecture of the application and degrades performance since nodes are heavy-weight structures. In the worst case, a dedicated node per publisher or subscriber is required. -2. Side-loaded DDS Transport Priority QoS policies: Conceptually, the custom 6-tuple can be constructed by side-loading unique `flow_id` values into the Transport Priority QoS policy of the DDS RMW implementation. In practice, however, this is difficult to implement for two reasons. First, it expects DDS RMW side-loading competence from application programmers which is inconvenient. Second, side-loading support varies across DDS RMW implementations. To the best of our knowledge, none of the tier-1 DDS implementations for ROS2 today (Eloquent) support side-loading Transport Priority QoS policies for *select few* publishers and subscribers in a node due to lack of fine-grained interfaces. A glaring limitation is that this alternative ignores non-DDS RMW. +2. Custom 6-tuple using side-loaded DDS Transport Priority QoS policies: Conceptually, the custom 6-tuple can be constructed by side-loading unique `flow_id` values into the Transport Priority QoS policy of the DDS RMW implementation. In practice, however, this is difficult to implement for several reasons. First, it expects DDS RMW side-loading competence from application programmers which is inconvenient. Second, re-purposing DSCP values as identifiers is limited to 64 identifiers and requires careful network administration as mentioned before. Third, side-loading support varies across DDS RMW implementations. To the best of our knowledge, none of the tier-1 DDS implementations for ROS2 today (Eloquent) support side-loading Transport Priority QoS policies for *select few* publishers and subscribers in a node due to lack of fine-grained interfaces. A glaring limitation is that this alternative ignores non-DDS RMW. + +3. DS-based QoS using side-loaded DDS Transport Priority QoS policies: This gets ahead of the problem by directly specifying the required DS-based QoS through side-loaded Transport Priority QoS policies. However, this suffers from similar problems as the previous alternative. It ignores non-DDS RMW, expects DS competence from programmers, and is not supported by tier-1 RMW implementations. ## References From c1f6f97b4810c39b6fa8efca0ae16de6e6e84d8f Mon Sep 17 00:00:00 2001 From: Ananya Muddukrishna Date: Thu, 15 Oct 2020 12:03:08 +0200 Subject: [PATCH 03/17] Fix typos and argumentation --- articles/unique_network_flows.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/articles/unique_network_flows.md b/articles/unique_network_flows.md index 2235222c9..0323d9303 100644 --- a/articles/unique_network_flows.md +++ b/articles/unique_network_flows.md @@ -35,19 +35,19 @@ Streams of IP packets from a given source to destination are called *flows*. App The *5-tuple* is a traditional unique identifier for flows. The 5-tuple consists of five parameters: source IP address, source port, destination IP address, destination port, and the transport protocol (example, TCP/UDP). -IPv6 specifies a *3-tuple* for uniquely identifying flows. The IPv6 3-tuple consists of the source IP address, destination IP address, and the Flow Label. The Flow Label [1] is a 20-bit field in the IPv6 header. It is typically set by the source of the flow. The default Flow Label is zero. +IPv6 specifies a *3-tuple* for uniquely identifying flows. The IPv6 3-tuple consists of the source IP address, destination IP address, and the Flow Label. The Flow Label [2] is a 20-bit field in the IPv6 header. It is typically set by the source of the flow. The default Flow Label is zero. ### Explicit QoS Specification We briefly discuss two relevant explicit QoS specification methods for applications -- Differentiated Services and 5G network 5QI. -- Differentiated Services (DS) [2] is a widely-used QoS architecture for IP networks. The required DS-based QoS is set by the application in the 6-bit DS Code Point (DSCP) sub-field of the 8-bit DS field in the IP packet header. For example, DSCP set to 0x2E specifies expedited forwarding as the required QoS. Expedited forwarding is typically used for real-time data such as voice and video. +- Differentiated Services (DS) [3] is a widely-used QoS architecture for IP networks. The required DS-based QoS is set by the application in the 6-bit DS Code Point (DSCP) sub-field of the 8-bit DS field in the IP packet header. For example, DSCP set to 0x2E specifies expedited forwarding as the required QoS. Expedited forwarding is typically used for real-time data such as voice and video. ROS2 lacks an API to specify DS-based QoS for publishers and subscribers. The DSCP value in their flows is therefore set to 0x00. This specifies default forwarding as the required QoS from the network. However, DDS provides the Transport Priority QoS policy to specify DS-based QoS. - A frustrating problem with DS-based QoS is that intermediate routers can reset or alter the DSCP value of flows. One workaround is to carefully configure intermediate routers to retain DSCP markings from incoming to outgoing flows. + A frustrating problem with DS-based QoS is that intermediate routers can reset or alter the DSCP value within flows. One workaround is to carefully configure intermediate routers such that they retain DSCP markings from incoming to outgoing flows. -- 5G network 5QI: The Network Exposure Function (NEF) [3] in the 5G core network provides robust and secure API for QoS specification. This API enables applications to programmatically specify required QoS by associating 5G Quality Indicators (5QIs) to flow identifers. Twenty-six standard 5QIs are identified in the latest release-16 by 3GPP [3:Table 5.7.4-1]. We exemplify a few of them below. The variation in service characteristics of the example 5QIs emphasizes the importance of careful 5QI selection. +- 5G network 5QI: The Network Exposure Function (NEF) [4] in the 5G core network provides robust and secure API for QoS specification. This API enables applications to programmatically specify required QoS by associating 5G Quality Indicators (5QIs) to flow identifers. Twenty-six standard 5QIs are identified in the latest release-16 by 3GPP [4:Table 5.7.4-1]. We exemplify a few of them below. The variation in service characteristics of the example 5QIs emphasizes the importance of careful 5QI selection. | 5QI | Resource | Priority | Packet Delay Budget (ms) | Packet Error Rate | Example Services | | ----------- | ------------------------------------------ | -------- | ------------------------ | ----------------- | ------------------------------------------------------------------------------------------------------------------------| @@ -138,7 +138,7 @@ We list a few alternative solutions to the problem that are limited and dissatis 2. Custom 6-tuple using side-loaded DDS Transport Priority QoS policies: Conceptually, the custom 6-tuple can be constructed by side-loading unique `flow_id` values into the Transport Priority QoS policy of the DDS RMW implementation. In practice, however, this is difficult to implement for several reasons. First, it expects DDS RMW side-loading competence from application programmers which is inconvenient. Second, re-purposing DSCP values as identifiers is limited to 64 identifiers and requires careful network administration as mentioned before. Third, side-loading support varies across DDS RMW implementations. To the best of our knowledge, none of the tier-1 DDS implementations for ROS2 today (Eloquent) support side-loading Transport Priority QoS policies for *select few* publishers and subscribers in a node due to lack of fine-grained interfaces. A glaring limitation is that this alternative ignores non-DDS RMW. -3. DS-based QoS using side-loaded DDS Transport Priority QoS policies: This gets ahead of the problem by directly specifying the required DS-based QoS through side-loaded Transport Priority QoS policies. However, this suffers from similar problems as the previous alternative. It ignores non-DDS RMW, expects DS competence from programmers, and is not supported by tier-1 RMW implementations. +3. DS-based QoS using side-loaded DDS Transport Priority QoS policies: This gets ahead of the problem by directly specifying the required DS-based QoS through side-loaded Transport Priority QoS policies. However, this suffers from similar impracticalities as the previous alternative. It ignores non-DDS RMW, expects DS competence from programmers, and is not supported by tier-1 RMW implementations. ## References From 82c11f982b9dfe4c325c628d6931b275c2975458 Mon Sep 17 00:00:00 2001 From: Ananya Muddukrishna Date: Thu, 15 Oct 2020 12:07:24 +0200 Subject: [PATCH 04/17] Update metadata --- articles/unique_network_flows.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/articles/unique_network_flows.md b/articles/unique_network_flows.md index 0323d9303..21c27fdde 100644 --- a/articles/unique_network_flows.md +++ b/articles/unique_network_flows.md @@ -1,9 +1,9 @@ --- layout: default title: Unique Network Flow Identifiers -permalink: articles/content_filtering.html +permalink: articles/unique_network_flows.html abstract: Enable unique network flow identifiers for publishers and subscribers in communicating nodes -author: '[Ananya Muddukrishna](https://github.com/anamud)' +author: '[Ananya Muddukrishna, Ericsson AB](https://github.com/anamud)' published: true --- From da504e0283d004566f784419297caf2047e34677 Mon Sep 17 00:00:00 2001 From: Ananya Muddukrishna Date: Fri, 16 Oct 2020 07:41:12 +0200 Subject: [PATCH 05/17] Update distro to Foxy --- articles/unique_network_flows.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/articles/unique_network_flows.md b/articles/unique_network_flows.md index 21c27fdde..7cabe48a7 100644 --- a/articles/unique_network_flows.md +++ b/articles/unique_network_flows.md @@ -64,7 +64,7 @@ The 5G network also has the ability to sensibly infer 5QI QoS from DS-based QoS All publishers and subscribers in communicating nodes have the same flow identifers (5-tuple or 3-tuple). This disables explicit network QoS differentiation for publishers and subscribers in communicating nodes. In other words, publishers and subscribers in communicating nodes can only be assigned the same network QoS. -We believe the problem occurs by design. For performance reasons, RMW implementations are likely to associate IP address, port, and transport protocol to nodes and not to individual publishers/subscribers. This is true for all the tier-1 RMW implementations today (Eloquent Elusor at the time of writing). None of the tier-1 RMW implementations set the IPv6 flow label to differentiate flows of publisher/subscribers. +We believe the problem occurs by design. For performance reasons, RMW implementations are likely to associate IP address, port, and transport protocol to nodes and not to individual publishers/subscribers. This is true for all the tier-1 RMW implementations today (Foxy Fitzroy at the time of writing). None of the tier-1 RMW implementations set the IPv6 flow label to differentiate flows of publisher/subscribers. ### Example @@ -136,7 +136,7 @@ We list a few alternative solutions to the problem that are limited and dissatis 1. Dedicated nodes: Publishers and subscribers that require special network QoS can be isolated to dedicated nodes. Such isolation indeed makes their 5-tuple flow identifier unique. However, this breaks the functionality-based node architecture of the application and degrades performance since nodes are heavy-weight structures. In the worst case, a dedicated node per publisher or subscriber is required. -2. Custom 6-tuple using side-loaded DDS Transport Priority QoS policies: Conceptually, the custom 6-tuple can be constructed by side-loading unique `flow_id` values into the Transport Priority QoS policy of the DDS RMW implementation. In practice, however, this is difficult to implement for several reasons. First, it expects DDS RMW side-loading competence from application programmers which is inconvenient. Second, re-purposing DSCP values as identifiers is limited to 64 identifiers and requires careful network administration as mentioned before. Third, side-loading support varies across DDS RMW implementations. To the best of our knowledge, none of the tier-1 DDS implementations for ROS2 today (Eloquent) support side-loading Transport Priority QoS policies for *select few* publishers and subscribers in a node due to lack of fine-grained interfaces. A glaring limitation is that this alternative ignores non-DDS RMW. +2. Custom 6-tuple using side-loaded DDS Transport Priority QoS policies: Conceptually, the custom 6-tuple can be constructed by side-loading unique `flow_id` values into the Transport Priority QoS policy of the DDS RMW implementation. In practice, however, this is difficult to implement for several reasons. First, it expects DDS RMW side-loading competence from application programmers which is inconvenient. Second, re-purposing DSCP values as identifiers is limited to 64 identifiers and requires careful network administration as mentioned before. Third, side-loading support varies across DDS RMW implementations. To the best of our knowledge, none of the tier-1 DDS implementations for ROS2 today (Foxy) support side-loading Transport Priority QoS policies for *select few* publishers and subscribers in a node due to lack of fine-grained interfaces. A glaring limitation is that this alternative ignores non-DDS RMW. 3. DS-based QoS using side-loaded DDS Transport Priority QoS policies: This gets ahead of the problem by directly specifying the required DS-based QoS through side-loaded Transport Priority QoS policies. However, this suffers from similar impracticalities as the previous alternative. It ignores non-DDS RMW, expects DS competence from programmers, and is not supported by tier-1 RMW implementations. From 4612e8cb40f9180802b885dcc89c8245de42d4c0 Mon Sep 17 00:00:00 2001 From: Ananya Muddukrishna Date: Thu, 5 Nov 2020 15:42:07 +0100 Subject: [PATCH 06/17] Improve argumentation based on discussion in middleware group --- .../ros2-app-5gs-network-programmability.png | Bin 0 -> 37600 bytes .../unique-network-flows-arch-options.png | Bin 0 -> 24111 bytes articles/unique_network_flows.md | 55 ++++++++++++------ 3 files changed, 37 insertions(+), 18 deletions(-) create mode 100755 articles/ros2-app-5gs-network-programmability.png create mode 100755 articles/unique-network-flows-arch-options.png diff --git a/articles/ros2-app-5gs-network-programmability.png b/articles/ros2-app-5gs-network-programmability.png new file mode 100755 index 0000000000000000000000000000000000000000..eb88d46601a0cfd4df8f24a418555c979accbd94 GIT binary patch literal 37600 zcmaHT1z3}P*zZ6<7y?R4qezI--5mxnLXk#EM|U?!NlAA}r_^ASlpu{TIz_sr8L;z= z_x<8r=R4nXUCyyRasT74-~GE622oKY08@cMAP~U|B{_8v2#W;-!W6~D0X|tgJoEwn z!*EnrlmS%?(QE>LV3|uROM^h)qVccZV*`KV*(qr|f)S+y`dW%pG=q zN>P`x6mH{BFf8;LQud5OYPl|Zb02w`Pj{wQ%3stP_YjV?dVAX5O5pcf9u`Gh2St?h zeh+%CTR+f*p2L4Ho2mV{G7nDLLeDt}6plSF@-n`fd8Xcao1IDB=)=+4pwss4SMgXp zEHC3R@7LAUEvVh*?k^X{itu5eKZUU2s>C-KS%*!O>w3a& z&toO{e@D-)qlh=sOC##KcbrJf4oLPoQ*2t7$E+*#!CFaa#ot8YH(FZ%aXlRtOZZgD zo6-S5-{XX046D0uCU#blr6#SBMyN{{2Kp&+hT;l_UTz7hyF`H%v)(laaBl* z>#b+kp6fpTtZ*^)%~-K!RN^18_`n~?bKr1zGY+#Ha?<|x!%*t>u*altu6QQF&NR25 ze0d~vJ221(dAr=MA_X}S*?@v3+S~8Y(=Vg&`QiJBcQy;n(?wC6zyn%5i9fdZii#7W zPw_)vC0A>^xG<2+-MP?uxmCLR)wKLfqrU6wi#V(HeqEc!ZWVm;=N9$<+_F1hNsjO- zLySu7&tQBC57@8CQZ3HeH?q+g!j1(QiR`*&ID|AC)}!Cs=KuVR+x_YYwz=JPQr{Sa zVr9V(oi;|=aR_oP(qb7E&1kaiWFz1r#1#c{cD-s8G>5i@VX?xuqIZWc$wjd$aam40 z_w*demKMNWE<}>WLtO9KA2~f;`UXX?rBUXH5SKj7@y5CHNT;H-v?hhP$#uVe_M>+6 z?KaAO_#nOa0XFDL``(tygFnNGBlRAynW0dq6{OR7ccdDXtOB!{-*EveZHSRR$D93k zLZs{-D+bIn2QZoxS;|-K$qHcHnPiuf4l#+Ibm0gL{C>N$vy*S{O@H<4XE=k!t;C@A zuyi$cTuJBR1J@M@PGFlvPyl6n52+4U_U&3r(UhE)!XG=x)X{bDFq2~8n!PBB==qiZ zswH0z*wwT33+gvT@63>=yDfnA0FK|%YjT8$iHT2Uj99>~!bPu;E>s1HJ5kunznk^1 zAP?`S_ddcVizkRJYYN)atum7C`|el&Qwv74tA)Is`s{}z<&3I)xu4qBf&z%+n3b#Z z+@9bYU=^zb!^4}8hC~-qcQzXeGd)TOR7@OhX94twq&-YbO-h=dl?EGYeD}_+Ku;bE z#j*gp5?`X|L!wV|7Zg4ig=DzJx|?qZ!e4SB<&g?;oq1=TM7PTzK>r% zjuk}0;w}0U+0B+af)~^sW_qG%MR#_Wxc{2|_)KZGGxcqO*v~;h`ZXX*Kr#qIhpX~O z4Q=D@=QI{D^q@_f?d14)5TSGlNTGt44p3ewdW*ATuxg>_f*7!&5W0hxmUZE`orUj9zRc@21KYo|E0Oad5+iGQ7zI+1V16AReB0=bRouy=E2)b7ZemXwY|P z1gn(kf19>)TZXM2d8163=N!Xe^>zz$&7H5Q4)+(-9ciP*2GX71b2@%lv{^{~eO7oT zDPSmi%wSu5kjUxQF2;BCw8o@AV3wRA1&j zraNZW9JMGhK7LPxW5u%Jd@+mr)xPJ#oZV#fYgneDn`K0nXG?CnWPKYY*Iz!{r?Cr` zb+0B7ZDwhMSqsRi3gpcDyD1ecUj4^yMa;)k3f^mT_5zhYopV0j!-Lssg=z04_txiU z9e_b)i``FWC4S$Im-KYG&@TY`A{1Yt;T|4^5Tr_GUQREu`wo$q5xxnV)N(_1u&f=X z-`JO>ASxD4$(qq-y+xrbuYJQE5q|K>J(Z1f#gmPla_fU`a};7~V@v#xoa13v`+|6P zKg#QOL*|!{b{`fqhX>mh&tnLP8$Ru75q&h5xLhKsSxs1+-L7O94Qk@#E}!x0_S8Ie zYA#M%v|RAH7H=}7-KQr!|02Fnf9b4~wRz$=DXuNs`c9eFyYk|oC3;|UJ$Ll!?$&uJ;Qjp*vW1I&q8BrDnkb+X%@?qkUFO5vVKSx^U`Zr;vHZ{S^=q`ThM4$(4WYkAqEm6(`B{h~s-U2VJfe zE;b`y;{Yx;bG-60*aZ6bE;EQhtJU`><=PEG!-`qv>DRVxv6G0KxHhG$wz`K;BMl}6 zD!$7V#CKGF4^ltR3y=j9Uu~>8TFYwb(HMpjPVaV;mWy(nK4`S>@q>mNQRX4 zo*6+tkLzFJql7zcRzD`%h6DWZM)@@vUSv_5YO(4vX}~l4Qb$j)LEj$%F`HghhpEy) zDt|vm1T~Ikd+1V3HYkRu@#PrDR}!RC4FTlO#4A;ss>nHBU%L_*b9w;wWx~xxkKy5j ziDKJC1pL9D@U6E+4-8&9zqrzyEOB~jDU%iKFL`D23r+(b|4f7OZt^wFHF-IyG5Kba z0t5;HEcDS`M$#aNU&pAmb3fz<7KkAoN-Q_Z+4ORoe&&Qqbkk+LvgHz6d!8=7y29Fb z$}xA>Ec--f!rSL%qwB{rX{berII~qoVx7U<1cXJb#ds@YLZCWysoSn=G-F3BK~~gzerWO1T2}vwB7qaFmOQQVwKE626+Q&7xB&$R zFt9BYt>^2Iv#;`Cs%+?6uS=&GOma0gOsL&7tPH!{tUhUnF_1NDD#%us;k6AZUe}19 zSv73Ky|&TsS;bN*J?k$VFUXEvLoni5~_1T?{R$nAj*M zX*VLgm64N{*^ZMZ6V8(-XIT*yu!^R|Xj{?6%%*TFwoV!8aHCK1@CEN}$z8Fsy#Nyj z{mv&%6KoqPn{`pu4x?LU(~|E~ScWEvkR`6q_z^X|r(*NY-o`5~qIW?+>_F-EJB&r9 zhPgWDd(bCJ0`i4>FU_eztY$AWvc#ZK%F6U^hnovCIM9(z3KRa&z)znqGBYfD<<_Sa z4eBE|jI1%QQ!9V_2Xk$+75!=Dv(@Yi2G%k!>sDwZ$U&QDJ+9mo0Yk@xz=9+(!9eyP~cS{POqqyb570u@A+?3S=IFl~4yzLaq8Q4N(&ku%g-%NN-lHVw4 zb>3DDX1V|EO*eGc?I|3=cmR?D5+Q#q&er7Q$D$v)aBF~Uu*UPuMng9(Ik)7y+$*RK zNb1qXr1mwm?5Fc0Bo9Ksmp^uw`lO;sK@%byN9B^WkK}C1=qK*|YU`WR@H^)Huyh>$ z$kl2Ds@{>GRJYE?or`IHk4A0IGN#Ob4{X#5E6+j7Kg1>)=gPWvl69rQChOqA>1Y7S zCBLq;OxDxiHaWiP3=riwdXhXJ`yweyPwkN=2m}PBRM>$8eTm83#tOz0a+L-xJJp6< z$rzw+h3nrZrQfcZ8e4NpjDiF3p;96Ib1uHMlS9|ikySs*GUiUQns%EXHpX969i6FK zw|QjGb4(I+J2kmBa8RH|w)$Q63<6Dfek!TC1VBa5W`EbRZSP{y?MzMsmdT3V1@?k{ z`UR?0O|O_!?y+STR|tYYe=yteo{;SfjRMK@J~=3!nM_%;)A2IljQ>Vvf1N7+CR<_Z zkNPFiO<}!@!RJ;N3sF%bQD4*_IQbLH81oK9s}gK#896Ii>nI3StX61=Q$G7Vc302| zp_UY``Ni~s33~aqbj+VUA5+|S$5!r;pm<+z?cEbKHA<}JEDi!CgrMid@)c{`Vh}5%Z@#RZ1|K72si;Ob1^yry z7SBHRTk?d$M=Y>nouaO{QErGE%>4kW&1M{%qJd5kR9wdTjFyh~Q=SNhv27~mq)8EK z4u(HOSdCKtZBv${Wbtpx6X+JNqhQRA<7pt!`Ga(+fUT{98<_Cf92aqUL8MR}I(hJ#Tv-?rrQz_c(S{qE;^{~z}1zChSY4kcO&-3ur z=7wXI`0q@byQ$7r-JDU=gRRqXkef|8jFS~pJLMmxE>90Ab@a^Uf1?o|WH=3H!lKUk$l!%)H zz|5yD^fH4$n!AlxnhIHA)OkXs35SP9!2M?wmIL?a~Nn^xkDf+~{I~O2n zSJ*g;7Ua9U?sOH*e(+xT)|SH>gbNMVg>4&asD;hvoB(Hr{z~^u0USA4WB+ zmiT5k8B~hFx+M&@om$m;z`wqcrH?wQfn<6nZk(*0toZHfhO3tKy2S;>PrD@McifvC zM;%k_&X^)2ja?UVk*z)((x?0+WHwIPY%XqU``1Z*K;)UMvM4b?u1?VR2M%A)F$({5 zxqcvgpb|7JLGfa(n2lY{DSUH+#lOB>Wa2-Fn_ zmD<(osh^wbslxeo>^BXxD1t^T9#yaw#{H~Qsn252$InPxXxf`Q&Kz%i`i$w}b6kgL z!zPCsZ^;Nx8!+5^wrW^srbQqL<|OOzB@Y8x^#O1Nn&dBv&s*lK6w2 z(>Q#Oz4ybD{5KI7;wxLhU*o!$*Wu}Z*eqrKMNl5xAsBT(L1?9+aW~8GVF1b-{7lIQ z@p~y}_xKDzH9086F8cgYBW2XGubx`Ub%uRH^gzy{^yoQl;E5$;hZhT+4pgCNfMwNx zR0RB*l6W5O{A}xc&_KHISZ3ifFJSCU>US<@vLb8cRZu`AxiB+#e#2xw^}XwKvBkWG zm(TlrK~bPIdk&K%ce`-AXj|K6m;f~5V4^}A!^4b2KxNY85D#`f9Er&VvE5A}0lmZn zULbV6>8uQM7tCW>)rd*fx;elvH!#YSENuqLL*lK_-hd{_0osj!hH)2-uQ)Oc^SJWpyo zKkpVwdyM_irg4?rw%8vk?z9op%;u(un)cng9}uFrh#acu+8Zeozq~xpx{1N zoWfNTEPn0_yX#D&=Kx?JIW`*2pca#s&AQ5H2*s>JJjw0VPauoF8bi{jRnmIIz{cg) zufD4$O`jz!XvFPN8>3U?DP%5PV4oCPS2_(R1tBC6v<$WYly1J@b);-t9EN#^P2^%Z zgnnk}d!*ok1v#g|&V1Tjv-{CNVU`b+zBC|htU#F;bz+EGou}UIrZD8wt+)CX?FsB_ zYbnN^xv#d1tvULtF_NM#`91{xth9tT)vnJRJ#YTTPT%~rZO$Jk@^_mT9vl2Y;cGr~ zV{AFLy+gOQq0wr6bMwmAyuSIb=h4oWq(MQEt2}h7=N{c#sZYPhU7A=9i##107GAj8 zK3x2q-nz;^GH(pUp))7EDNMw6|GT;0b__dhM#BbFS&Ojf&G^&0$Rl^QUdY+sw1O2F z(a{87ipZrwjopm;QO4!^nj~in?8bVW<_J)$*C_g9o{T85Ihu*h?SjGf^2hXnefEDQ zRSs-0Krbm%NU-Bov?4#`aWM34DZN2qHUdbV&f~O!Fx15&N}v*T@up@aZgO|ms==_H z6YgVFW4Mx_#(nbF6M&;;P0hrG+iRgmWc!&{_R|>a(Yv-dpT!1s)AYE_NrTjI8QyEtMWC)Jk9HV2)9h;$nbIeZqFX-r+{7sk61ZOXI&HfL!Rn>QjUmFlw~(Lo z0H*p`Bqi_5Qcl>XhfG`;@toL?w{DK)kH*uE2I(bc;+>6TW_(Vh9A`VVLIm!UeSXLT zue?A*R+ZOM$p8puzGo<&KkW&mZ@%A-jErP*=>VkoB|f0c)S@m{FR)liV{o61<7JDn z;(j~n)N&w>w^Kj4K>fuqtUl65y)e5EXg~3T^iY3HQYv=YBh5}9@-LXgBJ>$aaEJTJ zpuXsU`(Onz|LNHln;VuPwk3u&;XaTTXay@=O$i9WO?@%n5CAMU1AbVL~g^{M6i>7ikm z7=77A<>mZr$Q&an?$?0O6#Bw`e~;FO7kKMzC816BUN^p{9+vp#tGNi5h0UVQ-RAhB zSjoeHl(lKaiBz<2pK#v*U_RXqjXLUy3MHCRC3!Yy!x*Y=X!5IfXNhMpUyJpOwz0pK zU;Wx6JNIkX%SP3mhD2QXj|&amvtVx?ixY`w?Y4Oi-=lJddk+B0e;4@>aStyqRvV3M zNfeyL#O<8>TsrXMK#A^@5tdX`nQrbC$Lh&X~U81)4jTeII4S8$fANIktH8f4Ce%cz6OOB?2|$x5}L z&wCL)>8$O7s%$C$T#lD>?R{NUIr9R>pD_hXgULI3R9aB$tJ+78#=ou~miMPYIRa$? zH0;XCST}qXa`BU&B5j{;w89>~tQfd^={4e0hgv#Z8*WBit$T{Ll;2PKfcr9(%D?6Z z>X!iNip7Doe%s(p06l-ObLD{sPCzv+&dov7T2`BLD6HN;0M~e%`ikvXyU8g7R{!Xg zlpa^d^zpCl`cVm^u8#umNWNO(n+n&)4`~w|sFB))wE7R zP11)B!xJNFET4@s9x$Galu9)YbsHNXkMb0;w!-V&LSyDw8u7<47 ze>A(+J+G_@E%1xN6lKrP?&{wcqx6KxXu3-I*gq#WlXtD-2Yq&JJ-)ff#3wiu)K{*?dB6kXq^K`Dz8 zIds3coo=NVKn|%3{Y&%+`Q;YXLj9Fr-cB_^>dgt?0d^Xr5JtSeHp-=C zx@rxZn&Eu4FLJE}wNJ=<{QKFiUcc6&L*d2s#NgT|`i!(VP)pX153 z-`@oJ=~VlPB$|GvdwFb0EGr%Vs_C~jV$nspqFC0BQ2MraNPMrH(qIA-Y+Xh0J0*kJ zPM3O4Z+bsL37$_onE#1~;`T*LSey?$5k;X324-yM*mubLMj%?zTXE4<)A(DhYOsRh zgH?LhtDi%TYH^Gij)a#0Y7UeT$)fM6psNZvsxPI5#Q^}64XCSUca?i?3fF9Dy|O+H zz{GlgNklvPF#8;_LgZmt1M?ca7I^i$L1XA(;>l%U=gh2eF4t)q$~cfX>t+~yl!Ut0 zf6i!kUfkBEAPPZw=U%`PDTpI`yxk({cKGAREPuaQMLbY8?A2pE5U7Y0i^Y%lY^S=r0O4;v(8$PfHW8#zwWO=)%&RK?*AK=l zsw>zET{lE2Y(_RfZ1f<%<4{M*1TQTbA6_9NiKackY+~;S<+0lUC}IC?EvE#MbN_9e zt-CXe-XsB8)A3~7hEHFtfZHz*;7l9IrX-r4vg4;L_ZiSyDcIa=ozl|8y zJ3XgWJR#RC{IFzCOMQnScV*irBU)6D?E2Vtg<{SCIp$NhK-m)v=$v(70& z<=paQX9n&_y!&cxq7o?e$Dnga^r&A>q5l+->_~$1qN_ z|2DsCRD&pwfiRcMm&hU1hjm*k_rW1r{t^~Fk#X{2r*8Iy$ebSzRVn~t+mq>)x;t|? z{v~)}Ch?+kTX$fiy)b2_5Y>sJLwMQqDc4D)BC7J}d@U9cDK9A4s%KdN&(5+wMqa=s zFDyL)eMVgBjj}R{w1!82(&v=4rf#lZ6lx9>FLZ?gh;!b0=?T0dch=^w#(kOg(0NqN z91ctPQ`Sdp8%3>GTYYLA{&;G;xQEd{mJ0{&Vw(hL6P280`voiDPdCGJ>Vw*(^7kfN zZ__X}aR)V9nK|LoXD^c5f=oP)&-Yxdm_yo>%!}Cy*D*m-`3z{Kt@L5HFZ#}^NbB94 z;d`S!wt)qJtZh*j#;56e_Z5bS0 ztFxU^6Ybb~L4(Mv*&TY|>|5_N1pYiHxcbd(y=)bGER~4UYi<(-&FYyKl$_h){($eX zwowPBe#)2xz5ujO$_6NHy_{0?h2G?gYP_PqY`jMJ{z)JKY&`VRdl847FF=mV^Y>|% zB-D3$5oQh}P`@XV!Pqg z**o|ZdtEir%~mQqV`6D?(nOI|IY?1&(Qmp+JbOCfJiJbxyWxW_(~VZYApf}Wo1HPv z9dDn$Kgmvu4?VN$fslFO$I3>%L7`@6MS-j~fydCz^}0?D+CcQu0uC)$W1>U*gaD|C zOkeD!5HPYgAErgR>{)nTSGQTs&k7hzVc_h(ny1w?d=&RO+?H+{0VHM8cd;?%4g;gk zC-Zi_*6Rg#=d69bDkWTmS+_1sp@T=N(`8{r-#bQG5XpkRH9vJrY|}mDl{&Y)p5$eB zWumkVh^6r)`9R?zqB<96{eDb3#&gGTg2i6D+m+FhJ0tfOYS0tH%KPDO z2fd<;vAr{slfGVdhbkqZ#|?bZF#0i>E4zFOCA?)TjKF1jwAEJEDN@|r)ZBtPqe-Fo zp`d3x5Tl`t`DhA7qqYP$>=!obIk-S!(xbZJYP!<8*`FI6xo<*OJr8 zBbz&{#GK8IvrP%7$-3#XZ>P`D>hRs)<>oL;vi1movxOi6{%}z}@BWPX(*U@X$@Va6`oOY; z>KIHXr>yw)tZKG#_ej|v4-tbaN`d<0beWWA<&9VFxx1>POQV57=&KJ_bLkR{zWCB; zvqpqZo_s#rc7qJseCF$#odKi?elmavK~zRh3_*i+e`;cF)`B*YeEFBiNR{M+pS_{) zFva{M;@F|Op1()Lc9Z_1#Vy|TIYGt5j;1)R4_{4=xHywY%u?t-93pU?ZtZ{IV!&#H^<`Lc z(R{tw1nyl=nmpCSa=oM$19xJr%9&KgmRzkYA>=Z=*|tri)xDV&I~sX)TA$W>td&7s z*oG|a1YWO8D+N`2H4q%{9VqXo224KyrbCJw0CM6R7Vd;hd*HM$O9TQ{(F*cTPReZn z4pyjEXbc70TwQK{x`(0V$CJSj%ER-uKNjJVNq`L@RZ>fo7MVM(tkBske?jWJ;4!eK zW%xHyUG(Dk=2YZT5%1@xaK9~QH0cM(^jY06nG7&+2+oRaaNp~>5+YunF9=h(SyJB< zK`GY9HXFHUXkjYqGilEKZGKhxu}u81w1=Lr=4KcE10)wSVX36HNg?#M0C?(I$OW(M z$;1;d4ldtT=#b#MVQK%c;z`5b>`V@MmRMQQr@K7VS1Oyl-*NZ~lpm6aV1}HooaQ3~ z*MGAzRTX)G??TnZ8j(%&5F>jtUG(;%{M zrtkDy`~P6j(v^;WlfB~i_HHVLme_kXrru!o#>wEY&|nizh@on&uSB3?zWHw~n>e(;e>W^r)4QXcLf-yB zG@i1`ez)Y(SlqORQ11+jLh2#U-|!zEk#f{A_pvQ%ZNsQa-b_k$7QmOH8~j31sUH{7j%xZ{ zTSk+LhceFf(yx(6@+W$Lr_Dc{ODgvS{Eaujc+vca@dTif?!=0Ak5j09_@;1(bE{d{ zq(%H=9DvbCF`+q$tNy?$1t%Yjc`BC*SP!ym>62>H5Tn=cH*ikT{VlVD{sih>(w%}^ z>Xtnkm_#1{BN0=o`}~ADE8e>WO%u%~{ZB6Hhgw1g)o|-tLd~ees0sA>q*MUHh(7)o z4r`YUK>aGvh1t}*=%SfjJZyKdx&6hKYtT3Nx}KdYYK}nO1b-mSra+TpBun z$4?xm-1&)scUbJ`GtAk6-uLQTilA-g<}A#%L!!0}2s{&Di$1fj(}hN9Ja@hgp5_1_ zeV$Owl4zM;ak`11;5@DG%RJo&s@=B?z>9jvbroKK(P;a0bRdOy_nOt0ELJ$j(Q3B7 zd~-A}1jantm+)-Y>v&^yZLev+9@c3!UEQ7TI3$c_1rDCVe6Bsv<F;txMKkEb>F}w94JPB zKru7bru_ri(t*wSYvj-0837E|W%eAs2k8ezq6$eI`mdW@b_W2)WOJSJ_^J#gxR?rM zIZ?(9&;V;OLXFn^d6>sH3|l;$EsP#hVhiL}bIYy!9<<*r^vYDzTQcbg!dv?qr@~{N z>0Xc`V4X@SXp;`Z=0W5CAdr&vaHhC`r|9c60qX&v5~qx&WAd|VeV$kr-pCHL8j+sH zgccz`YWw4vvV84(2^o=|e-__;zsxx2-rb*kT#tSN(D;#-QHS$k?>J+$*oSr5PxrX5 zFHdUbMfO_G44Q7~m4D?ia7{BqQusa+`-g$0?K*l|HqUlv&DWzW8w}l+gIzMixBniF zYp71ULqz0xYjUm?OyRvOTcPvKw5eHY?fWGUzX_J4E z=;vxdzK_10Aw)3$PpshAG_G^@^`#L3Bp&IQPjueGl1=<$)t3YKUjs)14>z+oh(Wh6 z`@nJ@UgwIR;I|Z{NML&XYk!!|Zv2f5!TM{9Cq(=s*Bq_qA7(e+uIOst*(5YfC8WS= z_#OUiM~s96WdDQGe5!GSYD0%L?=hKm-s2pwO3^}a0c^pe-+)(qvgjTdTZ6P~)Y&0g zBO?kxoM*__#(P?@l?tT+giU3N!zJ2YPbDHcw`)$Ak=4CYtr8-LVqnV zs@w`J57Ji-H<_o54<5;XMUl4q4KNYBHgiJznFEb0MV3ljd2(?)qT};SvzHI=(z#`4 zRO&ZP4Dsfag!Q%8dcIjc1pZ8`+zkKAh-j$-9D`VJC>Dn8VD_?M#XJUFB9HxKva*}m<1ih7=y z(sn#|j-x)zMnB>X$xK$YLcBQfoA_U9-(F+uR^vgMum1MCER?)?CaRh=q>7Qc^bUY{ zfhY9krF6dAu`{n%$uAa&A;YNNM4GqPp%AIGFqf zSz@$Wlo8LqWlq2z?pT!j8G+^RNL$gbV0;vX|pG)@weKqmfEn9*RfkJ;` zWOt19^O^vkix)lV0C)}SKz#tatKrZiz*oNMky9=%imhBvQAr|H)EDhC)jMy5)84Fq zmHG^ZLiys0i~=eGGb`8j?0e`u%gRf0K*KB1l6%(EOF-GIYQ6lab|kts0_wH~YMy2Q zW!((e!SzG)|Gj_1wB!ccCYEOB@xD`(_vP_nV(#utou#k0?5qvi$pZ6f&Fi|S*Cc7Z z&J0Msr)qYxg+Q{rP9E7g6tVG^_3`yKo}aA_qgz=NK*Wbm^Kq8K8fGEW~eNAmrdJ-LNxGWY|IC761VfZ;{p_slMqI{HrHj#rp z43H7(22gtDtBjHHR&V4rx+h}{kbC!u&i-Nrz;U+$XkSs+=spCEDVNl9q{d~*vFde1 zN?l0_8+E(Ps@iElwfNgggA99DlBDHtAG*P!oQhroAWcky#d^DC8QY)9mz*J2CD3fa(uB|H$d3C%5C;Ye%54IP3f>iG!cB^*mR9oz?OK z!1`bNH{`3|j+f^KLWiK;pf<_;XrTNH9DZ2PE7y&YN7byY7HLK?BD6LZYFdgNhht#G%ttS9#hz=zNWpE~G!R`(KI^ZcTQskWRhE({=KQjd@LkY4={ z4J!H3xA74uTditVt|~DQ{+2bZoiokPf15lvTeiB#QZ%Vu+9=lxAVy7SVrhJQmqOl2 zO4jjw{R6rqN~&1>pq(?L3^xGB;^{w~Uw#TD*vo|+WH&Z6eMSe9ArLOc^JE!(98HMF zzl~s2#X0*5y>{c*GP0|HL#Wrrc#X}R8*nEWtb;X7kq?l<(8FMmwzjsm`2J|hJI@<= zY6eU(K<1xOtdfwg_=plAiWm6zaadrs=lXCy9!43b#Q>nfje}FG+{(jRYq(3FUkPnv zqpJ5)_ta^?&whS54T9jsuWj##lyQc(m+XQ3evx0b8Ayi#e1K`xejGMufM%ggH~*sh zI<2kpsneW>syP40B;t5-09496KwpjKOIg1T&d0NfkYCpin1?kmu-}?Dld1p#>!XgkD0I&=Eb2m=$2Ze}h>k<+nv8(MMQoKF_nVJY z`cg^2FRg%9G#uJnw#aRl3c#YgXLF72or};ZVBE5ejmpbSRZmD!zr%m^T! zQ8;0lllX#YliEZt1aR~|!=@sI9H>@23t0)cSJ59*%}g3U@&|0i#+74*hs`wS1Mqi% zrXKXUep8J`!I<~KN(~;QKq$!iaM>oW`QL|j4bgvw+?T-sE5i6mi`t&y_Tm3K<{dqX zD5Hs>m2h)2S#8SpJ$fRs1HWCH0CcX6zAJ>3)DNJv*9M(x0D)4W5rd5Lb09t(c{hv+ zaB^>jWCg2=9m-iu$O(`pKNNN_sp&|Q-n0t50%~;VJFtRnrbhH23UXD?Lc^`0V0G~b zJz9$CM?w$cHpdFnfY#97V!#`k{q71|1AN&jUF_VfZJRZ0ADu4yM2VliYw`iwuoe=P zo!aWr>fq;a^gDvn14i}vA$JGRht;{W&@u}|kRF!QtX!jCQpqiZ14y_Wm|<&@|y9PDIaek z`ZmwHdp}Iv6M!~^_ijt;q`Qq6lreT1uttg>lwbZ>jNNtb+=3_)2{iU=Q3?t z060c7fK{~y#QCRtFTS3D0WvXQd6{r05+Tg;cC~~AOj{K946>_$9;(Lnz`L{V?@)zJ zd+`3DHyf+jfrQnt+qq`Wv23!r8%U(3@0{RT+K%pdo0p+MUjsC+&A3-Nl)>GfYgczo zq=xq0HoN-0to#4*NV^KrUAiuB`F_jdyQ;GYt(AMsF=cI?QG92;uDO4J2&(mHQ$ zOxG0do^k_$x2?6=@k}u}w!XyQz+oJ}m)YNz6WZ4|BC<fs97 z&I2ms*H_SSt!3!#+_4z&f!Af{9k20^d$xGp;Wo6~8g|kI@00;#p*n>)mCpCdp4YUG z^b_uFdJ!N0XtZ1xmP`1$7meGv!#k2#Re)xvP|OEg9p3;%1_(;mrn-g5>kPg~c z(OTDetbob~x#D+!T9S)}1Ifm&5*@IG`O(nT1T!3+ck9v5nCMHP7l=V?JfE>dwC9bM zzUgfTBtI76xU)D}WQl!0J8Wv{Qyzn?h=1gg3zS|STi}bSCj)xO$qJ(o`+?=okTjr; z+RZY}B0dV}$(-h3=%IDhGr{Rb+lU%&^!>B*zYvgZ5xo`y)0_tqzR!g_uv)}^rlUBW zsu`95>LIM!M%aL@QtJZjb8&Go-eA;20J=#Lb*&(P9Pdnz)mlyt9q-P9`YiJXl zetuzEU?eU#Y?FA8ta{4kc#CJRhJl|bz3VjGCM8v3W;2S}UBQ)#&n&s#8W_WT^X5&x z~Xed{plX% zhY0J-&AL}1)x4xw(q`vY*~DLwC|tk4?TZ4S+- zy8=`HaY~Fzlo47)=)f9FU3FMq7)Ng*WJ;%=2)e#ukF)DSwi8i z=i0TQKirip3{KVZ>$V&i~$>=@;wSOsbq3D2~5@u14_oKThQFC#-3xvML-82>50 zU2e6$_ikMQfbr@=9j=`xrw8!j=>tfLo_HnQ=~&&akw(X`F$q;OB;%=o+DzZZ7}dmY zJp**m52Xec5+pToHBDlo38(PiMi@um$8XX?ghGLSST>9B> zFDT=VXZ>e^Ub?fY5rpBA)_l0K0zPEJTU_**<|z{1@Zyq^@N^N^Ju7CFbPI`_bL0I7gBNFqr?a-6mT|hxZuTq2KtQ)rZk_E6 zq!=fOx$@!?y%~ljyJ*t^QUVNECavQDcL9LD4>G5SIF|tCFJijdG(um+?B?oXq5Kc< zBNiL;P?aA2Mu&VD=@7leyC#=%*JpG1oXtk-ELmRF8E6XuiXjLfaTfp_Lk$otn2iBA z^T1rAAq~gPF4x~V>P7E>6G!{YLX${ zG~wtTFTRkkZ?Lduk!=5XbsU9 zNR5$PCJb;!+G)OJzU7w)qq}HM`v49Roz*%J-qjp&f!SE8e;1_L{GH${B&w4*0GreT zec-xjz0Pe@wIAa$Le!*WFhb%~M1FE8$_wiIbz+O%&XYkjowy8d9e=TO#b=8;l4Du3 zhjom=&o%7S_wC#O0TuCSj%hiCH3fM##Mp3pX~tzVS01RjlpO;31+T~PhMB)Vk;M7; zp-#H930>Rip61gT%eA#M+xl74l~_f(Z;o{dPY}_qp?BNElAU*^Mu*OJrj6ItY+6p< zBIy-0bta6X#m&|!486W(D31E73`-!(VDY`!k4fzy$LhJ&&-MHQJUL9ZEMG2f0E6BYODlt7L)7cu4~d`YiK@O8^t(L-`jIkN@|3YTI&6iM0Wq@eI#4pHaqMw+Z8+$_-xVWE`aj*hbySpX+b=w#pn?b}h|-EdC@9j6 zl+ulo7Kn5s9U>qI=q*UMv~;)NjR+_WLpKaVBR$0I-Uo1QS}3?3gq^q2M8q5czE8LO*{OP?Q1nv4RNy3B%+ z&=P>G7wF9Vv_9G192F1;F-tyT%_UD~$TRY}(g`Kr{e#-?p^R1w56a2px#T3(wxsgw z`rAFW?$T;l>r4G*Gs0uwg1siRZMS(nHly-9NB+d}Nz`=6VS8`HMMbQew~b+ZN9>-@ zDO9*6rY+8q!FXM(J6IOywdI+SGE=Vp4G=-BKmW&4Q@w9hKX_?|xh*9B~kyD0`+Ju(Gc;Ny4? z+)GzcHip1iDAc1yBVE)_-sp_cvD#3$;+4DbeSdd-X+O7Wxe(v^TrE;pyT zL6M*1J;fKHq_2K`>E~6Bp`I((&za-UrGG=FNsK!dV3X_aV`#DzQ*uMkgU!*weU*t8 zr1YcUVc+Ba1m2?8=vTTF{OEDC*7Djyc%d9(oDTIZqd*8Ocx?Hy6-( z%0D``lufTLG^rW!NCp4>L7tdtu1T9zB#VZ`v^v3gHj0;-v-^8F05;aP>2@6`&XVIn96?^uJORm17NxQ^Rm_kaigkq0&l zz6-4}xFPbqLvlss1hNY`wZe_(a9uwwut z%w%)l(m2&|li|OpmE5CRcpD$+jhsH1q|~&6s-Mi=OWJN;5ey3~iEx|KgtIQT>QNrx zbHg4U;2FMf<#M{q9pA#cq=MeIZUKiWMNbp+fy`meae3v!++1rGOZghooep^r8_Weg zUbPA!U|FUv%ve{@>eXX5e3>v<#*n@c{w}^~Z!*SI7saBP)tOjFj}zl!i&3PE{A6GB zgHqgjq_xPV`C3H(F0EDd6HLfqEf()`U>F~SPtcDz>+o{zMfiyu4g#Hbim-zdhu8fb zf|aF175fgeG)~-j`k7M|cP`y9sBmc^EQdLSzoBh&;S<5xW>JmYDu4NK2X9i8_!C63 z;o)q{%#$j4i5f?{-XTfGL-M@pOl^i?UNX!5B=<_2wFB;-#U2#uPLe!_ulF44Nj!gD zu}C3kchP&uV;}b-D{B_#pt4QKp6svc=NDf@Tml(lDs3BhEKHj(@a+t@AB)cgoa1`_ zgIoyT$m#I}lUc5ChHp?hEqVTKMpe~e9!&hsN{60W+Gj!chg&LpPni_rOC{8 z9ygR;_t9doc&!hDIVhM9(vt|Hk~IbtYMDyI_Ixr8d#`>O^_{A8Ew>oSGX+t62mTxG zjWp@5{h^i#Enz&BS3pa?o%if`o_6LOnkzyPlaBIUwyQfqm>q%NDyu88gM)D|=;pL; zmbq#(LQM@}Z5Ro*b8%;>BRhClr5as8<59bU(0g0>M})TD4n0h059)GqNiz#8!o8F5 zvf&>n%s0~4C9^|TDphu>J}#8ws*)euYw)crlw0~?PY7)6Q!$0QEJIcD+meURb<13c z?5sLl%V7}Cb;-jdfUW0IGfc_~jHe<>vqZJ9ZrGnxo`PCg_*26XitewvcO8gP?{!;l zc8mo+fAcpTwRJ~?xb_Y^KUO={yc|M?xSvGKW*c_tt?EWmdOX>@UTQgFit%S~$%yms z+``dLb8!|XamI1nR{asUi6ZACqyUrLGk3V`U|q&}s^YEKV}#gVi!Z>zEQNib(9qYn zc>PrsX2LUij=q&@I~5J#1ASd2w)JcFMM9M2on!g=xwmjaSr{twluXOSJqo5T7N0Sh zbtU!hsOTldcCr*_$l^`#I=LHPY5^KB?8NG#e*?Sb@@W4Zp#khB*ihAaS zMz9A}K~Ka3DxP(_y2Aze&^@ltXr}U*HXSSbX|Ja32{8H}hYjF4~yxhiT^sz7KO3X2pd7c(E z@H|dQ_aUFDKt|^;`@$lJ!zWGK-6o7K49iw3Y|AS;tHbsdR2t}ty@Zd4pG@$T%4m_u zq>MU|L767BgPqv#O4uLz1JAOL2m0(6c5&tiS0es)r&-zKXcQlJwg9#zObNnJ=i#q<(B7pW_U! zPG5gXX_@FWY1OO$bXxD~yOo)$CifuK6k z71aqYY}dq`Kt(Qn{h7DW458!87P>MXYzTiNI3MEo4NO%lp;Z?uMRodc;|>}k;&HDH zn6+v1pw1*r&7W$|ERYzMGCw~iUZD0~K(o(xZlwv@xy3D2QYVX)XyL&Gv49+-oKVvMp_V!I zuew9*W?w}N9-Y9|@v+f<_57ltp7o9(?sX{%iOCiW;_`sPDRENb#fWl7Au zoYzQ=MP4E$=)OkTnav?h@7oq=`0V#*S&@;Ik4+du#SY8M99pr}!{zuCnPAfk5F$E>cy_OhFdYH{FZQ6q*Z+;_!Ryxh&57lF5tSHXYkA&=hq53K7C z#s)0re+7KW8Z>OLiVFvJedU5z;suXtujKT)V-7LA&hn|LfFp{gu;?`61}5!Z%iR+? z8Sp8zj!@vV(yA=6Nyr~`kVy<+uC=KuI&u!f1X2VDBToTle^Bf|i2^(EmJ*A0~iI~*r{ThT8dec%t) z6j*XBJme{f(kh;CAI=D%RR2CB|K?i%Nc14H;0W})W-?ms%$s8id=vD?h&U6PfH*8* zzcot;A0w)?4idlXNlci6CI&OaiEMFcd8urS_ee0A=Ndcz*u+rP-i>N~LrkWrt75NR z(Oxtu|(HgfH&nCx@Z-xVqfnWwhRqB`7h6pvnH*`{?1E>yqazk>&>IXYk80 z4!$wBq5%LqqeNUw5t$$e6PGI&IP7MlUPJ>}G;m-t2(qe6qHg6D)&8o-4?#kCXM5N# z^d%4+Q&XRAK6<)MwZ%`!NU3sB%%d6{r=1np;jvd!q*bEYlzLd;Wt_JLUM$(@l$g$p zJTpf%#(4r(vUm_D=un6;)YqqXp@+xj$ph3y1X7iIp{z5@LsOByqILi{!O_9>_a*J9*I~5#!I6lSTx_D=O z&@3+s8n78+!J!fNKnH3gSsP%D)=R0Vl*0?;r2~$$_2+NzetH z1(47{uZ1>puuqoF^%ta6ER_zKPZ^HT(D4-Z_Q!*a3Wg0xO>>_}KNAG={SVQtBix8L zQp~gVm!pdKG{U3x@per0bdb3Y1B7CRnfe}FWGA-NL`Y^U-7eomY#da+Zdh*pr0ALH zqjvcy=H|+DKt1;n&{mwH!#cV7I{F8hnwP{Q5(GlYNl#>YLo-0Eh}2L+`dDck2jU-eF}W{g!Wc7*LLaeT)0H^jE!u}+5vQ$GG>WXKmI_yVS_tofc&is9MgXOi3AeJddGBUPn7b z0{~k_1la6(WB?6galkwTaqTbh)srV?e?~o{J%kA_!2G8nz@oos<_EP#ZuSJ!ZT9Kb zl#n9w)GFm*Q;IV#b2c<Hr$wI4ggVc$$+q-| zhhrRl?OFwHmR|S9|CtydlQeB1JbPSfH7I_nR&W0BB%|By3HgMCy(<{J4;HhPvbQO% z@~!@|JHq~Kh~D7uhQI@pwR4# z42DAnB4_YB>i$Ke+ru_Z^p0TOAeS1s&r!YF(6O$PKLLaMT~M`HxW#k)um*_ox;?$k zMkL(x1CEz%yiZtojsTP5{)X^!*J-7IEfJvc)}giK6}55_)z-6-9@JO+o%2vk_r%fM z6k&*SO~!YJ3eIQzsC&ku1f?&;eBRd2(<8j`*OIK^+(x6O;rs?6_hRd(y(`oXbvwr| z((;*RiSU?AO12`Z6G;;^_f!FyhxHs`NZq$#WUS-z@0q{N5Z$oRx~|@~;7! zXH<*Ex^#miuFHON;V&HT5l<;k3i%^#o3GiKPTXjTR5J=Ei2YLOBFH@)Q=T+~lmm&^ zHlAg9xGUdCp$qpOmx`hlew81`2}#v$MH9H=M`$L3$)bK-^Eq#%Bz;p~qSZX}RU~i- z-`kKsB&Ko)o*M_^4`q4Anfm#U9?17ztcDsB#x@pxHlIvxoxmpqs7Y>{Nhx(`s1N~% z6)78i{afO3A7ZQOHNUtQ_l^Z4a;%};pi{_K75mCiQS#m70m;b+ZbAR=KhXE-&tF%_ z*Ox;6nwaYG`(r}?lbqf@Bwuau)izBwFMk|hYQf%bS=G~C=^h<;D*}?223;0AiXk}0 zYccp7BTHxF8+S7EaoTt4EoIOvSkU=}ef=XmOE{&ZfkKDMcQ)S!Ayobp$zFP65Os{r z)OU(zqchdh zUCdOl2p;CwqG~j#@eEhFryE_d+W#zcVE3!{@g+L=IMg-vdnD05^PnqF=a|MFb66g( z7^Tq=gs0sKjaquyr_Is{^{Hh9|=wQei+*?om2Ejc_)A3tApAWAQ{feJ<`2 z-Hut0V@TaQK-jgo1gSuvB1q)kA)%C2L>SIe4pQPY7T4Qs8C)(KCctCq?hONZF_U>z z4pJSsbT{a0&=E3iEJT`0%mqD!ilW-}9jRP^sd~n3BMl09{q6B*^u5I@Xo?a1_-nF3 zulpBMJLV(Uf;nR?$WL_sGdf{{PidGUgHvUD#v$+AinzexSp8V+Ps)WC*%!t5#A{de z&z_3NYbT02e@Er(6VQZ|dWQRu;I7PHKX_UUI#pLsjmHgucH$rMWzzcNtZQJL{osA1 zu7uS0s3SF>_%)V3L#J*C#D4NY;6oHVQ6B`nnDlBzX`dsr%w*3hn)(gP@?pLU znNHrG{;w1eI1j<;EGcTulxasOx14K9O(tDa2Rz}!1yqsm0r}*$%-)ODXqUj`PPPeh zT435Z2j;YwFAj)8;+_hUc#O)h2GD~`ztaAwV6*s|9h6jg%JdP>d%eVZJ@7stZ?L9RvNby226N-}Is04dws(dx(FC*gT z&*nOk91|c-X`_!D0o*P2k#Rz|Tm0DZ-tQe##ftAue2#&@w7TO&Bj z%2vv!&I*yWY*hTa^ZT(Q`N8`}H>xI);*#q00ZL_+)_i&2%BDZ8$weV{eWLh74BK}rI*Nk&&UjB__Pf`y2S{VnI{1 zQk-BE!h8-D){ob}UIM0eB6z4RhOa$WUdH>qbkK!?#ZyaYPvqW1>|?H;peL`9g<(%k z3fO-yuKZVPPZ2pAEsM(gr@#WIIjxK$TJc#JxzsgWNUtrnw8a3Y)vix zX!8G&x2V0$Tpo>UX9?5-cxz`k@vnad3mGMNG#b`UlFo!cgUAS?n4QjlbP_NiFsxP_ zg%C_bhr#6GmT)|}6@&5lBUJg{&>{R8a{g;QHFA|Q9)e&I;)I}|JqY<4uTUGL+AJ}i#fwSTRo}U*Q?SfV$TMO#Bk}~O`9KDLdwL_+n8!%nY zeA7j_aj39v7^E{@R}o+dcq`YiU9q1Wryl)#w*rJOSBCw$=5^95@u1rT<-@x>@}F+r zftmZZdYZ!8vV`bA=UW4`)u2L2U_Ub`ka{g4fuzzwBLI0WWX=gW%s)?4iZd1a8#ywS z786Vuna_EuS=>Ti*}B6*-@uW;Eld#p5!3>21b+WuUUK`VkwOf;j!Hy{y7OQgYisD_y_dh$Qa_IQv3)q%xHwwM5-CO~?pL?GBDP>f!xhE=cqhzAqK^-v~GP01>qZGsYODoNRg;HG)*%cM? zyY9ps@AfCie!aJA=RvdyWxSmMK^%be@4`6vt7pjaE_;G|AKj~Q?^)leP(Gv@t5No( zEv-GQ9DtT^WW|@(s(9gJ=b=pKrDq=7Rbx?Bo_j;Hf1g*neUi@FYZr=##oA(nh2{9W zO|DXhET;<12e|B{4|Wr|Sppiz_%w)X+h?fZG=#xvfb~t%7if=$AEf#Jw9|2cR+Kt} z(k1VF?EO#@rJLn=kM5jG(31>0Q|moGPL#q?+rOW9=m>XHjcYUuNitN_3RMS=R*Kw& zf(*_Zs&?qbfzzI{bcAON+&4Av?!?+v^XwL_dY}CH^HI?&vT&#ik-NH;6K+n{qq~|? zpl0=n`YOpxAQk%}u*5b9d=vNLT%^jBzW3V#&_5QDG!UHcDpO7Jw|I9&3%6{T|CP>hE-7h5#3z}6+M{LkeTL;Vxa2lR zV8t;O{v;YQtQwO<$<^=E`3#axU{%)Jyw;U^B`SSme&!ZW`qNms&28$L{cntik4YTn zZ7P-R05+eicv(|vA7ELt;`Di9l_Gd!i+w@IpKr7^#q703OLfd z6N``B9sK-ZWE`;6B*^V`B()L4&9>i?J;||)L?paj@d53zC zj2`H_i=}t!-wU@%Zkp?NspV6Oe7x<{7_L3IWS>x3meG;Xh9=~M`&1a@c+E$tlUX4@cFtnr{X0y#q5=(HKJRPH$WVpif?uB63>2cHR zJL&o3_bw9B+3!ba<#3gaImTeStIC!}6Pey}3H#{NDPafh7MuC|acZLK8P9assWM&9 zqP<^fFN%?R&IK4(Y(A<@`z-(0s@}hiv>`18!SVjNXGo{>Cxm)C;zcc@YnIwi@J|LX zSL^;l`W0i*@&{2D2h?W0e6AA5n}`wL%oy?)6Hi*#ia6C9VQpVWaLT><=W)zQ(=BePw4`+jCFN?Tr+&|TD;)XUSOh;j&M_0x>C$G-( ztjLCnRpw=(d-|-OfCbBW8{tEq#hH}oTPP4Fy>er`fm1o%a0Y-MRQam2akwkRx3><+uFQe=$HOnopwgjG1^OhsLhAgoZ0er?Q zA!S*BpBMMnIcxLRH8eh}e|uxBlZWinb!4BW2lVY`r9upnD0tIG9b-s|PE@gGEco=9 zmA7Z0-q5UY*6kNUB4`&#QVNn~2s?2$Q%yrd;0Y?W6XUyzabL#_trOD{GG~?k4PUu9 z>@cuO(wcS=P<+z<4EFo|hh6khP=1H`WgTo*4?P0;CfBWDLKR13PMNDY0|m#0 z1I_f4F#0vq15u4tH3yF2T1qDVq*`x}i8s`K#!~z(?^JABsTbO>EykAM4fiLhXTh<0 zXQc*TtB;9)0sh){v$j|PM_t0p20A4zRq5V7XF+nW^lFs&s>$cGp4PtOQh??IW`vAB zNUGeQ3QX0xjpp`OBHFYOkE$}DjK)uZCKvWSP!!hj5jc!Y0@ZZJO%AgwY(mFo0K{cG zy+p^P*GY6kg%0rjpV06knFcjHsP~?LDkTs*TMpIM6kKo22;$_`es^hVBuEaj62+4` z+cN3i^{EnR<>(ltgSt}a80tMc;~`d=SNGl~t45LcSCAm7m|!$6aNTI}N`8y$kaKY0 zQI$p!O3t28c?3+5%Kc(HHj*e5Ezcz0b;iV6Xz!q^4{!rzWdohAwT_gh!~y!AFP@ZWFIz*|93S;<{cv=3k- zKz$#xel1{z2;@;-lQKv1FN;KG^4qhpYcq2PVuzi>JTF27R(wbn^ZhI<-F{wN@1F{d zQ5HMs87g!GRiB(R`sqwjd$g!K?v)u$xmhVx3DrE!6=reETa8NX-}GOP+sow{7|&aj zMTtSlG}@=wRRi9{#g4X2^Ak7ZR0KPv{4=h2*%m&Xk+U1!#l-B*w)0u&l<^`}%Db9* zw!72AcE4O#YF=2zaW&r5Fa0@GX|4D5bAR>k`E1A0y?1vPh37yTxNRvtq=%nbajM&k zE7Ne(v0d7%?c8e;i?iNe+j8f9-hUe$CX{F4jLN_n{Y-mw{4CU_ETe#Z&LRRBWs%*_ z6;-toE{&0QjQ@to^7Q6Hf?nU>~O2>7hm13A{4jh zmHF$dw-5JNX~*kuy?H;MY@(|lmP`+NuDFgBtuW%bm$rs}eS`(L@?yAHDEDEkVO*jE zeDqF~)X~M3i>thmk(5C+C1N)lDOtBhx)Lunl9l}nXoAICn!G$&y{S`suZf(^&TYzr?(qR-AzP{8uz(>9*|t< zuRPOCaIca<&tv8g<IXzN(o{TznU3Fn zFdlXsp3AdaZJnk2s5+(9^me{$V*4A8(_y}H!fGcdzN%0YarorCR0_Sc6z7_KoU9m= z>*ddoy%6z^A${Dxs@A1?Yf9#6rv;(5{g|pO!^`$NLuLuLYRI~h(D(Dg4er)BM%y1~ zYS=9-&kV;`Dgr zS!?RM*FJLLMoyh2G0W$3)wRUj^F=C}k` zi-Wub0^#PJg!I|UPc90pn;k6Nm3>aX;z!DBs`><*UUsLu{>K5SG;V@Ml`kgQ*P_aK z-FU?S#X_&TN#2!YINLj}nRxj=snEew_iaFsqd_!|J$&rsSC5H(hN;_KvRV>8PpnYh zv}fcR(wrgh@9{?Y!b9ugV46-c;f|ivduTmxRIu@fcV=NH+x5{)rJ3qAsn%@fG}X1o ziO%HP^tU+Nc*%RIEv)|aPu zf*Kp->A69Q`QzES6EhL|#>ZLhS5PW}CKynzK@BW#nt`RBTlXW|?&EesJa2|6uUR)N zu9)5|zbqgIZH`d8RQmhNO1icos+12qws?4WMx%QuxmBn7j7&0!-q7?mo4S?TiKV_! zwo*)f?!b0+@i5PQ$qdXhj2=5lPT4a(y)sekhYxt@dag;MRqEep9Aj|!Hh2EqL)+w0 z`Ew4zVGc;8XqAc8ktGbOU5{Jo-vvAn0p*XtER`NTaVB+7TzA6aKKn_!$CxLAuE$>A z+oxf?&WPDrv}+%5zRqkuN5k$qH6a~jPA+wcf~J`{%D%0pAM2du_u5m9^rR7|aN#Q( z6W*W%c0FhD_&o2bzN}jl#7}VP>NVmTyb^bj)kvXY6Mq1TI;nw-Ryl3oE`Jj+ z2!kB-nOxL9aBcJjW{2I&meSs5J&A8U<5!N1yy|5y)f+atvNU<0U5QKVmt}OH9%k%_Sc$5`A z%y4U*%o)}b|Nd4Zp&kyDM&aY3dmjgRJc{jz0iIP;d$SLme^h6R?2yr~5;rib56&=+ zP{(d0f$HGj=9)4i)A=UaQZf9A#Qm4|e=3k1R^SMmX|{)|MRD{(_WA4gS~mr<1!&*tW&Yt4>!55u#~OznjL=tSv&3G?^KuvK-IkxQ{|7ZoF3m zobp$>&D2jww|No4gSO3oTAs4!(+yoYO%o#nk7-z3`V_F?BqIr)1S*GZ{0H_!bA6|X zu^0S$@_x9yn`e=79%kQUn0C()PVwojv&gbsP%n7JZc4adm}i-Yq^vBQ`7y?s;-`)Mu9>yMlz;GkLSBM>(= zy_tR1OGf3Qh_S-6%MDCMTICcsru~^^2L*}Q@6m*+TUGrser9s-W z0_+46>;w(M^jVEm_O=~k(r-CcUQ>H)$HQGRAOGk@RBO|_0#Y6xRPSwJsDp%}y%&fs z&->@8vRIa{$=D$kw8iE{i)@(^8no)8$59=3rUw^t&4d%|XXfd`>k|#pc57~3o3C-} znp*z&;UzPht%0QaPi`j7CGCXzR@9$6&C<|qH6Anvl6em}=G>E~hWR{eN6O+~Ol{^G z9`3e#E?b;N1sq?B{lpA=sz&{RKSLZbHd*fZRuJ5kL;uD%%+Enm`**5u&U;H`kOHh{ zL+e@0y;weYl0uV7h*(8MkBJl%#u~_FFE{jOaB%Q-g4QBJ{XfB=QwKn&$(VF0PeazbQdF5BS zGurZb?i_PGT>^7F-Dc?zBxp%xU7(nE(JhG;3K?B+X|KX$KRcL{75iOEfcpD#u471s z$}F(TL;a&2OO2bJuYK$d@08gc0lg_Fu!`i@89B2@!yt8x8_4Rj2S!R(GMiP3j^;H3 z2pURKeRV;0CM4Xnib+3o{F(hvu)oBBJ2eJk8KS{V7K6l1kS!v%vSjI?bGUGUc7@o&-x7F~cAuPHOBY9S&XA!rauwLpWo~>ov zv0Y~+bFk_^3yBZEb`RBWv0ML!BJEW>Lxq|?zU*CI2#PrA{Sw`2GgQz~M=#KJUn6Yt61K)V(UU-%Sw3v1}ZA?srA}grv2|y}ar!k7KQl z_*;(-e=~=^YKM7YKPV+G<=tt*>l@Fq2QQjhDIlH_M#{)?__A|cmjY|zpywe>Npi5P zJ8+~Wp93JcjJJZ?;8nfHTyk1|F|!!GI^#pC4AmqWG52yxO^vO;!4Xqa0xA@Z?aL1F zF7vFe%aC?dk@A}LplOpQ#(p>zAk^? z;sF0F!3`sH%ivJ{0B>dZ<>8Z2Si5TdM*`kllumSb|JPs%STzo3LvWn840G^4Ju_nX zyDyqrpYImc?b5D99Y?)4v?;ceYU+9a`AJF;%fm#k23WHhQhmqTl5}O?vOlj>uN)C< zNH)T)Vr#+1IQT<1i!Vn=i>==7bea(MwTQ?5&Uebsw*{@F$*8b)#R;d+U+@2sl5(ga zr%S{2JRmbSSBby$P_j$VlbS-B6;cp?8t)xzc`M3pzraH8S!0#(5`3@5XlG26vg@Yl z+HVYC&bv+EP~XpUYaBk@8;jxHY#pQxX2Q^k9_bA(1OA|F46jH*!0L|(;g^baQQ1R<>+PjXcsMjeL% z6+!jFg*%{4xa%RnPs(c!cQyFnA>Qws%wQt=QRS8$>A zn5|+07<0vEKxLY!LOjs+;jvId-o+t-+yjbU-t*K_t5)him*^@Tm@pfRGS+H!H)`s; zcfZc631U^)qA%EQbjl8SXoTQ2S!w1)d*sa2Tt2RY4sd|Xc=dRuJ)3(+`1RB)T!Ysi zP8|^yYwT%`AqzT?9%tQmbWq-fpOiL~s4QM931Y7bX+KvFOP*PI(F=Nky(CA*+Q(uy z#oRlnY&42hJC#QI{P`aSfJ6y}>aszcBW1};fGt+_evTXx0Fg@Hk=3qQlnF`kW4uy_ zxH#KDwP%K5JA)*ASOryi0+r`Dh7*ssZDDoPrzDgM^Gn5}G<>@?H$bq7@=~C4nM-nH zxZq`dq_;m3S;)f(cS1L8bFS} zo0b6-2GEnFW)3w%q3b%85S1As1*IpD{Wdl? zy!m7aoWM)UkR!{;Z*x$f@kUGH9TTBQSn21z*CY3ygHL$tn;Y<`gA@|>dh~|9&LuTs z_YjIMadm@U|07gQW#4G_8t?0~_FV5Yp)s|a8}4212%t4hdB-3kXdl`6)B*!wjU1!f zsy98Owv|$+UTbcGT7mkw1nUEwHQU9B_@FC(S+^IyFOb?EnGGGbY{x4pOb zQUHk+ymf||CQGBKcg=x;UF*yJ`v{DO_;o{aAA~6;4v|oE6t|{|FA<|>J1VkAl6YHS z%2*7|e-c;{r(R+>m5}(H%4OdWwc3rBIMVf^TJ#ZhaT7{~9Klxp|07cWLvGw?LVqoU ztrRs$x#>yn^}EsO4C;-zQ2K9Z@NL{oj?fuez+L>|5k_ABAGxdlvC;oox(omIhqH$t z{?{xhvSpOl!z1rH{Pf=}h7LZbp{Y4DBKLPIlJJ+<9}Ee9i^CP&x~7G&uKbEtGDLSd zmqmlyVDu&|Go^bk6JjT>>~tVkg@q5f8WwTL(vW_agdD$=`Km!Ikko1t@=&RSK&TEm zPSRX(&&BZr|lIR-0oB22a8#QIsD+JLO!)< zj7N^~nd?K0Pj`=%Lh298<$q3jin&L?%*p=W_WnK-Qn!A&{&--WoO%PfMUf~)j!6Q_zyByi{`>Bq|29?r{|y_GP61{- z4K9MM4iZ3PWOo#iEb8u3jwNOgGXRXgZ;^m-3nMwhchch= zqai36gf5DRj3xEDDW!pRJ)toyGrk<^@8BSIB9Z8zh;xk^B$P+2#U9?ByH%XfS4O;<@Xg zIBfP|k$<@YD}oK1G&_Y7PunQVO+|bZEX2rqYky)#yOVwrfIpc1DFjx1s8e7cx>uLW z2#G|a(I&_cSGTVWl`>>XKr9s(D7x1_y+5rlWJHA|O(?_(@=WFOHZI=~+kNZKsT9rK zjHow{7Ex58JQ8ThB~%J$#CXwEo9?9j)y6wt+F4=u!s_&H;#c1z{jLGW#-ZBMX$NL_ z+@p@bC1Q}lm8{zr(#C>yXYl4po>O+xJnT@o&fZF$C1yYbO}gEk;4!`6~2FA)38 z6+~;4cN?mxE@)(nb2|+YlCXmZU~DAMJqwi)Lh*Ue!Ep6XXm zb+AV3VA5uoVAZN+p5I~#&sW5ZNMDW}PMN?Y*a$ofDv`w;>BzgAEv>_55`Dre?3XJr zgAIPU60w1mf3b+$j=66G;GKCn)38f6n8qlgM42Z`(^{Q$1F^(k3DgmTs;Mftd^5S? z6AuD>I>u#qXMTONCHAD@glkjXjhZbkXUVpmciss`)fHN=5wl0!07e{a~Kf8vLSONiqW;>YeZ(LGxsaji+|3j!N5WpycH_!E$_L9-0( z{i?Yxm^Q_L5%bH<9O8f>hH|>^q#?7Dd$F>Zles>cX`4ZLbCd|hME}IIj0@J|gDVJ9 zauUZ}!>|#~VVu!J{hJ%?c^=2M>_*6cOYq#Br+v$7VX#RNcIm@SDg(*b7BC*Zz<7XC zs(_Gbggj%V7M3?Nbit-NE=M$5CpFJounZ;E7MKRP5@93kMpiI|Vqxy&!klqsebmVz zK83U!(I?xL+pQpwZd47Y*B6+px;kjO3!#bab)ut#Nb- zPM)hBH~Z|)a#wb5!;=F%M!7#GbJ<2&T^K=wb_P*O1~y2Ni9A0}kl^0Er={x47+-$C-2e?&X{Lmv&DbA^&$~ zk4%N1(t5H-{d|Zx<|sa++F^i$0T)KsEl8zamy>B{PUx|a3SKxo*aIBRsB5H2wTw|< z>WIYpdYo*-GF4fE+aY09nK%}Us~DFA&QwQ>7~71^uCFayEk+ks3i+RaKs`?gUk&Fuj3`@q*xU3zy358_%w(qNFbb^!S{z z(ZW5|RjJKOKF5d;Y;z2;2MdhO(-cD6d?CHon#Z*+UCMp`gjdp(%{uwJ(?fEZk6U@) zJu+Y654M+tW4t6Nwego@zRFPQ+x2EcKBLDE!~ikIo;g?2IwSDLW`#A*uzyeT<7l4i z)?&fXF>oIPx$gB>VlKsUgmqm8fp1Ios&XdRaW&OvYI`uplms6RSyuf5N*7`*2+mDb z+Ml3JHvP)REJ>(0G^-V3xo}l8TGZJNeCfTGpUax%Y6ycR& z7ofo)YQMNu!Za*fBHNIJo=<2Jm>UfWGV@XsUq+5y;n64>rCVzJbdyPni>_MBfk*l9!!Bt)&05Zmkord zBN0j2k%V3r>O7T=o=8m@pSnBj^Kwam;CI_~_RBjlW?f3W>Ce>rsbiPrc9KGXf=9R` z4|kV^0^98rB9DR-##|FJDdlkUFY1LoNH*IB@Crs&qjenHH|Yi$)em|^_qZkusxmBZ zR1mYUe3ae19e9iXI7<8?a^?CfzPmkSmLuioDeSS{_ZfLl?u7 zW9!Ju5;a5>){Ru!{o50}m;^@L09)hN3#ddl%_VKB^b0B_mH2u-*|a4W25(Jd-^lPP)JKmH?LxWtO5zr+%Tq0@xiLlC=z5llOCiR<&7 z)&|eomUE{J_^3wg32#Vy7HN*%8@cZ@d^=mv zj%ugHJS$j1+)8gb&#Hg0?iN^t7fP+hmMy3hhZG`jKk;-=DP3FJBrQDH4Bk3`Y|@cB zGc<=)>vZu5^v_#pzUSxLr1!c6<6f{08eq=J?B!WKz8;(I8vg|IP+L19Kj+iIM)hm7 z;?Tmy`U9+J4q3{=XV!Op3k6tfX!yHoJeG0gn3Sdpvy_?qb+r3nnb0Q#D>0|wVooSQ z%)|t#Z>6tCS6ZaNl=8f(LlE!0;NS}4uBAK0bYY7-Sh^+=Pem8SP0^EZSPh4kMuCVh z*!IVPn!dI+I&uP<7?n0#3hEUtFopT1iR{M%X(g%Ldyjqo2fh#>z5oCK literal 0 HcmV?d00001 diff --git a/articles/unique-network-flows-arch-options.png b/articles/unique-network-flows-arch-options.png new file mode 100755 index 0000000000000000000000000000000000000000..fa0b95dbe9a848fbd2d574fc6b38200f4444f57f GIT binary patch literal 24111 zcmb@ucT`jD*DVT)0xG>rFH!{rrI*k_g&?BzqM-ENYY^!jq<0ad2#5%T4$?bFN03hF zy$FE-=Lx>=_xrwc#~JsGJFb63i90)cKYKlE%{Av-ggk>P;@!G?3j+fK@3E5Ha}11Y zoER9GD4c8H9lL3gr{FIRCwVO=V_Q3StG6aj7>aLg-a5Q?di#dK$eqE=$;nQXm)Fkf zwT+XrwH1%Ctu@gD5gKrY8uJ%gPXE4-feB9Ime!}PqPp>buuiwzXTsPMWAYgbDXxE9 z!t;^f;PM~y`e|=sb%b4Z6|*ksYO@0NtLj=}tF*pLl!-kwFf^BqFU1S`{EC-+KBY%9 znCy8=4o)!4TE&pvYiv|aTl{(bk;8t-Z|Wh;F;mCb6gL(Sfx6N&oKAg6jMLXYT?iK zYbZT^!N6r^z89|G%I|`A{r#ukxT-!sZ=TGXT)*c~R3PpBsz|}tf*OxN$qvzT28mQ>JJvYBR4QEIeZ%tP)HaJ){-p*1$+8ha7?#4?qGmz%+O)v(&R+Q zt-)c#rRZ^Y!GoSR_?D)vjfcC!H2!{z^X@xDPsbYrhB5tH%pL_RFbA{!x+eZn)nnKR zUq|O9LtL*<{O?okwOHMnTszx<*LVtZuJ!UmK~~?z{6n-J-jnxhj-Cq2x!x418?P9pd1Mgnai z28Qn0V>#&;Zj)Q}1X_A8Ggi_o3#nuXNp#*&r3l3~p?*;gwl94)PYvPeQ)W>z#U)9k z(ClkjevNp1FxE=>E7DxWlrJkNH@P$Qv%9=}cno38=hBp{vZf?CIkCXCu3YmGd*yJV z7!}n(XQp)M0Ss|SV>=7UgOa_JCp$V97Z=almN@?H*uRr8dyLwac%;fJs`tgsm5>yjr z*FAf_{o)ZS;;=#u*4({N4n8T6?@jw5j$Tx(n}CsKB#yqD01-sa=)W(E%qq)95jIP! zT>axG6Sz=zNZoUDA{pU)4v3RW9K8$s?B#9@xfpO@!z zbT(VEJNY(LefSsB!v0-`s#0)C{se|@aJ{y+Q2pWa^tEUTLlh&Nxzwl)kKSvy(m~AO zQ*Ewe%`bm+`OEZ@>wAcC*2|M@l>g?F_}F*WQ`@t3NSkVkCkMQj{I4Eua97?Z zQ{0*;UF(3%FuK0fNcgdkB-_b7QR{u?^GO~jnFBd^xbm6}vcN0&C(6o|N;mlNw)gS$ z4d=dr%jE%|i@FK?dE`6+{iFCly$@%74hJ3F8L&7+kX&4vJROS4^(wFje0w^VP!&@l z+OS3bh^VMo&&$)2%kF%>Bqs4c6F%gM^?!7FYMfm=^>Lgc>3t?Cw_ow@gQwG5g-SxX zV|7M#k%&}05EEO>j>5m~JYBUnWhA!TIVOdjmFc7^OnC5?=4d_$($azGxpW%HxWtaN zEUDP{un?_X>m;}cSGer(IsEN1#qBVjA~1V0S|W;2D<~Kbdq<}6?naCj+$!e5sx9o` zATDmhj7X+g4is0i1bQLPpK-qG&j4S4uhlv^a(Ui(_K=@^<37#FS);AugQeMmwN2DFVWw0R z%JyP%T z9W-7Zdu+^zZEFQ!u#1Vcc#z^Fqb}{O)dm|Meii=F0hV`BM-J>g%;?OdB;L!zyvq_d zaf~QqtJfg7A;=}N;0_S-iZk0myghyAjUT0qFt7cg6B+VH9$lUqDq0V`d|5#h8qgeb zk?2G9!*?Ioza%fW>|kmk*q6{Ll(3`fxbeJli)TC$c9A}rvSX22|e{_mj{5*pi zOgF3jzez#e1l(8IN!C;m$>{Hwf=*|se{$GW>JrXxQHH^evgsG%26uxJp8t<@I2@m8 zc9xAk-tHOp8+6ufMCRG2nJ8S$Oq`r2iwfw_R%lvaaMMZMRw$Fl!Tt4djK1tt)&a`m zLWPwut|BLUo4A?pDUz|A0i8)#+MXs8${@22n67?4xRFPHcIhS{_Xzw*Q|35P>`y2o z1JW)b`jY>B$wGNy((5UTULB+7reY>7(|IRcyPF{E1}-DW*SrSZ{)F8G=@eH#XY$VZ zZ72niv-lfp+=Cn)9aWOZ;4oqu?ab8c?E4itf)W#rVc$XPdcpt^5fORmPiPcm$L{a; zw(0#1$sU5Q-@?E<*m_u*VT82)AJHW#j-GhvGxpyk-}|F^<6}?Mk#y|rc)9qPrO2>| z8hg*9{mrt0+}zyu)>e=)tT7>p6~`F;6;5kIe%J&a#i}PK0#i1Fje@zerFqbFnY_Na z_{CwVk0`?Kfi%J>QRBdwnVGb-wB5warKLAgqaq^%rC`yNR8-ovF2y|vkMpDP z(W5Ms(x1&=i!QYsptcZy-Y3!cPDEtWR$yoLa0rxDRk_2|NEFtYkygEjv#P3UlAbTP zYUG}~|Erpa$H#p_v*Cvm6B8;z(=01NZ0!N`$tAz$YBgfo)rlY7`@-zv`RZnHqB5=WF9gd$k12e)<@WQc~z5o;ZOK=6&0~>wazHgxi6J=(8)60yN3v%FtDo;JhbK>QWf9B>OM}n z?2KX7*4F0Z<6CtZ4%iz}SLN3Fh7r)IBSO_VvKu9Nii7>-_IMH88aFgddrUe{_E9Ry zozv4(c8Hz_;r0NG&SRA^XJca{;Mi-o2|EJW)6+vuO&vkvm{*E-%$Bx8ZCmHtT3k8v zh3avB`VEVt^}>?7`el3s76DGOve(lBB2iY8<&R%H?;LF*fD{>K?!b;~mBvR$n|c^9 zGb1C|G1ur(bJ$9=?knj^Ez=D8y1JIS4WDGA1IvaB2+}GD#iuyCtWMn4M>t{v<9#j; zGN(Ax=;7q%C+deR#GgNZ4n^Ij;E`LXKb|3Rd1OFIAin#qO2?*Z{$_=*GG36)G^n8K z%>sqiy6+c^5}bdo5%w)(rBN7q5tZR7Pp_st)|&NdsiXG0Jl!biz_Q+3Xge|MjWW+o zN?NqgsAcIbifJz&9H-W{Fx{IEq7QwvoHr((l*uH)4{q#ZG81Gy1WPXp<~>&xu_EJH zs&;Z;*-eeXA}Jw(RrD5#g%t#bs#4u&p30Y$gMP9ud{dsTe7lUvH>r1x^c0>j`m-yl zUWtLwUm=vUt3H@W1<#c~x?uh%ov@She99=CTiTv4m5I_h$V`y!o0K!*T^Co^;ld9% zZhsEycf3_MD`tL_dMMEH(n}$6t?u@ZQ(kg-l?p`AecK}B^H#tLy!^zL>9zkDWms+& z1Dj_(|G6of$BBHCZ9X|^36tYAO2GJ9yDiE{-Eg<_g=3W_Bbg+bjjnacJ>dw3bo@ap z=&MCKC#b3_fj(yl2GV_}{`i3+*Zupg{G-?vCuNb?;{1F* z)SGxx82m;=9}BSr6Ms7&)ifki&GPy{Kl0?aE=6(lRC;&ZK&afk;-}Y1SBRtB*EcrW z{K&CHjcPj9 zWC0uLL7B7>2&+)^hI9J7^{Y%V$NKhh!6f{iTYZvs>Z>in^kpo&V(I74i7SGh5-^#p zHSH~S#B}!ele@Q)+z!`GZY(V&8b*#5YKBc6u2baV8p6#+|4n~^d_$@noakb z51@GPtF@PwxkD4_ezsMqW~wZh>A5Yk=#s}z#3G$fxJ1QOu(mxYb&}|={-Z>@G38*w zI4Wc(n<0WbB`dy)&GDd5$koZ|q=xW!!IO)wsL?TtVH!~<9D8EDGB22oCr_1gFxQjF zkdUT$6Fp6HiS90`cJZf)JQB|9aY|F<+v*k8Pj@}|rUaWN8Abc%b*}TPFuj^TvN}(J zE@>p`D_(ORh?u-8q(RSNN=iib1mo3eFq3#xE0joj@xcOT?^kBb5YK#xxJ;^OdKslS z*)7J0WrshKQbAG6LjC&!N(-n>ahVto&x2n1gOcU%(A!UpOuitNlDSPRBO_wM0YSh0 z40dgrXmN4Trc#W9gM*coRYJmh{Hy-DLwkF>Ri&qR2FU6kK74pr*QG{ogkrj3v3k5C zNKQ`v=g*(+sE$b5sc)~Zqw6U$xipfUo7>vLqQ(JnYlwt|*_7%AVhklwk+%DRr@7x@ZH z1Ll7%nX_mBZarmTVG%vi=;NcWuivWQt>+=7R37n5yJ7b(pR}`c#n#s_3f|?Fm6que z?XsI|k6Vf#6zX(tIqa;ini&|-;SconkyUOcaO=9vH{W1srPjN7^JcUH>GwBX#NK@B zY16?KBG1O;!^r1z^vG3lLU2r~QEB4t{1Orp+}wRzF0Rhb^~mK^2TghTfGSz%xgVID zM*^oDyu7yN=E-Ru_sh+PSy@0x6S*@Cf(@UN!f6z75TK8VL{5AbLr!q(lLLdq@n@B)#v(L^~a}AHz;LA?;}kzJ2jr znjd0gvlMV~ah1S|&qLxK-}l(|c=GA@ly0UQ8{khtH!3X5a;5hr=s zdwJC<1xL$2Jo$DjaD*&N$jBHhU31a<_t$cW&NqKotLF$HOhum(fJf09>Q}5=DJrpj zYa-O)Z5fym7=W#b19~l{$k*BUM^}K#2y(N%gxJMZ2aj5U_gFMF3L@u4omrDNAX}F6Vm8_c*0noe+eR`dN+lT zco}>}&%o*``vQTSI8P!M2S>q|FBV2dGMjg)sb$|?tMBUUq`Z5Vm^v&h3}jActn2qy zmX{aoN2ls8UXL8-hl(raP#T~pkAJ|&7(YZSUa*y8Q&-Smxtk>$?(kl$>!*0rSMWRXXY9^E$R6hsEuw%jho z@{r&Vbw_l^|8rQgB`>Z0LTgA}op?;-A;tIPI-jbpfu#}EnXDGDZP6S-tWBkEUA*b@ z#Q-aKKW#{EE-Q^0N{wbwb%!DP$&)7**47p0_k@z+4b%O>r!SrNRb1OMYZX4#*Gus6 z4Z@A!qk;lsDNU#N^<6&a?pY@%Zk_NNk-omZ{l3Q_bFf3yxFO&`F;1+pI?5t`V~=%UsWwKcWHvaUHQ5?_>`v}K@f zTufE8^D{NgV~z!O12y#d>H6{>M739)Y+4y$X3RyiFD3BgrCHnkeY*@oY- z>L~@EBx);Mmtlp#cYexJw++*_N`ddwYRpFSE~Xk?su~&zZbR7zGJ<=AbnbAgU!OQ2 z&%Q@S;K}y6y^v*elY~dmRdeOKAjd~%!aMFTn-Kntsm@3tR26f zKBYsR#8;xwF((dUj+Z8F_-k)(?{J=~I_y23k$_W0taHuf8~t(mk}W9j-7nL&)OYOX z1sG#)P#Adb1ZQN>ZFs|9G1!L5v#q|#oAvT@Zk^4lf}E}G*An!dJ+yov2O@vJCDA(e zhi=EADPADxo7f>SoDfu4RpTE>jS5RYt*8w8n1@paRZ`7%4nHptQ=;Tvg=4eAIng1|!-c^4dIHA5 zrZvF;fr)PIu3r8&go-!Ntss<~hMwNFcKY$-$DkEhTU)Dj*?OM%P*BjMc6xEIwY@zd zE)Hucef4{^$jxUhV5yv8kS8wHdLX)~1+6G)Y15%yr+FEY-z%@X1*~cgtlh=YetFcl zQsE5hTt_K8qoia*dAZY@H!KV-EDZ2vkP=$n*2YA}-r!qGNlE#tSN;C5wN-AEG}Hm) zaO=w3)>R%y7C#V*HgZTi66mc*=^xa#Kmh7eP-W~hdwMD?b^P-cc{bB+VH7>x-GM@ zk{_RPNu>Kf963J3Y~Nezj-0|LAN!fpL7!xTjRN6?J@;r zTm&sao)}}huaKx{N@8MA3xJ+&TPbgQ67;L=FhjDk7~fD+QWCmC67Fk%;w6S(pGu-H z8v(^uWV*nSlj`9r$Ual3t*xzHf%a>~64!sG_4X=T$=Us!oa7Y4>{IMIJKjk&EtjD2 zJl>j0%*e3$@~(oKOhedi4&$1fFrhnJH(Wy)oy((1$k%pugq)T1^{4yO#RN()qdNTR zL8V}3UR)i_UcW6X_;ZpyCabj6{sM(u-2&ZbOAtZc=g;mH*`r?V`U;MX2mNA|za4J( zvx}Lwg_1FSYmhAnd-+a+9*XdTMKXecc0~LB!Y4&|7BLiR~;V zHY^X-4Gd<6hM2@ha#d2J^NWi7DHJ||B@hs($j!aEh6V)$A5pKK+%{bzWfPh9l8qO% zov})t)%j>W@Iw}G3XFq;gP8Q3j$`BFi*|2fO`X;!;Ba_PB5x|^t1Sh2Hh|LS^*W=} z^DS?zM9R6k3h`6+_+wa2*Sx&EE0u}!a${?&!XmGr8DShwx%kLWg?vH?fKGPVIUOyP3xEAK7O$y2S5^zEWR=MCwxnr z&@tGESHC7llE%}TK3c0*ZSWD19#mPmGYwuLbE9ab*&vCo%z06vWL!#GTH4&4qrBs; zf|i(G#z=wM5SFNm6MtqbUFUJOlz#f#=^o7Va|Hda;7CQ?zGeqm3AvFEv&+H&gM&DVOB{n+Z$-M zpE~_^g>l5^f>e`Ub6qa?=fm@0?@9?0ZJTM3A;PB2zOBv8eWcGVwT^3`JyJ1|V7_Zv z^XKDlDk>@i)z^1fQ}S+0fSfe;!!jw?ZJ(~jrUfkLvKj1v^am$Z$0lSb#JTM!2Nm(U>}E@ju&vQ|2Z61&CAWTaH$$sxFHAoNzh*AbGnwd5YC_Dc{r%3nkM{O zoSfcubK0K`P>{oaCtwEtKcK-Y5kSf(!B~@)dbhG3Dj%u>5^;$DBv5&jZ$VlQqWMwr zmoK-zKAF*!e)oPbJ|>1S$jsip)Wi3?Jwn&Uh6i7471tdD*Cw>xp!7qUCc%Y)Z4=7@OwGRAIF5uPP2XOB}rsT@*yu`7$8$ z54Lmfa~KY{l8o=EG*$sANpi! z86FKIF6R+#8$Q)}@}~MU=t>uyx$kGUqIF?N%QZ1nI;UH5j_{!pLACFEMG3w$y=o!( zdQ=}Y(_U3a-hGmAJ5!f#f~#9&V4T11bM!-~c$ti3t&H2{8l#$k$$3oXjoakE=}b_F zw2~Pp6V+eD98H{gRakLxYWB-qcV@(ziC1@NSw5iaa6jK|?ZFT9l2xpLdSy{EPJo0$ zaTwp+1jOb4>|*~L=PJmA2%1ffaspsO`2SJqoechVvkmY5;sSu;<545y;|Hz=xL_r3 zP0xBQ@)`iXHrdy(j(k5hHdb98e4!u{3nSLaK?D8B4ikX)zi`sL%rfYZ2XJ-t&C6Az zlzrAPuNG}!|KWMSxO5^t(yI&M(3dV&`R&{8R^{xNh+O?NM^wK7Ei_#e@UqVhtp)pH zPsb;cYov|}DWXp|NQnb!$#N2)L4br11UrnCbs0WolS(T=o}n4gbyRFn?_p-vS-#;% zAo-)?c*axJrb$r?8+VNj@SZ_Y?9yIxj3-GSJ^;40?+*YYG9J@4&hxww&p(^?wbOB# z?|M{iZEc}YC{|b0EE&G8YjRrJ{ko~u)m5u_`s~qbqY8#4Sc~q4Zu2*Cc4{_wVo|SZ zlL#ph*mT6WR&u8&dy5X5hK8BFo1yPWZrY9{fK!*4)31GX|SwKY#chc9V9b%&U zQe51Y(B1tchu#Ne-#t*i)L~8;d`O}y?h7>pywC3LH>S%NX2tK{zZ)4DJ-ffmnx~qX z;F;;MUH~HMBOzZK#$Os(!htCRP-0Tv;hzx*#I)Bz-^t0KBK^G7U-}Pooat*ZV6Dg# zvkSDevC*7dB+-fV$VJPY#X6zClX1c?ECA;l}ttJ`bnATM~gWG_w{Q?FI6+d zIc%ImxpOCt0wZU1tL)}uRVB^E!)oT&#o0;7o<4qD@b#Apn%|Pm%wS4X<6u%jULftXuYjfkm856* z`t@s&>m<>O>jR2Cu^pVBnD~PqeiL~)t@+^4tDmEz0-gzUost2VfZQg^*=ablNq-$; ziBBQ=m-O9LQK|z$Rnl@D)-*zo{3;8Okf+2g%HxB^mY;_-x z^Ijm8)6*Bv)v8PYPxa%7^9$n_)B75K>2BlJwdf!^4;*?l%RTOfX1V3#6BC651)1yz zB#!f~^UP8>9x9Ziwaz!1eJp^Ralm(`DhX=C1M;cA_B#SvG?ZWdX7Z(_;o0368Wve~ zmD?4{x+EJt1fqI0I31>ae)x(tSy|FfiJC}ziwdZB!`bu>Q^cAetEkeeawFy^Qg;vR za{ze+UBludspOt5@Em0=b9uj9QEV(wV*TPpOTlbfw|zJ7vg%LD#jiWwJDy3OoA?$Q zcO#e}<>W&=F8V&_N6*G$Vu-AKd5n1B%+~2zSpGmxqk6cLk&&VOeRuK%g%^9H%K6R< zu!tA(dCb1qq{vbVTxBmiS_TWg?KUixoY#<2T6n7>DHkiV9Zl}T(5QSC9*=+G&<~4R z4ck)j7j!9IRIW@!Aq_mJ;75Oq1S{S4jJzI^2c&FFfY?*lQU2cAcZg;xtMdxrpN@YL zzWAXPj1IYAmexN6k|#eN@hZ^xnX~4OE;x+d2cdSle*k#(l&{Z?>9lnRFAWeK1*4)z zM6Pa2)&5)*+myb%x8(3ZHEtDn_l{sHiXLLa`TL*sxv1D0wJ?~<&ks~@6p3FXec)SR zIi6xe2WU&BUgYlOT$5k>K3|BmPH+MxGtT>f?a(l8w@JSVt~ZT$DTY#QCGnmn!)X)v zBb4Oz_7)7jMC0WR**V|k#hD5t`*E{5>f}Y0bwz!`+=!Ye?WbQf%+Mf=-nLeS4HaA! z-PK2Zlz1&MS=UVa(}DSr}B5feoaO%)^CoajhU4Gof+{g#$Skgo^ta>)K1>CjUG zFv3(Pi^a;iImFGQQ{K6D?O@;y?9NNt!nJa>755~l2plOvM<7Ga9;&s%>WaHYTw|2- zYTD=WTq(>vnKEkO<~tlI#CcTkSO!%Bz5D0M{>`P{YsR|XA+I?7G4y`t6RAJUOIr}4 z8?nK^NhtIt{nl~CCzv&8rNsbY(hNQn_TGKo%m)3-lZf zfsv{_%pET=}g)FV3(o_il+aHWH^pc;#o{=b%`ZH+TCUDkhcD!j=9xqRUQhUeK=Q*V9-~L5j zh#OvOQ@5wULe6Voh{KGFtECH2#^iY^NMMJTw}}kYKXbtx`p+B;pK!-p|YrJi-jICMlx*bakt7 ziHVDgAPKh`s}3K)vFY3}9KBe&yStq-&e>B~ZC-}V4GFi~1y{h0@Fzw_SVE7wBBAa- zvH0G}H!nSlq=)d>1Wss7z|lL|aAmzbpT$FpQpv`Za~K3DouT-$^g-Lf9XI7t+E~{Z zJayauCBe!~Mx9hJNY(S76l!v!>LLw-Q1RKDA)LtVYGUAvPI)e?b>Q3>>3 zfHLR4bNKdS8*`ivl5s3IG<2QonX{fmlJG}_#sNK`;2(0F@FG#a{}4ANb)@Ov_m-x# z)GK)_aZ6q4j&bicpSeshGX)3L0m%A?BL(XJE(+#2iAm&ySy7-SveYFQ>Zm+H$JC(v zZaqYN0p3v6im-a>WeXLDsw;k4&wWiCq+dDhl8RHkBtcGPo1CE5RLwj|V#(w6JzJgb z5hEKL_A*J2EpvoTuGgM#t_ z*_e-wj%pt)hVmD`Aqta6tZz{nS`q^118yTEOUfVLNNGT6V@}fSwu`M3f8PqaSt;Nl zI8c;JUci>O+NryND@P2VMDwqG~|mjPS}?PZ{r}?e$u}7w$412hozJt4{-M!HvsY7 z#QqzOs{s%%nlcAqG&m##P%+hj2?bZR;^Hm9cf*9h+M!J!Q9XzcwSW^^$RP{~3HeFKLKv5iz$W^j z^rI{ORdS;FY4}OifK6KgJ~nbmCN{ts=@7^yYg`un%LLkcFIo&Q{af) zTs=xY^lC#OK4xZ0FmcCaWe^aNs8^@FMgc(H(mqDF_T@N+PM-)D8D%v6XM!W#-Q5A9 zapT4fCFav#)sA^e*bh!ibGtX?20xtKuP%aehW1E?MhG8PJq}TM-(K5-_|YkKv+2V4x4I*oX4wI80M`O(jVwA z?=A>sdhc6;Rv|7kKQ?oXMBDHq?m$m@<5EnNp?erm34nJ`7KvNn?D24C%iv@yk)nkD zifvVc0WNbl;I@g8v9WuB$^k!_kG>)I$*(UFyzM3Mo3dzq>xv)!u%w8hMePgK-IVo@ z)*jfbMz%KwpLwovxii^s|0Kc(3?3vLP^GNcLCeM4$_59J#!D8!UjIXyiaZj^lMq~4 zHS{uF+NEx+-Pd)&2N&C!09r46AEa847}$(4HxlW*=I7i5{L4^oc&=bqGTG>${;K=RmWBP_GpgVl1_e!kg!A(~5Kt(~fWB}=$~qa;#c{id zXCx&hiKv6vq?Pb^^rix6Il}|t9zcG)B3idL(M0P{0?>(EkZEaYO-xO3LFmL>tXcx` zpenl+U#=zg+_;`KF*vA}o|`P}b0HonkGr?G2l{i+rNe)ALPGtObLJz~PF5BDul$Ykhp*jQP=NzJ-G+{-G6SIy|rcyNsiBIEm$?IOO~dJ3MBa+l@L&a9!PCI!2ywZtvP zxRUx?xYAN4lU1I_Hd9ZVue8zUFs8u`hj{^9ODpf7v8-nJ_!zK73#*>vAt2 zPlFGLSm`%fim$UZ;ezb5W*m(K&jX(X;8*QGK6A>1DuU{ELUKOw^;S!c z!Is~M$b1%>p~S=SyZTa#rU!I9zqy?FL`0}E9-)(R@Ae-3#k9vP2c^)g-3-WqW&@#Q zT!+q@xBOx^IhKBQk&iVGZskJ3e(LP(-1P_rdJi-4{*e88HUvEOFZB{-U6mgsBSZwaO?W%nZD9q0n2JGZYP-VqygOX_a5)rk{-Q5Sj6Q;Hj zrsh`>-YiqZBk$ETG*~tGK3?qOKC!m8_V5tBxGkZCPVeU3Q3XgzG>Z?U?pqxZ3XGHA zcsM3Abdu-k4ihJyjagF*RKYz~3_jqt^M8`jvjmKtx{$!}6prGjK zAb{V?zhUg;^hijEB8XujCd0|jj*zOg<=QM(R1!-Rz2oT6OajkT9Gc47XWE@ z{nghgW=_f`&(@PjOGwO80?d{~#!Vz_&?N*Sq>Y>LviV40kn z0+tbLE)!jAv;u5pNW}(IVUv`k`(A7Vb##2_!x(YpjE{q`adGvW9GIopLLVaSH&R!_ z8_U-A+`duZR-p0?CGuc-Y3f~}p})2T(E9eDc>8>JZ*K{QyFS{ZaFJCDC*cxhK%`wm zlbNFA;bCD9FNPYHQ-_sdzfW03H^jyWYx5C`=j$RzVu-XS8ZUZXN(xHH?!ZgJpF*MJ zjq~1XL-(8=!*_aFSttz>Zn&*!cK+8HcNNB7JLp#9pF3PuQY6a=zRx3gT40jtgR7IQBl!DDzHOlt|X$4_vh#5jJ1DEmcoQz%R`}& zTKrI@=sF$>gIJ~SuK@24%$*#NrkkaAYf=*L;eHwajEV;Tp{g2nB&qu#y0YyPi4(ui z!9ULuZL#epz)iVf`X`E&0eFuIfRl2qx9A4&IQYYQL4`xRn69MSD^CK@(LvmU7c?P@ zR0HyH3_jX>Vr`rNIv?)NjUWIo1VFDtMUL8xH9R;)BNilMR@q$Vz|0B#?1`$h0AC3j z8#&R}#Wmu%-lRvRCK4au*L?RoZ^5cVpZx?E9U2O^L+w^RZcsCx>C=Gq&pNIF`f2kG z%PMfXH{{^ubyGkF>|%-iFE~nGHO+VN0cyI$BcY&M(pRWYrmv%<&e>}k{wHZ_(zx;Q zRmU>6kKVMAqwmLF1=q`UTJ2w*cltdO<);wbIfA_T4! zaKk_TuQ&XEq;6pG>iv=Q`JYo{9V~oC1pymQ?eum|P7bIPhlhvywJrrcRMgaE)7yJ) zASj+bd18mD+6e#uN#V{#1E6Zut4=_D>sLG6SObmB7YRkWaDD+ZS?hGvy{8GQVg6e`1eZJRI}d)6hq|t( zJ!!uqDk(W5=;P@r<`q#YWjTQHi0$D+5^5rvayYyjDs`Z?e{p_>pt2_DnDIXQ^Eof? zO6LtA!>18`he3odHxu%`m<(D2#HuBK`O~vA724;Tnw5?#&_2r_i+it1dJx^6ox&c6 z`KQg%V1Vt(8#61bG~gifs4Xiy*qv_yg~edYOd!)faqu_VcgJ1%`~Ac2e(h4D035UM zBE9M%3!=6MgAn~yV6OPK3~&_XE6Np{%G$!h4Pfhzs|B#_Bc9Tag4ZP_gQ@^7K7AUt z`|MvpWYqC7}XM^k}+6dJT*DVW7uGIrRk?X)6fWXWQ0JozLKSh5l>G~cfPt$a_yV* zhQ_%ihBwoT;>ybLFX_4XiFi&Hq+hXCG1q~d8Nn{s5vg$?vW7YNJ}p`|ZCWswNGiUV zOyds8`7mg-N--VZxOFu*%iK}QB4ps;c+B$Pn)>tSUh(w3uiI#SP!y2_>N@s*4f*+I zXMeVmo-ym|>D^(YltRWFXqSlG%jZmMJ2|QPdl$x@T zRw5`~{CUcLHC;8hA@B_R(b2iJRT@Y%hM&mhY!Ua!H!2J$8~Kb=LN0eDH3kG| zk$cQ}*VtB!wy3I7fg59$YXLWTcy(?h$a71}*nB;g%5ie+NydoNNgy6|w2Hr2%1e3V zg5X^%QDNcLAag6~&$f!*vDyQ{?I~X+*j*JA0>{;p0z;P90O8iqd{V$9dPROlt9X-C(o*d``{Yz1N+j-`uh5!qFy(b^e>-mR3Yb178YMj zf;_EXNG)NXzY;$~!^wVj*&0Q9@59y*l6K=gdtfsiQ)dqhxmc6gaRz;)f5i=8uHRT+ z2PO|Tn{S)bwVIM4fN>$50>SjXtPJs@e~mSs1Cy6E1q1)K zMb288I32+?R#c`vvUH1T#M;e`QnK?T3ERB%g9XsH*IUxpji(}@DkM?XIyyRN*H=Cn zC&OUs;$a|$wxmk}mEh+LGDX>uv;FB=eZ%*(b37_7tqXS+< z`WPwCh^LHB|6Du=1fXDHQPI$o>*^wjJptxAFJF@B#n+X*$GDLVY$~$&g9{5rN7vpi zc2|NlLlYUSh`-je`MPL`HN(w%ZX=Q(or<&~rdD!{BfZxY?~;#)ofEm7{3PfYGb zmgMgON%-{YNI7r>YMBD1km=6zktgCmw7`D<7Er3}raYp3aQ7XN8V(QL(qQI47mveu z1xpuyl>%6brxKffT|q$sG)l&KRX|W`ZFx9#OAL=B6Uv4JbyXc;%RwHnX7ijpt{H+oJ65d?f&N*%_w%k|T@8_)L_j$|v&to<0f2X@c^B8WWgD8KQKYO@Ng z)o@(%HVkgD7`9hf+(ZcKxC{<%0xG!C&+sB}9Ib(ybRqg77kj*SE)7nfWGJPHfkp=S zE?G#2n`93(TvXJ~j8Fggb{q#RybUzen07qCJ&A z@5phtb3gltu08+dn9k}~bK`Ja!$i|ArpRVX-2y2rW?~9ngWiFGpjv=O@rGgpbzhG; zEV)zzEBhl}UV1_ir?oEmc8qwX7u+L8Dxii z_-l5;#;38Xoo-No067+PGO_AW0fN?({&H{MB0xQhF#{x_ib`=ofpkrEHPsXVbcA52 z$k?ls7tV6;UPvqG_^(oMCa}5@Da~I3mX$4Udow;RjzR6ki*oNX7p#{`px1JBbyWv* zQY@3%^iwZQ|9X9vNeQd~CHf&P?T)|`B_%0Q9v*x^SDvT>h!w5P4hj8?og46`-Smv< z&EVmMMpN{^9c^Jr2`@Xl)ZM`WkT~RZ)L~VJ?+A(S84K2s8+eM-dBs`IgJK8%?-)66 zZ4G?V&r1P)& z#CDQrOXDD*uJ||14Vvn6h!3h;7yVA=FIEUpB$pF8E+&m_&}^Vj<6;NAiSS=rrO_2_ zI5bWGLsA>~G0Jon0U}a{6SA*Ldj~yE7z3IG{79eI7svaVUr9n7gCgzNiDi(l4`ttr ziMXwC`CN227Vh5XfVuCT8g83|_W@D={>pSI14EOUwdcN($XSmq-piiOc_El9q7qye z(XIajk;pX1UlE9=8m|C94+`L0GzrIV$@^BOb2ITD`^eJe`FP5o&_EHGJJRoYJB03Yy&~eJ( zgn0NLheIzM`Ze>k-$QoJr1K0}{G;=+FCJKA4)^v>LD7K@h7i#%7J%Q4P3e4l5Gi7= z+dw1;3Jg@Pl?2m1US2>6Un2KXR#A~(01guJ9Uekl)0`WcE$JiXa0Kin9y4M3g&#EdWR=b<-M1F^eBvw zzEmM*eND}v^57r~OUu&FfVNxfp!X3MHI7NPRx?{t%oj#a6b{_^gH-NPU8Sz|I9dnd z`-AF!FnXBzu`{a9m(_)goO~I0_n9x4xeq`NVD^33t)!&Xe|of0dJm+lH9)`)uRTD@ zPsDuq@LXTtTKou%SlKf`C2VKvK!OsHCU+MV6U(wrk_>y*V+l(;Urx1c?=VCyPzck0 zZv(TGPQxmnB-`(8LFH*h99X!xZY#bPdm`+8TAn-Z_~jcPfhePx*x96IiT84{=?Eg9 z+Opn{6Q8g8rZUmj>gBsMTr2%TGj7bE$n6(jP<|>Xz+D4YFp8-`wUh{X43=D41JB+U zvQs+ReWtBraVh)Ea0nRCT0MAKr~%rS4a&U`whXD^y5&mis)@6ICk=zouv5m1t$hLT zAqst-2`yuU-+Q&$!n8`?E2+}<#BtRrW)|bElhGPl6m@3h| z%NRwv7^MB{rk5f5+B%p3}_l`XB&XugG!ZKvKjYQ6l%AH zfO-Ri0trSDpQx?~Kon|siw(~v$LE|{y&V_J*h}&ZfMzlTKAR!D14b9uV^LaMR=s|9 zm}|gRcgKwu*ffg%wQ1~i%H87pv@PhX!C^}qS}9)yoD}S zgEaVV?&r^0^%IdI9B_IF>^B>d$ojCL0~k6jC7BIW3KGEVTpIMzSRs)bUOEbnN8w4N za%w+4NshDSE35hUY5%~Q2-eX)>mzN-ukGBqc~jPN7n0J({2_r3%*7}hXR#Zlr~baj z!<)rbr4gPucYZt@bqF%cp5bqqCMz6szN`ciCg$C{@I^T*A)XfXD6VvrgJ{(0;n5ND z1u1K)3`^@UFW>$Vg6*4mUiVJL1>U0R*0Oa8bBFH~^TJRz3>9o16b51CcCtnSzV_p2 zlT+(LTbQ-&m`HJ?rIl5A2K2tcyD1s3SNETwYW&RiKUXa{?}vH5V@gd50~cmuF#x#~ zmm3kTE?dtx!7`38dDg&zmfA`ppUw71|mh9UYd%+-H;C8X@Bx*=)#4WAU$_5-B zn}*k~pVb9TlEm!$g|PyDJ#oJ)mMx-|@PwJsMuPf)IW=uD_}g7(z3zd+k_HP2eQoXJ zXRI;bB=OzBP;u@Y}bj1PZ53p>#nu)+VHS7om+Og650Cq!ro~c09m6{6jRhQahFu7em zJ{~xcz>smiIaR6pqW4s(3$8ZCvtQ!tJ4z?H6QYES!Hl+h?2uR`udb)k!`I;Nx55Wl>XAi1%6b>9sFmGaF9@MqoI3X%C<2*K0W}K`HSi4OuTh6G-5P{ zUitPM^Rf!&>>F`8FQOGNg-tTcA+2+tdXPhJVjE=V1g@WTf0CfZr2C6+o_HghaumHv zb;LjHS+vRq;m?Ox!4aZ3Q;xp`y+p^0E2P3f&4Y5B{>lUQj1l(HxO7m94~RKiOqX{% zD{g|{U_hnyA*$!l$m!u(gKsO8H7jVqlgVLCI=Cs$2aGo+S%hGBg8KH^G{^J}8;OPL zBc3{ZKHj?BW-!6-|APk1zBwY#9Mqm}k{&W_zFDBxvm`V?Hy?x7fdwZ9Md^s1{2%0k z{J)tS{tJKn^J@yS-~VT91x+c^Ms+Y7e8gB-Scs&Pxcgl_Dk1{;V&LDIOH6ezb9wF7 zY17WmN7DZ>bLpo};#K?wl`Kh#(-s*Sx&ElH6+m_S`%i+SqIkKvX{o46`vfH=w_R#u zKYaK}{dbJzmK)=#GbN4`Xb}_MX?!{q{ch{oKd1ZAE>9e(;D@hhz@KjximNaxh$TrJL<530lO`|dc;K4SmRn{i<+UAku)nl z8=3qq?24e&KOxULgm~s1co^ZeQ&6xO$QI_=vuAy3Js_@2TM`2a-n<;RJ>&dljK-L= zWfpm)o=ckLUo*oTHDZgM>0Q4 zhus8fkgKJrjLl2R+>Nb+)Yo1al+zkBdBtgIn%x@8ZeIfdYz5o$v5TK$=Sg-iD=oEp z>1XyM9sD;@&`AecQ(RS_Uu3!PTH;rxH1p{{XG$e{=Ca2xIyD=9engzVnR&6N(rm2p zLt+E@%a>?w@=cK-BJtp8&M_Z7J#+Kzd7z2Tz(uC~Mq!yZ&BhTTj;(`kWEbQhZ^`U4 z9D}n&dxIK?H;2yNI$>L=bTP$)RDlNurwd`BFxsSzp1GrAmbdirwdU+N(wtcY997bz zc}x!-+kyW`%Rx5WrUBO{7M=e*;p=!Pf)=DnnMb zKH$fdQK@v>Zoe9z;^N{|RmoF%?>nr9+wPw>!UW^Wp72Gv9Pp}!o{~P)LXxo>2J6^9 z6w22_OHPx&Cv#@>xC2E>QAu`ggjPAw)Y#bh%)`;dq+?;C%5~J!S^wyQOOB+bYi%7w zFPn&LtgJCQ)7Ni>m1Hg&I1XzV8b+F4LZ*>j;^>hiBVD`@Lmi(#gfY_Gww+M>Ig~Dr zOWwCLJo=Bz-=93#Fe7_D%kL9U%%AVJX+WP`@`*&5{F-jb1{*Cdjdwq5<#ctbskFvm ztv%96y}fqcJIcGH%4LcA05~$k@{TR%dTf(hI&dBN7LI=3 z+CG1dl){OZ<|D`G9DFqW*)uXp*x5*5|6OZqmFI9=lZu?&adY!M)`C~R_|Juk1+|}e z=n}%eC68z+jS@CCHcn7LJ=vTxf}A^GByrjO9q&Xy_h0b`LZQ|WXs#0=(CBRf9YMLH zo4<$cZq(%rw|PY;m%Wb`l@b#ZVfvw5SilkxA;X;%`(?@`ndkoA0Er|ZasA?*GW~z@ zPm>M;w#+7)gL^(P`R65}Hck@sens@mupnE-XO~I7z{9SE1&N-^y?Ud$fe|({xCnGNb7$|X_6@RwWP#}#8352c)p17e=nI6jgedD_M*o%$n_P~=w_EYU;% z&nvzicAem$Rp{WB!nQk5zm+Qa6>&gIY#)V(S3CGRg;yIo^vN_H&|yN@<6Ly#BQ?w| zNW0EHjO-iUWH039;kFkPruj*rdkCh5%aqH=*VEdHGTqL1BR)9J%hOZHPNlYNN8!Zs zvC2>Nvu8A4BXQYT;^%R|pGG(-!IK9Z-P~eY@xMN^A|Xph2M0<xbP&3)TJxG5n$@I4d%zT|5ZiAI<*G)OBn%~8(0_y6ja ztlP^^Po=DfFT`@geBx3%3s6X4`F-^`K%$6226IG011FMAm3PuWXe7EAN{4eA)>uG$ z+3;fXm<}+|&=s84jdX_jKL}|QOj&2&7Q$enX%}6?==NC%osD0a{<6^+1Cjt?Cm`4p z%RWDrC|~f|s{6%JSOawgR%$g=-TY&vzC&yF`N38E+Ka;t1>B894yxV=xz2{G29F^S zbPFner`xQuejp(BNUYMH=!*o`=3z-0Mwf_waoq{Ti>@w%A0_hfbBoYI&Y$VMZb}(ey z!B?qVg%*2mSwJKbIRy4cL`C7UPA{D>LX*4JS8`3~4tjDB@*e*jnCh&U8f>MQ-rkHx z+nAdx%gJ%Y*m0>_oWM8aF=~OOI1Gh@o{@pUNyF3z7%n+Kw>nn?aR3Hsth-2K@IANV z4~K6@T!85;siJ{;8~(h!Fc#OL-thRbtD4uSv~{KFsZ;w^R3a2^s=K;Q)I|yuUHVxA zUs}lBEBl&h;43X{H7RpZeIktEX*k)deusVE;b@+DmYY|mM1qoY5T0tqq1RC4P;Vdr z(JaAAJMeYCvE7xiv4%?KIa}K+aAesl6`k5ccBfWSdOgK!4^nk>b*E@FH*XL>iB`qV z41Yg#f zJ}h-ymvk#JF*K5SlevY(8S#hn5sEFR+uoW2QqIIp3o+@ zvxzUPr(CnbW6XDaJ{Ey9Oq60Wmp+{rg)|u;>?` z$z-1&)ZUU3g~H1vNv*_p=f_zq)jU z`vT&(^`aOguH#8X&Jc4xLxmy(r?WtPpjWFqrFVb zUu?MNeJY%^35YEmET~wbW>mPD{-457m{qr;*A?0&^}W41G_tM4L`KBx&`>0a+2qE& zNM_S7so5!vU{wAoNE{-kpy}Kg@LbjD?$cKfy@2SG0Bta#Lv*Pv=yUc5)*x3kxH}Ex?r~O9}4o+&TOzTEk`18`yL4HQtyN1 z0W{2AQ*TuT1yMEOYqQ(qHHTJ#{2~FCHwlWfLEkF8{OV*&zq?)1DPvK6TD$f+-Y`2$tC%0F7C6lc- zPf++qpJ_)XYXRWuTQozwA3;^>eHp1rwc1V(2FAw5e*E~6v>TErYLnEavT9sSZC2Z1 z1kGTmh`qzi!e~dc;!fK}o?Zzag9qHHqpe8x;V3@D3~y~t&2P!58nP~N$-Z2Lqwmci z>>epxy5)8YJwJDArI-qjLR*v;JYbnIQ8ba|pe^i|hS#&Sy}KWV6I)*kjYwH72uu$C_>3 zMES6N9ObtfJ7$)R)C57$Z+Fp=aK z{>*Ot(|DseptrQ10P6uMc9 z?~0Vad$FcHt03C*)8Dn{GP=K+f_3FpxskxA-GeOo;bu+V8I)rMJBb-f)`zaz%QopL zzue?{%{3Lq9iC!(u+=9cGZVs{LZ7?4BgSr{SN7^LC+5?4Bkiwnr)c4e#1H`mjF;rN zxW3#)QRuGKS&;|_R>{k;Fe3vMy1HILBwMorYK-U4@6jkJtI-yn*hT2cL+Nfvgwh*^ zd(ioTM5PtU#l;m96&_v_Ai=$izg|9GN`63(os$D?AV9Yi$*~g%>T5s6k1P5@KVaOc34A$IlOE zzzB04Y~@-@@K@!5mGa-g3Z!Yw*1tDn^9M)sRSFBnuFor7AFr3t8n?F8hQy16j979O zD{1)?mB}xcA=@5&VQv(qh5Wuh7^Mzo!9?T2!g#j#WK!2QI_(toge%MXXJd-|R@mYD z;EpS@rzYN2f{PA50hH4d5EyLbc44ufTLQ)crkkO9ox8$$AY=&EeTw$gDZ>4h`!LOe zs&9kc>e03GV<0nnd3NTD)w~qQxbTl!L6X=^H)J;`q8w_JM=5qgy_ve$M6*#9lHUYH zYz%oLU2+K3TU^b{L*SiYz@Qh}iVPMBD3sMa+`}gX;SARc49q+CwskL_o}6lEiXw;H zMoV`^{%@o2WTo$?;C_+{hlQNNfBra%|1BO5zApgT-)sfYHvq@Nn`sHB+Y7Dy$QX;h oW%cihZ~9FFe#l+@v9jKyr}ukiEjtseZeMr!5Kg;5(>Cb80IH2qi~s-t literal 0 HcmV?d00001 diff --git a/articles/unique_network_flows.md b/articles/unique_network_flows.md index 7cabe48a7..0f05e2021 100644 --- a/articles/unique_network_flows.md +++ b/articles/unique_network_flows.md @@ -19,7 +19,7 @@ Original Author: {{ page.author }} # Unique Network Flow Identifiers -For performance, ROS2 applications require careful selection of QoS for publishers and subscribers. Although networks offer various QoS options, ROS2 publishers and subscribers are unable to use them due to non-unique flow identifiers. As a result, the QoS provided by networks to ROS2 publishers and subscribers is vanilla and uniform. This ultimately degrades the performance potential of ROS2 applications and wastes networking infrastructure investments. +For performance, ROS2 applications require careful selection of QoS for publishers and subscribers. Although networks offer various QoS options, ROS2 publishers and subscribers are unable to use them due to non-unique flow identifiers. As a result, ROS2 publishers and subscribers can only hope to obtain undifferentiated QoS from networks. This ultimately degrades the performance potential of ROS2 applications and wastes networking infrastructure investments. We propose unique network flow identifiers for ROS2 publishers and subscribers. Our proposal is easy to use, convenient to implement, and minimal. Plus, it respects network-agnostic and non-DDS-middleware-friendly concerns in ROS2. @@ -29,7 +29,7 @@ In this document, we first describe essential background concepts. After that we IP networking [1] is the pre-dominant inter-networking technology used today. Ethernet, WiFi, 4G/5G telecommunication all rely on IP networking. -Streams of IP packets from a given source to destination are called *flows*. Applications can uniquely identify certain flows and explicitly specify what QoS is required from the network for those flows. +Streams of IP packets from a given source to destination are called *packet flows* or simply *flows*. Applications can uniquely identify certain flows and explicitly specify what QoS is required from the network for those flows. ### Flow Identifers @@ -37,6 +37,8 @@ The *5-tuple* is a traditional unique identifier for flows. The 5-tuple consists IPv6 specifies a *3-tuple* for uniquely identifying flows. The IPv6 3-tuple consists of the source IP address, destination IP address, and the Flow Label. The Flow Label [2] is a 20-bit field in the IPv6 header. It is typically set by the source of the flow. The default Flow Label is zero. +If the 5-tuple is not sufficient, then custom 6-tuples can be created by combining the 5-tuple with the IP Options field or the IP Differentiated Services Code Point sub-field. Such custom 6-tuples are typically used as workarounds for technical difficulties. + ### Explicit QoS Specification We briefly discuss two relevant explicit QoS specification methods for applications -- Differentiated Services and 5G network 5QI. @@ -47,7 +49,14 @@ We briefly discuss two relevant explicit QoS specification methods for applicati A frustrating problem with DS-based QoS is that intermediate routers can reset or alter the DSCP value within flows. One workaround is to carefully configure intermediate routers such that they retain DSCP markings from incoming to outgoing flows. -- 5G network 5QI: The Network Exposure Function (NEF) [4] in the 5G core network provides robust and secure API for QoS specification. This API enables applications to programmatically specify required QoS by associating 5G Quality Indicators (5QIs) to flow identifers. Twenty-six standard 5QIs are identified in the latest release-16 by 3GPP [4:Table 5.7.4-1]. We exemplify a few of them below. The variation in service characteristics of the example 5QIs emphasizes the importance of careful 5QI selection. +- 5G network 5QI: The Network Exposure Function (NEF) [4] in the 5G core network provides robust and secure API for QoS specification. This API enables applications to programmatically (HTTP-JSON) specify required QoS by associating 5G Quality Indicators (5QIs) to flow identifers, as shown in the figure next. + + ![ROS2 Application 5GS Network Programmability](./ros2-app-5gs-network-programmability.png) + + Twenty-six standard 5QIs are identified in the latest release-16 by 3GPP [4:Table 5.7.4-1]. We exemplify a few of them in the table below. The variation in service characteristics of the example 5QIs emphasizes the importance of careful 5QI selection. + + The 5G network also has the ability to sensibly infer 5QI QoS from DS-based QoS markings in flows. + | 5QI | Resource | Priority | Packet Delay Budget (ms) | Packet Error Rate | Example Services | | ----------- | ------------------------------------------ | -------- | ------------------------ | ----------------- | ------------------------------------------------------------------------------------------------------------------------| @@ -58,8 +67,6 @@ We briefly discuss two relevant explicit QoS specification methods for applicati | 82 | Delay critical guaranteed bitrate (DC GBR) | 19 | 10 | 10^-4 | Discrete Automation | | 85 | DC GBR | 21 | 5 | 10^-5 | Electricity distribution - high voltage; V2X messages (Remote Driving) | -The 5G network also has the ability to sensibly infer 5QI QoS from DS-based QoS markings in flows. - ## Problem All publishers and subscribers in communicating nodes have the same flow identifers (5-tuple or 3-tuple). This disables explicit network QoS differentiation for publishers and subscribers in communicating nodes. In other words, publishers and subscribers in communicating nodes can only be assigned the same network QoS. @@ -80,17 +87,19 @@ The link P1-S1 requires low-latency QoS from the 5G network, say a maximum delay ## Proposed Solution -Our proposal to solve the problem is to make the flow identifiers of publishers and subscribers in communicating nodes unique using existing IP and transport header fields. +Our proposal to solve the problem is to make the flow identifiers of publishers and subscribers in communicating nodes unique using existing IP header fields. -### Network Flow: a new QoS policy +We believe there are at least three architectural options to implement unique network flows. These are shown in the figure below. + +![Architecture options for unique network flows](./unique-network-flows-arch-options.png) + +### Option 1: Unique flow ID from publisher/subscriber We construct a new QoS policy called *Network Flow* as a support structure to enable unique identification of flows. The Network Flow QoS policy is parameterized by just one parameter -- a 32-bit unsigned integer called `flow_id`. The default `flow_id` is zero. -Each publisher or subscriber that requires a specific QoS from the network is required to set the `flow_id` parameter to value unique within the node. The `flow_id` can be computed internally in the node or supplied by an external component. - -Eventually the `flow_id` is communicated to the RMW implementation where it is converted to a unique flow identifier. +Each publisher or subscriber that requires a specific QoS from the network is required to set the `flow_id` parameter to value unique within the node. The `flow_id` can be computed internally in the node using simple UID generation schemes (atomic integer counters or random integer sampling without duplicates) or supplied by an external component. The example C++ snippet below shows a node creating a publisher (`pub1`) and a subscriber (`sub1`) with unique `flow_id` values. Subscriber `sub2` is created with the default Network Flow QoS policy (`flow_id` = 0). @@ -105,26 +114,36 @@ auto sub2_qos = rclcpp::QoS(rclcpp::KeepLast(1)); sub2 = create_subscription("sub2_topic", sub2_qos, std::bind(&new_sub2_message, this, _1)); ``` -A programmer-friendly alternative is to parameterize the Network Flow QoS policy with just a boolean parameter called `unique_flow`. Setting `unique_flow` to `true` instructs the RMW implementation to generate a unique flow identifier for the publisher/subscriber associated with the Network Flow QoS policy. By default, `unique_flow` is set to `false`. - -The RMW implementation has several options to convert `flow_id` to a unique flow identifier. +The RMW implementation has several options to convert `flow_id` to a unique flow identifier visible in packet headers. We list three candidate options. If the node is communicating using IPv6, then the lower 20-bits of `flow_id` can be transferred to the Flow Label field. This creates a unique 3-tuple. Else, if the node is communicating via IPv4, then there are two alternatives. One is copy the `flow_id` to the Options field. Another is to copy the lower 6-bits of the `flow_id` to the DSCP field. Both alternatives enable the network to infer a custom 6-tuple (traditional 5-tuple plus Options/DSCP) that uniquely identifies flows. -The parameter `unique_flow` can be similarly handled with a minor difference -- the RMW implementation generates a `flow_id` internally first. +Both DDS and non-DDS RMW implementations can trivially set fields in IP headers using native socket API on all ROS2 platforms (Linux, Windows, MacOS). + +### Option 2: Publisher/subscriber delegates flow ID generation to RMW + +This is a programmer-friendly alternative that modifies the Network Flow QoS policy to accept a boolean parameter called `unique_flow`. Setting `unique_flow` to `true` instructs the RMW implementation to generate a unique flow identifier for the publisher/subscriber associated with the Network Flow QoS policy. By default, `unique_flow` is set to `false`. + +The RMW implementation generates a `flow_id` internally first before writing it to an appropriate field in packet headers similar to Option-1. + +Generating the `flow_id` internally is a matter of implementing a simple hashing functions parameterized by existing unique identifiers for publishers/subscribers within a node (for example, the RTPS entity ID). + +A suitable interface should be created for publishers/subscribers to obtain the unique flow ID created by the RMW. We prefer to use community help to create this interface. + +### Option 3: RMW generates unique flow IDs for all publishers/subscribers -Both DDS and non-DDS RMW implementations can trivially set fields in IP headers using existing socket API on all ROS2 platforms (Linux, Windows, MacOS). Generating the `flow_id` internally is a matter of implementing a simple hashing functions parameterized by exiting unique identifiers for publishers/subscribers within a node. A similar hashing scheme can be used by application programmers to set `flow_id` in the Network Flow QoS policy. +This is the most programmer-friendly option. The RMW indiscriminately generates unique flow IDs for all publishers/subscribers. Everything else is similar to Option-2. ### Advantages Our proposal has the following advantages: -- Easy to use: Application developers are only required to decide if unique flow identifiers for publishers/subscribers are necessary. If yes, they can easily generate the unique identifiers themselves, or obtain it from external components, or delegate unique identifier generation to the RMW implementation. -- Light-weight implementation: Both non-DDS and DDS RMW can implement the required support conveniently with negligible impact on performance. +- Easy to use: Application developers are only required to decide if unique flow identifiers for publishers/subscribers are necessary. If yes, they can easily generate the unique identifiers themselves, or obtain it from external components, or delegate unique identifier generation to the RMW implementation, or choose an RMW implementation that indiscriminately generates unique flow IDs for publishers/subscribers. +- Light-weight implementation: Both non-DDS and DDS RMW can implement the required support conveniently using native socket API with negligible impact on performance. - Network-agnostic: No particular network is preferred, respecting ROS2 design preferences. -- Minimum change: Introducing the *choice* of unique flow identifiers into the application layer is the minimum framework change required to obtain bespoke QoS from machine-machine network technologies such as 5G. +- Minimum change: Introducing the *choice* of unique flow identifiers into the application layer is the minimum framework change required to obtain bespoke QoS from QoS-centric machine-machine network technologies such as 5G. ### Limitations From ebe2cdef0d5e75ac5622a0e38b4f073924b6cd70 Mon Sep 17 00:00:00 2001 From: Ananya Muddukrishna Date: Thu, 12 Nov 2020 11:25:18 +0100 Subject: [PATCH 07/17] Fix typo --- articles/unique_network_flows.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/articles/unique_network_flows.md b/articles/unique_network_flows.md index 0f05e2021..69bbb5a7b 100644 --- a/articles/unique_network_flows.md +++ b/articles/unique_network_flows.md @@ -49,7 +49,7 @@ We briefly discuss two relevant explicit QoS specification methods for applicati A frustrating problem with DS-based QoS is that intermediate routers can reset or alter the DSCP value within flows. One workaround is to carefully configure intermediate routers such that they retain DSCP markings from incoming to outgoing flows. -- 5G network 5QI: The Network Exposure Function (NEF) [4] in the 5G core network provides robust and secure API for QoS specification. This API enables applications to programmatically (HTTP-JSON) specify required QoS by associating 5G Quality Indicators (5QIs) to flow identifers, as shown in the figure next. +- 5G network 5QI: The Network Exposure Function (NEF) [4] in the 5G core network provides robust and secure API for QoS specification. This API enables applications to programmatically (HTTP-JSON) specify required QoS by associating 5G QoS Identifiers (5QIs) to flow identifers, as shown in the figure next. ![ROS2 Application 5GS Network Programmability](./ros2-app-5gs-network-programmability.png) From 6638675627afe614357fc9829cde15474f42573c Mon Sep 17 00:00:00 2001 From: Ananya Muddukrishna Date: Wed, 2 Dec 2020 19:35:23 +0100 Subject: [PATCH 08/17] Change interface candidate to publisher/subscriber creation-time option Signed-off-by: Ananya Muddukrishna --- articles/unique_network_flows.md | 58 ++++++++++++++++++++------------ 1 file changed, 37 insertions(+), 21 deletions(-) diff --git a/articles/unique_network_flows.md b/articles/unique_network_flows.md index 69bbb5a7b..249efd397 100644 --- a/articles/unique_network_flows.md +++ b/articles/unique_network_flows.md @@ -95,42 +95,58 @@ We believe there are at least three architectural options to implement unique ne ### Option 1: Unique flow ID from publisher/subscriber -We construct a new QoS policy called *Network Flow* as a support structure to enable unique identification of flows. +We construct a publisher and subscriber creation-time option called `unique_network_flow` as a candidate structure to enable unique identification of flows. -The Network Flow QoS policy is parameterized by just one parameter -- a 32-bit unsigned integer called `flow_id`. The default `flow_id` is zero. +In one variant, `unique_network_flow` is a 32-bit unsigned integer whose default value is zero. -Each publisher or subscriber that requires a specific QoS from the network is required to set the `flow_id` parameter to value unique within the node. The `flow_id` can be computed internally in the node using simple UID generation schemes (atomic integer counters or random integer sampling without duplicates) or supplied by an external component. +Each publisher or subscriber that requires a specific QoS from the network is required to set `unique_network_flow` to a value unique within the node. This unique value can be computed internally in the node using simple UID generation schemes (atomic integer counters or random integer sampling without duplicates) or supplied by an external component. -The example C++ snippet below shows a node creating a publisher (`pub1`) and a subscriber (`sub1`) with unique `flow_id` values. Subscriber `sub2` is created with the default Network Flow QoS policy (`flow_id` = 0). +The example C++ snippet below shows a node creating three subscriptions. Subscriptions `sub_1_` and `sub_2_` are created using subscription options with node-unique `unique_network_flow` values. Subscription `sub_3_` is created with default subscription options (`unique_network_flow` = 0). ```cpp -auto pub1_qos = rclcpp::QoS(rclcpp::KeepLast(10)).network_flow(0xf4688c52); -pub1 = create_publisher("pub1_topic", pub1_qos); - -auto sub1_qos = rclcpp::QoS(rclcpp::KeepLast(1)).network_flow(0x88181c45); -sub1 = create_subscription("sub1_topic", sub1_qos, std::bind(&new_sub1_message, this, _1)); - -auto sub2_qos = rclcpp::QoS(rclcpp::KeepLast(1)); -sub2 = create_subscription("sub2_topic", sub2_qos, std::bind(&new_sub2_message, this, _1)); +// Enable unique network flow via subscription options +auto options_1 = rclcpp::SubscriptionOptions(); +options_1.unique_network_flow = 0xf4688c52; + +sub_1_ = this->create_subscription( + "topic_1", 10, std::bind( + &MyNode::topic_1_callback, this, + _1), options_1); + +// Enable unique network flow via subscription options +auto options_2 = rclcpp::SubscriptionOptions(); +options_2.unique_network_flow = 0x88181c45; + +sub_2_ = this->create_subscription( + "topic_2", 10, std::bind( + &MyNode::topic_2_callback, this, + _1), options_2); + +// Unique network flows are disabled by default +auto options_3 = rclcpp::SubscriptionOptions(); +sub_3_ = this->create_subscription( + "topic_3", 10, std::bind( + &MyNode::topic_3_callback, this, + _1), options_3); ``` -The RMW implementation has several options to convert `flow_id` to a unique flow identifier visible in packet headers. We list three candidate options. +The RMW implementation has several alternatives to convert the `unique_network_flow` argument to a unique flow identifier visible in packet headers. We list three candidate alternatives next. -If the node is communicating using IPv6, then the lower 20-bits of `flow_id` can be transferred to the Flow Label field. This creates a unique 3-tuple. +If the node is communicating using IPv6, then the lower 20-bits of `unique_network_flow` can be transferred to the Flow Label field. This creates a unique 3-tuple. -Else, if the node is communicating via IPv4, then there are two alternatives. One is copy the `flow_id` to the Options field. Another is to copy the lower 6-bits of the `flow_id` to the DSCP field. Both alternatives enable the network to infer a custom 6-tuple (traditional 5-tuple plus Options/DSCP) that uniquely identifies flows. +Else, if the node is communicating via IPv4, then there are two alternatives. One is copy the `unique_network_flow` argument to the Options field. Another is to copy the lower 6-bits of the `unique_network_flow` argument to the DSCP field. Both alternatives enable the network to infer a custom 6-tuple (traditional 5-tuple plus Options/DSCP) that uniquely identifies flows. Both DDS and non-DDS RMW implementations can trivially set fields in IP headers using native socket API on all ROS2 platforms (Linux, Windows, MacOS). -### Option 2: Publisher/subscriber delegates flow ID generation to RMW +### Option 2: Publisher/subscriber delegates unique flow ID generation to RMW -This is a programmer-friendly alternative that modifies the Network Flow QoS policy to accept a boolean parameter called `unique_flow`. Setting `unique_flow` to `true` instructs the RMW implementation to generate a unique flow identifier for the publisher/subscriber associated with the Network Flow QoS policy. By default, `unique_flow` is set to `false`. +This is a programmer-friendly alternative that modifies the `unique_network_flow` option to a boolean parameter. Setting it to `true` instructs the RMW implementation to generate a unique flow identifier for the associated publisher/subscriber. By default, `unique_network_flow` is set to `false`. -The RMW implementation generates a `flow_id` internally first before writing it to an appropriate field in packet headers similar to Option-1. +For those publishers/subscribers with `unique_network_flow` set to `true`, the RMW implementation generates an unique flow identifier internally first before writing it to an appropriate field in packet headers, similar to Option-1. -Generating the `flow_id` internally is a matter of implementing a simple hashing functions parameterized by existing unique identifiers for publishers/subscribers within a node (for example, the RTPS entity ID). +Generating the unique flow identifier internally is a matter of implementing a simple hashing functions parameterized by existing unique identifiers for publishers/subscribers within a node (for example, the RTPS entity ID). -A suitable interface should be created for publishers/subscribers to obtain the unique flow ID created by the RMW. We prefer to use community help to create this interface. +A suitable interface should be created for publishers/subscribers to obtain unique flow IDs created by the RMW. We prefer to use community help to create this interface. ### Option 3: RMW generates unique flow IDs for all publishers/subscribers @@ -155,7 +171,7 @@ We list a few alternative solutions to the problem that are limited and dissatis 1. Dedicated nodes: Publishers and subscribers that require special network QoS can be isolated to dedicated nodes. Such isolation indeed makes their 5-tuple flow identifier unique. However, this breaks the functionality-based node architecture of the application and degrades performance since nodes are heavy-weight structures. In the worst case, a dedicated node per publisher or subscriber is required. -2. Custom 6-tuple using side-loaded DDS Transport Priority QoS policies: Conceptually, the custom 6-tuple can be constructed by side-loading unique `flow_id` values into the Transport Priority QoS policy of the DDS RMW implementation. In practice, however, this is difficult to implement for several reasons. First, it expects DDS RMW side-loading competence from application programmers which is inconvenient. Second, re-purposing DSCP values as identifiers is limited to 64 identifiers and requires careful network administration as mentioned before. Third, side-loading support varies across DDS RMW implementations. To the best of our knowledge, none of the tier-1 DDS implementations for ROS2 today (Foxy) support side-loading Transport Priority QoS policies for *select few* publishers and subscribers in a node due to lack of fine-grained interfaces. A glaring limitation is that this alternative ignores non-DDS RMW. +2. Custom 6-tuple using side-loaded DDS Transport Priority QoS policies: Conceptually, the custom 6-tuple can be constructed by side-loading `unique_network_flow` values into the Transport Priority QoS policy of the DDS RMW implementation. In practice, however, this is difficult to implement for several reasons. First, it expects DDS RMW side-loading competence from application programmers which is inconvenient. Second, re-purposing DSCP values as identifiers is limited to 64 identifiers and requires careful network administration as mentioned before. Third, side-loading support varies across DDS RMW implementations. To the best of our knowledge, none of the tier-1 DDS implementations for ROS2 today (Foxy) support side-loading Transport Priority QoS policies for *select few* publishers and subscribers in a node due to lack of fine-grained interfaces. A glaring limitation is that this alternative ignores non-DDS RMW. 3. DS-based QoS using side-loaded DDS Transport Priority QoS policies: This gets ahead of the problem by directly specifying the required DS-based QoS through side-loaded Transport Priority QoS policies. However, this suffers from similar impracticalities as the previous alternative. It ignores non-DDS RMW, expects DS competence from programmers, and is not supported by tier-1 RMW implementations. From be914f3c76063bdc882904c0df5adc61a666473f Mon Sep 17 00:00:00 2001 From: Ananya Muddukrishna Date: Thu, 3 Dec 2020 16:41:46 +0100 Subject: [PATCH 09/17] Add transport protocol port-setting alternative Signed-off-by: Ananya Muddukrishna --- articles/unique_network_flows.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/articles/unique_network_flows.md b/articles/unique_network_flows.md index 249efd397..a398ff95f 100644 --- a/articles/unique_network_flows.md +++ b/articles/unique_network_flows.md @@ -130,13 +130,16 @@ sub_3_ = this->create_subscription( _1), options_3); ``` -The RMW implementation has several alternatives to convert the `unique_network_flow` argument to a unique flow identifier visible in packet headers. We list three candidate alternatives next. +The RMW implementation has several alternatives to convert the `unique_network_flow` argument to a unique flow identifier visible in packet headers. We list few candidate alternatives next. If the node is communicating using IPv6, then the lower 20-bits of `unique_network_flow` can be transferred to the Flow Label field. This creates a unique 3-tuple. -Else, if the node is communicating via IPv4, then there are two alternatives. One is copy the `unique_network_flow` argument to the Options field. Another is to copy the lower 6-bits of the `unique_network_flow` argument to the DSCP field. Both alternatives enable the network to infer a custom 6-tuple (traditional 5-tuple plus Options/DSCP) that uniquely identifies flows. +Else, if the node is communicating via IPv4, then there are two alternatives. One is to copy the `unique_network_flow` argument to the Options field. Another is to copy the lower 6-bits of the `unique_network_flow` argument to the DSCP field. Both alternatives enable the network to infer a custom 6-tuple (traditional 5-tuple plus Options/DSCP) that uniquely identifies flows. + +Yet another alternative is to use the `unique_network_flow` argument as a key that maps to unique identifiers maintained by the RMW implementation. These mapped unique identifiers are written to appropriate locations in packet headers, including transport protocol port fields. + +Both DDS and non-DDS RMW implementations can trivially set fields in IP or transport protocol headers using native socket API on all ROS2 platforms (Linux, Windows, MacOS). -Both DDS and non-DDS RMW implementations can trivially set fields in IP headers using native socket API on all ROS2 platforms (Linux, Windows, MacOS). ### Option 2: Publisher/subscriber delegates unique flow ID generation to RMW From 1a5d4f3b95c9511daef06cdc2bc3cacadc0a3272 Mon Sep 17 00:00:00 2001 From: Ananya Muddukrishna Date: Tue, 8 Dec 2020 18:12:35 +0100 Subject: [PATCH 10/17] Update figure to remove header specificity Signed-off-by: Ananya Muddukrishna --- .../unique-network-flows-arch-options.png | Bin 24111 -> 24960 bytes 1 file changed, 0 insertions(+), 0 deletions(-) diff --git a/articles/unique-network-flows-arch-options.png b/articles/unique-network-flows-arch-options.png index fa0b95dbe9a848fbd2d574fc6b38200f4444f57f..067e5c7d46f731db28b347cccf9d78096a451a25 100755 GIT binary patch literal 24960 zcmd42byStx*EVbef^esiT{_gFVa3 zR}L6#{6ydiT{apzZvPy=cOP8F3)-!&q`bn0-K^gi;MH?_Z#I}BQ?@v6)Uhrf1IH%t zs}XdSy2k1EQmn-7^$Zeo$gTxiQHxHc@T-M^^NSaRh3_2riXZz{Ad1%eg-DTeD@MNA zu%gi`w0=1Ao3Bx`qI2b;8l#PVz1^TL{@3G@wPKSe_R8wssTYqJs0pndhCVKQzRYzf zMecUdxUV~N1+m}>tJ-tQ5RpJ3NR?dX7V2PS_nN_K<)|`!5X|I^j~?r3akQ4q5H?N5 ztZEa7Q(f)#BHI_^kH>v+Rg$7uvj9&D^!@o!RQXjM^|zOaP0t?}ve2-N3Z5tCoTzqi z7bkVUJttw?*4Fc-5g*b$Bti#KYYZ10ZrWs39iVL-Q{=z(vVV=8dm}h? zyLcY0-$YNV0145}L!K7Ux}9G3=8AmrYk6V`Lt&~VBnm^yREVT|g`H~5{xJy~)!Uyg z_x1EN$&vvnc1PS_$IZm%aOrL#Q@_ef)5NzKl8eA~VK`s5ZGj2Al8lgkkrcx^v z73#FkvP@2mRh+-}Z34T}JS&FW=IaIr1u}!Jg5A`E$8uk?wFG$+2rixtY{_N$^R7GJ zyJsvRCnc`oHM7%lU&Bx{_189Iehn-f5^~6MKK;qRe6cCM!!WDV>_d(UBrW6*=lbqa;)05Y0}( zQ87j!I9hSYg&bQ(0@Rk|ZZqfDhr`6td`t|jh=TPgJ0k=3Jd2koL zb_Fu{R|N`wp+{no2giM+gxnrqi27aj>2vPvW!&7r-nD3%jlnp7QUohbeUI*}M>d42 zwAB_pwVYYph_AE!O-+)9o6(C)Zr=F95&PS`M)&5cRR60@MhDL2y;uH+FC*0zb)mTA zRF{;zwkwW>u;9j3G$g4^O-dH&nqVqj9*vd$F2l*uqDXBagk3SGiEi*E!+4T_k_ zrwXNmGpJSDw-jV$~ZMg07?PdD3c|GW$x@DI>@@y)<| zXz=DK7Th)u=*rt)%~H73oVsOob0~`y(=K zDzoueAy-$w1jliIOOxmY$H~5%GfDTDi%OKgy_|aMB>Db5ql9`mYnza&#_=HSr}M-g zBEHh{zSk$hB8zqN#|4Fjs|~+yNX-*pW}2U8^GWYuYO4oF6Yd$;?UK~s2^`1X@|kdN zm((rL$vFJ1JpFO&7d2Dk|C*X9h9Gn%s3nc}RuJ4ZJTg%xFP#rY6+lnlmNRd$^IS1* zuW64Z+(WK+ujOJLxstf@c!gBdnsMi+Rlxc5OYSYP@0ynDQy*9i-^HfX&NiF-T-Ic! zrr2QTPW$EIWFr~F-C$jY!zO9wmfQZ;AyxZMG z<(tKn6og{UcJ(nFh%|KHiu-c$+VISwZobf?MCTyA?II20uHkWdg894iX*n5Y6z(A2 z+nX`bmXGh>F>ytdAE;(P@#?H9z&97*_{)dTS>^9}QA$;O-Qh{v0%1Ja0TejAcGbCUk zc(wh{gs%c_E(ECjZ;pteEa;ePvp!mR=_$w*VqlIYtyZ3dlsjsB{%Q_Sblux7+Y5AM zV$_duxj9z8m9&Nj&LHew{BW+49s043@A;1#`)nbBARF{~J>Q$0R@AT>9hUDhQN7rcksf8~&(ufgr z4lPnf^1IT|rOaZ(qp5)F1iilZiAQBr*B~Ck3F=DF6?5wBb&G`YxQv_APJ3lk{u+Y{ zc31K3S{oPt*@ANq$0OyC5bvREr|)gob&UP}w>9p>k3>FvxDlCB#sq7wo{yR~;Dq%C zcKf~ub6;k%^x&fP`lnTynBFHZnDY3`Cti&(*zLnhs(y;{v;AeV1m)Xf;mD&B@08eKXunlCJ9pi3x}LaZk6OBUq!rzIY#1t}4_M@-CDm$@!L& z-jw5ptY8xmxMpf@<*FdUNBR?%BFSWY!#rz*k9y78zi%&NN-5G;EKyQeUgD7{4U&;k zLC4ia)<=(sF^b@UxOfGSrc|NU|H@574e(6V9f5n7rLq{m@L_b`eeD1<4}Jkv=*Cle z2t4I>MaRYWbNyjqWkpL%yXZ0NU@Dqib#`yaeI=TAcd<>>PBqFyw)CZqk~eDr-PR_mZxkSaD9TW@iozJg>bQ5#-59GVs=(v zLt}N$AJ&CbW5#An$!YHFVkcc3p>AnO(-&|y?KEs~tA4;Jc|w09x1N-|Qp)D*PRj2J8FOYN z>We-;YdveJ-{571VQ)s6UL39o3k$QmJXQl$&zTj%JLmKe^u8;;H_`> z^7dqj*X-g zh1X~E+9016>sGPRI(?(4^_gLK_r%C`keg3y@aL<>O`2o;!az4eLt>7)>T0sI{;h7jtC?5$KTA& zr8y`;6FL8U)~7S(6a!0DhVqehlxsxR?YVf>PombNV496mNvW0#Xa zVHqoQ7=Cu?cl&2irm>;|Lj)1Y%g(zY8P#a&z`oj0)yj4zd5YQ=EnLOT{HIGZpZB`1WI3Lcp_Mjd~r zGU`GaOuEXa#dK0!*nU{&D2z7~6%89<(|r+`Zwi6LxM9<*O#6ey{JBf6X2dT%^^IMT zc}==}lx}=G&W6X+y;`k_{RC3qcrkRO$L|>mTEbZ!o_TV{l`N~!)cgMD!Dq8bUBkTW zAyi45OC4`D9z?36!>Go%xD6B1T#kPIm@?Yc^Ca?cejs|@fo5KIG#S^xalgiD3x>T= zt%!>KM0rP1KikMIEL;hrWx4Hmbxc{*fxGBz)<@n@=rmLReNg2HwgWMnXco~&$v4}j zO%N$+orgbdQGYQ?U}`MS*dj1>^fN!LLp_)#V!-OQx$NEyL?CG=7xdgsigm%0W)6>2 zIQ@zyime@z3FAR&p6fZHgP_X}?Dh5avHg_U=MoYkvbQKsRKoG{K6}TIdW?U~M@c>) zCVII+nHTH&fxeB{EDggK`{9Hbz4~B~MLeTLZTP(rZcxE4PGNHY;sjUJF*6S61Z4)xHtCj2X zg8IE1iw+x9frOQto4fZR$s@VSmZm2C%6ac!>E~WkhMv7W3kycEAz!^sUG61E>OH20 zIBGssfQe<6}J;g_QGyw{PDvYL_a0e1zmzZMch=$M(ml)vsYLUR3mg zThZL(59+!Gt`w1=2TDX|r$jHMvOHViv-j)+F&KTHQz1p>e~I!g(b}7OWnl|K~<8?Tw#o4^+`fnUGzq+%owc33XzLXtF{p{ zIN^N^Hf?8VHU~zey6pFsR{VyImxUZix~3k)Nd2xfAW-E3o=YHpWI2aMAX8BIg3shg zX#b;JcYowlx-K9FHJaR%G=8kv8Mf(C0tr84YsPKUll6K-#B$4@tT zzIyrcgl%ylRc-COdH0*Ei-ZKs@Qz3v#<)-AbLOtL5i_NyF)=Zh zej-Fh1_n&L5Am4w+;->qsaCo9A3l62OBtkF@096TzzLm?tl}S@zc~7}^2K0788L#G z4&w6!0l^jgxVT*T2WV?gftQ%+F)=yW*HZTS&#CFF@_2P6rNQ0#CR!R&nG(Y$+=!Fy zDN0-MtcC`DK|y_3%H*V$eMTgow-qQca&mIk96b*K4#Y+v`($lv%fQGe`I++aJ_iQ} zmGkCkPH}OuNjJT)u;HhJrCTV!Yuv`~-{6>3R&%r8(fT0b_qfvhe5acrm*B3=@9~s; zSHqc-L}&mystY+EuF6TN%0DS)wS1-pE#_YnA6e^9=H%qe)U2}l@j`%`8|OAtzmka3 z-1*Q;AMF=26H_n;vu>qS$vZM|fwAhsn0|G^jo?A`yqp|4iB96!z$kS{(pLEM-WM4^ z!q7{qtFdjQAo_KV@2h2;Q~QF#FE4#`5e|kW(Hev;qc*)C;z^+=gg;DF=snpcwzIWe zs{8@v=VbT(c-D7L+Tr2hxU{sW z+brClKYtE0#KOks=c7%KLCbXWv&t%&#gKvH-6gA0bBaPQ5_tt18$INbgov0}`gf2u zJ@}!dd^wh@w6d~NX=iVb((!DZj+OP5mzNHYmWIZcFJHXkv62i74BmakU@j=zMSb>+ z{4J9lK0dzK<#>Vm`e^4N^_*srmM$qbv5*(K{$HrlknnFzJ5u9}f(}qDI}5}QO0oym z4b^J4y&=LeJd&Oue07Am&xIBCG0$$!ZU%i1^f(m9b(1 z6yEjOE{D&%vwVSFR5ZZLi|@Ws5AQ)+D>8fE1A_f88>=}A$uBy=BE@$r!Oo*USvpwh zc`cIhW#!u4kH;aL+c`Dm%0}Fc4?F>A9O%W+F0cK)w?~7+rVeZjMKPI~nfZnXk#d3>C`H&YP&E#9t_Q|ZQ$Rq{pX1{Z+mtM3~g--Pu>%( zPaQNwB>IpiF28>p?mGbc^13dw^D0ZCF5wx@*lrE?iC1r*I-vg{C?c|}3PtCVy_n3G(2qF<6NJLb@Gcb$=5shtpE`Ay^7{JH zbR6Yfa(WV>E-oG@WO{fgAfo&HjSwM^{XQLJxFIG6m8v+*x__i~b~tibI%G0oq6wXn z95Ta8cX@rYGh3&u$l}SxYh`Usw`v9NTgFf@Y567%n>>`fe2Fudo{4o0&&+J8LdkH} z*498whxd-ZaNwWuB-CSX&R4-PdyQvC)g|}miL`DC@2iF8(^C)de)ROJi%vl5bVXGK zAqpzS90!!lbZHFVu#@>vNKu2aaT%!hUHoFN4Poz~= zgHk+$3Axpzxuf0;6_zd_#KAy}o1wE%$C{v>9v}1VElEFc|LR~F1o8Oe zAgm*EgZky>#+a$~S9|fI=My!My1@e|KxLV9dQN|#d;#4d9VCJtauwCocDmT8glUCJ zlndHNXL&Ja*vj#FBy^_DN55xIbfeG6dn>|<$|dVDQiOOtS? z_e`v|hATS{)VoHstD28rk{aHP!lZI_q4*5of?0g=(?OY-pyelE`0q=UMJnbMcn(Y# z(BW>({4eX;ndZNr^uMm_181M+70V8gyTA(tt=#JB>fGF%!))!xegZ*;L)H)ji~HQaH#gaTof)~{GZK(%B!^Zg?zww#x{Vjw*7&QbsoC0YZA}z` z?oiTt9^^S^CV5!}g`&d3=L7^u{W^&KkJ}$(sD~iOuMqPhn-$moUyABJtK(&o7wk); zn_gG%;IL{@VM#4rb9VOE;((dSbkTs;#>UIz%`w>(fxVp_!z#;XJdoE+!D|ne)LcLc zDQF)onWl*a*?=5Hw&b6h3o&tyl)Aup%1$|sGh~L5@mT*E;=o9@j00iat3{HA3NiTf zv*|b;z`EVt-DeC_3_m0#nRJ!vR7hB+Zt+CT?%-{Q6B84kT=?-d*VpUpVleAfZ_G8g z%A%m5QO>rvznOFStNf1nJhJmw^#AoEW6uK(onU5mcG<0%0LuJ0-^kxzgoh$@y2>in z7CSZZ!-rbWJtKFuo#p8!*Iz^69Ypjt8BM1Fgp!=>x%wT$ss#GlVq*)`XY!!nWRIjlaGOHxG~FeB(rR=HN-4u5r!GPrBZ6yQxxs*Y);C>FMcVsKkA} zz3lk?mlz#bJ_`-~w%2D1zI&7Uueu2|e=&iY8AQ&gkj&zq+RKL9oB^n6XNu*DrKKgZ z7c@k{i+g)pTRwnJ7%!cf(W67a+z*{5v|rUwSFiFtb98n9HgM=`U_=oKn(=w+LtyZ)+64f0aZNUY#Glzx`?_V8XWU zMaP^9@_0#kIXcF>pE_C=ofcJ&^8!_RpP9cSv48tna}alL&_iqbuQ6de0MNa_u8*$% z>U^%UlH;BE2wGfxJUJ4|{2fX`Qh(`<7G~QR=gormIBcFQxV2eyl zrDSIkrup)1YDk3}Xn3R{(DNL?4Ar6|FO7>x8MlH{O zQfWN(y`H}`p3VRKZfzuHh%>9A78#}SQ*tuiK6Y)z(t+DZ7)%NitDbOnKy`kSS9#!J zBW;dA){9CE5!5lYfv)%$tJUt?T5`VFd2h}FB(P{56p)`jRcR?LEd@M-*qW3Hxv~XY z2rucbkl^L=N2^c0Lqi^xlc(?XuQ944!ZeY$sx>Eu)+}E6`ue)N*PQ%TFK(;7eaOSW z1Pk^r|8yP2+8W1M=(JzaLn7#ZRez%6H&)y9d=^}kr|G{LkAV9y4i5Ce8H0|y>KGHuY z^>7}o#r`Gm`jQJ8e{z61S~+aD@+}%Zft{;{g*!B*Gp#N~Z9*(FExu==&j?X8rAxZ# z&rVK$e&P?jy1G)&gLuQ=uH7F|lk8jyi0y%ZK!=aV^y3K7NEC)yRVMiX>?T4Hc&dkI z?L~mRpe*A|K_&DqCWgYJ1BIQQ9;vELDUGM2ryz|V(UFbm=+6=Yoq$$ZCqHK+briV%hP9%Ip3yDX34=<|{xm16ByhZ+Nkik*}Hxg1wx%ZmKVXrZJAY@br zPb>j-4LG{G9{k0Yt?jx*FTh2^o3~>MAJ93H2Yb=NVAU^sVg$`FzyBeo`ULQ|+)^OY zPIxijfwRCJ8i(SZN1cJ&nW4fpzdMl1rJeZcHd^&{HM|9SqRX=$KlcS$P`%Cd2_NPCmv6dDVK6`{P%WPr`df?bf3q z!OV%Kva(lzp{1jure6Exy9?)KAJnSL%FB1E0#n zp-g6}3)xslr=Lxsgd#gZ!k(&r<;1O9pcTrGLAujby>YE8!KOuFu1x z*Rw(Q9u*mBPw32KzIifbEb4d8R_MIfa^};i3cU! z#5#2+OM)o&7Gy3LEX~?!+o+$yJ^=4~D~n!NpD?Y5rX77&)L>#{EGjJ>O1(|@TStzkJ2O@@vP}>sDL5;#mVLEC`3|P0iF z>(7ab90}*##+2&uS8)@MbrckkN82tZ%F2m}*SJ2``VP{#dIa^@%|=Z;s2x#xR_E`^ z*uwv7IMvcGO>F9*VAKxWYGTkiQh8mhC1*20%Ard9P!1yi zeC(!JsDNm(_ZUhq_DgD?>^Dg_(z^HYh)jIzuTQ?sD0&9*_)latG&Knhzq*ni-2B;V zyWA?0_1o58RP!bES_;IXh3Hs#?ab70G@9p&9)72s9Q|y{ftYElCMS7Eec#uv(vU2) zV%D1IPyjS?3{IUSiT}B&pS`UuX2#*tTQr|Go?;uxG5e3~wG)rfZ)-Az^dfiQ^zN`R zj+tXU|Bsfl1x5?`C+~057O!Zybm(&$^GM4TOCnmLumlWHgZ16k#ltA)=jNE^aa5pl zdOJBHi?=st7E&*t2S0kj$dmQx3wyE*oZr`A!L0IfxNw}eovGgdn}th+bSTF4;|}aJLRw*`4f7X&vi->D2G*w z$<@(_mnbOrwXu@;!#5dhMl6DQoLPc3N`?q>QGh^CZVOTXzH(3d93Nzz_+ z@1WZU2ESyr3_`|bYM0rY3gvdNPGycZC0nFUPRr+bRWqUGgZ51PadAirq2th7I;q;1 zK^9G&7!OS&Bk3CqN=f?1Q>UBwUSA#b)E14W%1mfoihky6sF_iCy}A%Dkj&@fJc*d9 zN@kmqm`FKwFG$*66l8@Gr@CG4w1J;U3uQZRw2-G?NOI3z4wewrnIN}Y%{U4HT2;(4 z_`EIP*5B?T1xCi4K=bm^{p$Qm1N(S&nl1-OL&Rsk-c`t8wuSTetBa$W&~a4ES+n~U zGVRnjmLW$R({o$dqv0;*Lf?fAFmUQdrM{H-6qHagQ5n(Xch*dx>ICV_c*8XlXNWp9 z?B2kNdG?VNV))Z|3~`-9%;eGwF=01@dY(@yI&EYaq^FBm54gs zNj_9}wBFx-+0tfKQ{#mu^@`)CHbk8tz0TXIoCeu6-=`uqna9T)^;Zs&!wrB?QV)g+ zPfVmeBFZDSrgM$Y76U{_IcyRS=y=hnyfn`soxZ$3a5<&rpm3olqGm07E=$!@^^3a5 z)xP8=6ABye+xx*kHmnEAEH=Mubjca^%;;vOzBgG-V6(h4PDe`lwEw6!)<{sI!3t2^eo z__}U@Zeoza!6DuGZE|SST0HF!I+ah&g;({3)NP6rZyWugG^hUQz7fGHt<31ojAJFC zMgy7lv=)UX(sL9dSy3ujij-2N)r^I6jZ;y*L4$iEwnj7f#e5C#S3Vd^8BX%)c+P{o z1Gs{6F<-)A2We6wFR%BTcx~q)oR*PWYAyqA5T&5jau#ZBWotGBsjSAv86Wj-k35N8 zdO=^>U}}$iG>;aLA`aWXdN`A5;31o0n%2Mhz^qRb#bj~OHa0LLcwW|Wn#wIuE6go! zP4~C15G0h+VIW?(?8lb<7xfOG2T=Uaz0UVdz47gQhkiBl4E%tiC{-D0Ok(6)LbS|p zrbkC9x}R9c)CpM!8MG&qH>3fO((8ch5K?1dtMvO?fHyBEVdFk~99$#OFKXEc6f%-H zTH}9uRp7pv*cct~k1k@h>^Q&OZQikdX3fp&a7+Q2?mo@NCXC z)(7R)V5~iZd6I8R3Rz*tgQhU;k1j4Q)qU7Gtjk$7DOtISLBYrIHCWEVy}aG4s?pZm z9B?6f=wrO=jyxhPnBaaXv!VdHfMwvhq}=|Qr3a-v0|IMN1BK7;clZ7zH%eVG=!FSMwJHQ|$&q+N z$sV@v%&oG6dUg?M6pV*ekRxU1HGp$;W;Ez3~A-iH?CG=)%0Cw9)wZ>gY$d z@A4b_DAr<}$qxnG6=&OX&<`h(8}~JSoVE}+d~_^1rGS(Z(}-2k1g`6%#h1MhzBymn zDs9!vwfkD)+mgbhgXLQdUqa3nlN`@pudQEFd0_@>H?3M0FfmrG_25|O8&e`1Oy~tV z%@0cMWfBg$a5{(@b78eoL(Gn5;>y4B+>f-}UNW#o_ShTf+TjOAivjDf@CI56jSCyx z9pM(lKsxi;S_i%+AqiuUSRFcc(<}N`J$PBQy5x3GM)assr$Xm0n>KUPRxMS>y$-tZ z0MxUDKF6VBVVPWjKy7MG&W5y*89B{^tu1Hh4S<&u9PR5(n%1Ixe> z@wK<-meYY?>dt-v5_^rIW_v$f>EDzyM$CP<4sz);d}$8URMBr&Bx~ zNIm!93PKMI8z~_%(bvbP)Sv-#O}!_CMiq*<+^z`B2fns=$gnR%vd&LC07d$dpAE~8 zzCj$iYRC)Bp;N}!76O((JKvG~as8`WXP1g6QH9VxuYiMZ5GWJ6GU2OKgxie+xmzgZlRL^l0iy zkg{UDJ3&_^R;B#8adeTInkuiL@DCvn#_r;h`_}e*wN4y52PcAM5G(cG-d@=A*ZUm^ zvbQ^nH_VSd*VRo$@d+OfO9GrHYd>8s?w}FXSwd)MX9r}KqaW_pc8p9+Ag4@BOn_G^ z{vfCKzf=`?2M>rQ$s!6WNG7+r9+NC$g!q)a^HWK*+PbZd!xnc_p$!KNqR>V14ooPif$h7?ZVrmoAb5D zo@l(kC;ldpk?0^8uWgeP5<2bNBB$!E++VjGC-+ZYF*!7F#=(LR;iPmp8K(qv_0_F* z8x~s}@Lf4!{;DOph3)R;lwD9cueY0`N=qP8(sEp@hdr$N;AvTU{z}v@HU7>h3V35w z1Hqum1D0kxWS6^K3o`*r4aw0EC1ntyij1A2H`|- zLFvwcb+@Xere5TlbO5;(kjq6ha!N|VE~~QRwt%o)7v|jhn2?~Rs~h11xgIZ!D`NUg z>f_@h!B`IXxVk#+*FrwW8-PuoPer%a4Egef3Lmi>$eunnsnKAy!gP}2?r1k|4vwrr zj{}%=+hRt1eBi?Ozl7eNPgHQ18dBQh<*4P7B$?gczoEju9svOXZnp0~e%utUqU!-& zZ6^!Fu>}Jgw1^9^?kH8^i+aBcm+(M+EQun9_6iW1mfmBhKY)T-R8*v#ArgHIG|0?s zyUF5?7gmrBEzn^ck_n55wAx?-#u;eVvKq{Yo)MeP?#RbB%bg)31)3`UN{qD2IJ6_Q z$8y#dj&t=Dt*s;xfJov2A=n=)7%Op(V_zm=fwdS8oCt5~JVo+#+!34n$ryd85cfPvuT<;B|!3h z^ks(JUU*p7Ro|0f>*4m7oE&~nUHxFTT|gE~FK<0rN}yJt!mq3Zx$9Q}#w}Q!kDK)d z@+d2SO5c^iM-LodYYXG8um4>6w+TENbE=oMi(CFsRZ_Aunxg0U3TsJ^N6l0B~12nMFn0(Yy~y zrA*AyzGN8cN8edpS65GZC$gk_JwGr8hCp3i0#xi4v@JKV zs$|b8X?Ns#9zC)|vlv3U)zsETSQA6c!Fl3ShH@{rrSjSy`9pA)kaI=oqHK z_|%h@g2KIe%E>xw`38NeAWB;g1-jnw146deaeiiE;-mwC)cQKL_ED9Ub|IxL72#_U z(%h?@oE%w7mrgnA9aU&7PbjG0zZxIGY|@QnLMnnhMa~iT)iDi zcM8-RCtLqEV}~%5d_j%705CLUwqsNYI9bij%p?bad@IddSyh#(Nz&ardG(nOZ0 zQ}Hs)$L!a&8`Y^HBZ$$)$JAQG{>V5($Q=kg>p+-n%f-oi!;it)VN7WT;!bOpk%3|S zB^vzg5D*hHv#@+45C4#qLHzu=#1q{FHa505#|7$n134jU3iU-=C6yw!0$%%Mwe{&5 z`bX$pa`s_5c{Mc?GZgkedV4`tZ*W=;>J#GNPy?nnj$Ys=r0sv6o+mbsZbzXfFE1}a zRc$jyqew5oM+`U4!fmSvG~E$ScM5&QJv^zdzkH@3w}R4)iS$2c!c>xTKH-G}>yCwm zh0#$p*1f?aNRn}Z$@SLz2Gsb{luW`uudi?~xhlb{b0F zTDoLtVpihBHpAV$VO{Jdu=9fzkqqV0`-^FKf2;#~DPDBhm>u%oa}f2A;R9v`Y}%On z&%k>$;^4yAXz|9{6;;{sL(4qj^+gZ1FgFLzm!6%lUTRbSgU=VyL0ciglZ&V_SuHJ~ z^EcFIcc_62>BFsTD*nlI%*@ZS933Hcc8mcLF>qO)=wTr9oiI-z5W6>6(t%#_{ zU`3Yf$?u;p2&E-^wuTK~gbH^FTRL;Jc@1B^5dSYxUi?pI`@n$WMfwRc9!pBv%h$64 zcP3z1T|HidrV`G2E zq5&_^)prjn{_wLa92n`KjeNk`?|vJ~`_y*P93KgO>hRv#tHsvWW-blc4g8Xll78z_ zAUYmBiE#&(9ISadpECK<57T|Bj+5+6@D~4 z(aeODoS+Wndm^M;o|o6-jlhbTPw2>r<;D@99}9n(!GGJc&0J6v_ehn*fEWDANnEO# zvjrEpt1<21_8KHWI%B#51|Cj61AuX7yZY-BmoWUxUms%tNCCQvkp?!8J$nyEsqML%Me?rK?!m3!;Ye<52Jsna3V*K}aAuuVkAXnk*t7UDt zKHp`w$~9MBw_T@AQr{SUDSju}n?(T+!uE7w^iFSGP2Lp=+`OVr6a^f)U4jr~frs>` znB<&2H7@l05$?$HWYSnz9N7l&G!*uj@)8*Xj<&iZ{u^N8Vup|Sj>rG_1}-B|#ee?G zqz>og;eImU3)l>d9b37R1K^%^_^Q71t^?Ek%Qrwe0DkpMy?uaZ{M$q+DJl8+`FVLg zK|8qa^6lHV>RCVvgRlTrE$92nEwy*RD-|!;zAI95%B3Gvl_qf@$>T= zqY8E5;N+~CmF9rB-1+|?>7%ojzE|j#Q>^kjfHgJ>?63^5D9FpR znQ0g6kaH@zr*-d~M!9$W%Tae0j{CQxE;KAG-oeB~%&~#VpnkY%+(tNvTa=~^G@YJJ z2;nHeY~&;`i8A!zfd;FnC@CR^B*=U++)oGP6clEgJj(}RmBZ}>ASHD1PKA75gPFoC8c!FFR$mA_uT%m80!xaTLm&ea z#n^9saU=#^vB%~p^~Z!L%j472NYd)%-;9z9070wXU+wW8)C}2{fkUeACd^oe`@nDK z&HI-$UNcjo%EmkXz#VjcQjcEpe9-*p^s`vMZm1a78renSowJ975S3U}Iu6gf$9s9% zAZtL_PC|b$Gctaa#l5}V1#Ak{t$+K&PsyCY2j_DVr10nePlT>77s+# zH)kl>d<8a@`VcE%N~l?*t>N-%%f1-Cv4HA{4jP+)l{NKc^<%7}0r}^n|*cq)52w>Bu-Se+gf~?Aytw zgx`YB#z0YzKm9PeE~ubi2Z)#WZ@+PFmusOcuNYvf?7xEh zQ^h9K&eK7p=F@kgz8~A#90=1_4$fMAg7;j6i_5(9y?}K4Wf63yurPHR)wsonyHYM6 z`}DHuq^(v}oS;9OINeI>r2`(moA4b(w13&LvPlO~J=29tnT0E6;{X2?^qp7l(r@)+ z;I#z753nPpr@P1R>C8{};QrLp+S*!RgXWghsM8wQMcP#+(GXJ|rwj`X4V{}axRPlE zZPh1Ky|s~_?w+2M)19~FQHixVd{diadCTvLW0T`PgcuR>I@ahg3f2VUn1xDjE^ zc?6hUdF`i7@)>NYAq{0^KTZdbR)77<5OD9y&U`p&JbQbekQ>KYEoy26X|n1oi!70_ zkA|>NV$SNJ|~(zfmdLPBo4PA+mTn0+`;2!`z{5emJVGkg$G9kZSWkO%$O_OQPw zN96x#=Ks&NHg%HZH}J^;3EMVde0&`Ah$&GZdR-ZT6q*BUMog?(T5#js2{6tr#m2-B$E9-TlZt{4B?-u@P8vU;T!%WoJaY5wnrcSOaQ*B z2w*4!4DVw`2xzP^gO}m)#j<%8mpDcaZ_Z3#RcMU)b7mTeOITFN+~z$p?~;rn6eWx>zK574mUb zcV5-x0MGYSLqvQ7nX-tct5UhHTl=F^H~a4c3};))VSuT;JxOdL;Kz`_)PjOmlBKHv zkeIErq#G$m2xYI&L}Q2q4$}JDa)FuE0`vzcyRpftOi}7!Z2?Y^2Z0qhSChU^)p2S7 zyDJ=se|P&goftelB>2C@$F1kte}F=R19Kn zLJ@Eu3Y?{I_QYTwl)qZ-D{?Cy*9FY($8T zDS%ti6^q8>VK~Fe60ks0>Whs6_$V?mk_VjgQKV}{CUwu~YmrvmQ z+|MpbW6|}*wZTB;W>Ua&=VLZ2LEn%Du2CXhD6Z*=cW9crcm&4v7qRCjP-M!ncn4qzb7 zczvqt*Y&Nyz#Hs$;b=KA3MUGR>u=m{+%8Qd54@Evx)Qw{iH*~#b0i*(j*ga9V44^n ze#5*yRVH&8LHjA?qw~OVEu8?2>Ur$lwo#kkN1K>=8@wL>Y#U8ihgwGR%{zU4wzW%e z4mMjJU=Wr}bs)1gJMGj=vFO(-QZQ_9c{zMUfAafs*RyT=b#nw?fxc^H5n!F`>+wnN zoHqcSGH@i)$v@iMZQA>W3;0tv8JVyhw-i1~^WMI`!lI(8PnTf(Tvs3;d~@jMK3f-N zZvjEUOurN#h0nNbqC@^*N7d2_h=aFd21s;@kEKmuhgg>bu>moaSGJ{qZud&=PL%ZcVcxoM6PU6gg^-RPjZlD*y1 zZtL`LOX~SX;gujK*G}SUR63-`KWHUwCm##@LM(ni`g~knIn(C;ksTq6FZ(=H(39n9 zye}F_nZ83{hTkDlBuB>3OUY_#liyz{Hb@7Q9J(1?y*9Jr2=S5FQ0O7*IJDEfJo!)z zMX(d7)Uc@-OY<$qV1}si>a(dxPiLRY%{=Mo;wJz7x1^tG>OBr_K@@9VIREU3?=6Kp zvXtHwb>e!N|L*qqlJ0m6jK#$HfQ;pyidHZNJioQFX);6;-!`fY_+(Y8;vR4&DE82E)WlFM=nnO zYftL)Oy(;@%yKfgVFlw4#p_?`o`v*}vy2=rp4?Z#vsDq!EqOoYr`4P?>KC6T6UX^% zF;YY(tFW*y&Un`8=b!e6Xiz@qPJxkQmw8*{;GucfAHwTn-uB_Oj}$S!n5miCRrC=V zHPWp<2_+u*ME)zB8%z);?fJnQSa3li*m$>y0ya**1I^Ex+k?8mwP2+fK`#Z7 z0N<^8v?;_RrT;+jO?3YIszv*e*oG(xM-!?fq}yyt=7?r8iCj$ETDol*`n7c#Qc0Vu z@A?;PmZg>zK=GP~F61{wRVuKEs1h5z?fmf`e7mdu%a-w;=SljkBn_+REpkr9Gz43P zNB;WCU#Rm*T#Mc>C!Cte$08F{=W%`+yr_AW(dS;oj9#jB<)xB0#zSR-m)? zDK`l`<@r1tDN^0(EugCw82a6nTQlj(C(Hq6iH~M<*gr4bOVfg(p_;PR0=BKj>1=lm z7gUmDlUh*)A37{&>wk3QJ|0!&pwwcIOzrLCdO$iNo!>j5ZOrVFV;H)kRWCIP1Y0AY z0fiD2*lKr@rL#*6Srvxn$&%dQFBcxa$f=SNv2$JT-EEIX6G9l1xGoi)v&#n}y@y}_ zK%cpn+WG544SGK68y z1G_xgwzzb^y*?ux%5gZ?DAl(L^l!WErOj}gCA0@=2>`Hx*c`JV8Qqx|{Qsx5GY^OA z@85XKP$K(QL_{b{*>@#EWX+acB4ingP$bHlCCN6jB>UKsT}mTsWC@KeQ*9eeprrC$_8#J*Q!eSqhd@PlbQc}0hN;LvMe z9XnlRXkvwq9I4kZHb~FQWdQzD*URP<&=pG_&$HSW^x(~3oeZ?G^h($ia(giO7AW|t{|oDdvY45=w8q&_A} z*f=d@y&*W(q2(JyYH!>|cZt$99=D5&P;oz7gzS^)DcT~dNYww?L$I$eZ zRgDzL#g&cfRXQh=SZma-Ee+i9rOf5P(^}G8$-nMpq0gTxNjx8~R`&!atS%^rEI(#_ zD`<+FuoDk)F!c})Io#>jE~>|<)P_mB?P>*zUwmfs1t(nax1?`LGd8q!a4~^}0^lvK zA{@Bkh9s9~`(B^aj?A`Lix>&#jt+0tJltt_-rxkT9GT8ScFfJ69WbKFD#F|W_qSf8 zyTN`+YZfsn$X1_YJ`qG07b6gbH}J;)2S~bCbj8!N)ra>4&_dCNRMI`Bii?X=x6I&u z4-D_;D&~)PpHv9^*$`J!QZhVjzBo78`6$n@(8E!&NMFBBZTHT!ey}x#&OSje%^0(W z(ROup<>uzX7*hI9(7a}Hc~wMM_zeqWIN%(R`)~|TYw&8?+}UA4q4G%9&CYHDWS7;| z)kp42u)e}w0a=nw1G+askRE8 z8HaAH%wL?GOcZl0>P(=!2NL=%S=GqMNS8#e@Y_V;``Y~X_;!=>6Df>>rp(Dtc8%RD zb)z*$)80@_2{ai0JNR2+TC~6$Xz^m{Na^+$7^e?Qvjb1#0tHCVXlYShfQ=D|(;O`U ziTDe;rn=8Tpfawe@$|K%%Z2}VzWdWu)nvI#csiQ-MR7o9dvxK5StnhtH zY*tG?B{*3Ku#|xSOVol}z*5tyuWMIGvy6^rrI>26M;M)5-3yXOCa$`()Madw`IXpe zibemcaB4Bt`2vc}h_;ySdUdBOh!3=lM#fcFau#}mHvoDqHB>j+GiPq6h~7hiqyd~6 zMLUcj$nl#KEOP1ZPB8YraDwA%+`0z`+7kz=TCwtz9->Y|Rab3QBL8gzx9o<86k-EY z$Ny#nyM_fR9>w`2+s%U&=C#G3;bm_N@HaIzMWJLh4GbQ%WM+13cXvoBymb4d$*q-a zu;KaGH!kOLH)(Vt_Vc87GsS6l?mt4kMU4?3=E{8k6R$H?*}kVCV|6 z3JNG&{Hf0zr9f;^YdO=OhtEMyhfdIqNc-qUZhT#DPtu95zy>WaGBPSAfW}KOi;Zpc zq*D$ss(M#%ELkc*>An=k3=0 zCrxeR=&OyyIRG&yM0fI{_O(C$IGz}EP}9Nb2LL+x0{9B_2i!+bY)`k0F^RmaEI)hn zi6E0c>uh0A7Y`O&hk-f)`_5Bx#pLS9;{kF#BzMR;2?qkU{9)f+TmEod#y|MObmV{d z!(ps{_lISNt1vBbu7=q27jN`jflT>jv`kLfI7e_yKc&?fN5{OU%KZ;xV!(%S1B@)z z%f==H7@Of)pOG8O%_XNN__QT(iMy|WUQCUNC(VaWc7(ysz|&&Ak*~U`PY9 z!KcLIr62ib1t~QlhML;-J$v@#*M0ji_f|<<=*#uDF^jv#sei^@i zx-Inv58WoU-J3$)BfiV*F6D&23YN1(*%66*b zH3Q8*yAHcI)A+se2N0A9>z`kITkk|6?TsN)_BK4AXc3*^27A(AI%JXQb_5}sgJwTm>Y5Yv z{kQw5IF3t*@@ifYq~{@eypwq3&OomIWb(#sh3t@31vDD*Kv1Rc0agP6hwHN`LRton z4Q1gi)`vDk(tkk0jY|={vh+EsECtD>Mym2GdgW0SR=8B)Q05qqthtOt;Dso_)&aYk zAa*VtF|9*g&;$^_vo?Gp=YlNA)MWcw9)G+dIJcMoJE-X?&farlET4TpIX;V!YUrU- zq5d;jmnOal ziIV*LMO??h82ht7MQWqJKiYZ|xaY_B@d$ag6@0l;b<6i&8a3}u zs}c>r=w0|EDeaMge%@jXxx&eqnUJj~4$Ld-LS4Ds~q-%TQHp&y3ntXIKl zx*bFcGqW48lw_cjmzS0bjGvIQWXUtam#5Ub6aOE;;GA3({N^0F2XgrOhhH*{juA)24T1%gY@-%bhSbaP{4<;lf z$sG6UBlm&x2YIvzRw+V$EYQ(rwQzSUMSZ26Vv7UFxEZLnD7vBHkr9{aY;gFoCgt9u zrIqXA(|HCmEZqA%X7OTn?X2;sPo6A5zY4fNq!)}8IIk&o$aorKH$b2n@}9;@KAOTn z)=c%7%|%#MHAVJqnwRN&Hzzt)R#se7V8GYaH8V3aDE_&Fhgu`@u@gInseJXt`9Xbf`E_cWT7MS$gNpu)Q@#il(5X+C?~{D+$H-1@~sXjn~c ziXM=8U7Jh`s$r3~T*Iz`KiNWXbhOwYUbjHR4`j|W3~ zj*Q!Rv~Kw>?55Ej_6&+ubTG8WnDq(S&mYy=Sd$AlAX;XGE;(C^)bzZpe96@IP>#jE z&3umCx%JaLFvx7~nictE=LBw#O zOAaZawuyAu*xAX5IIgx}*LUM_FLOz*;&UA)=1a<9TtO(ACB97ymVpti zs9@Bw4gz)4K7C0jb^ZIu`4SvhX=K(6Y`g^o42Z`tVn$a>%XrLaYu$wF>Y@)^hE`T- zi5$5mM{So;%RE#^QhBGM;Y^%zb{1SeWM*MOrL_w?KRE+q1kL49Kd5#Z(8Ww6p~cg^ zjq0kciiLMsBoe90AMTFv%qdH%a6pcFzT(pI=OL&~Id+cWDQ(suMALo7Q7}VkB}33F4ToG(?xoi)cb?4zhu@a2JJDR1AUIg<0Km#8hiT^+= z2W)R~g7F-Oxw);D${`~7UGl2Pt`~dnTr@X-xPJgb|M;;Q&1Ej~0awX%?lE$3!K18P zU0to`$K|_Pl&fMHd*WPAAw)h+@N>QkKSl-zFPk7eE3zndVljYv9KZ1`w$p6b%?|;> z^06229N`KSKBv!ydEe7D+ABP&)kz(RT&7MNyDs+tnPo#;3_-oT#TjIF4am%C-wp{N z$N}-i3<6{||3=fqs=S!G=M%k!?~)fivbNGGW~MN&IPaZfW?0-vdY$rY-ax4XS;}FF zbtQc3g1VHp?~ei&4EIXREYhkXOA|U|g@;x(KP2YI>dJ<>$NQ;tRTR?3vYUe=H0Gx= zClZ+#3|1{g*DI{gxp;3N-WCasm8;XYFLQ#LGbqJO1-B~Sh7P}p$c-P(L$p$8J}EPn zMlu1_rqYn^jym5y|I&fTI0a$1Nt}z|hIBYmAKG^sjtM0(iAmG#N5+_fyRe>;IlqB& zmQcO;O=FvjtK};bCoHgS+W$CxZ)UQ#rtjNLzfIqpjbwk-hyv@505NTDZ=YKfKqfvi2 z$zXQTXW{61c=vF!O*2k**WV_uZMwrGMOG*oBA?b?lL{<(7W5foqy8rjGSa!~tTo#f z@r)QO!V#0;e*ai1_GPG=>5nxcnLwWia64M=SmphigQ=3nacoTw!ow-lOslT&9X)!~ z2n~uz+(C+siM9l2;_4_WDWMJ7N@^ecmDHxsdV}yF(}2bU+yR(#N5!MLTLQ|VC^2Tu>S?+~rc zppgQlLBETw!TyzCFcQi`C!QqOm|zeb=)bU)iO_}K^{|-HoMap)-la0)?p=wRq{sQCFq|>P*?vhG>YUQ^@d0W8^vy)94FjZTh$C$4KD0oRNe~x1slp8GSQ$M)b*sF?bP+Rwo8y@jg zne8a3gt!J#b_!vDxJKNwr67t7I7P2_qf06k)*Bi|S|#qQ6E=$%R4MA=?xW^0KRrh) z)oiiqvNo)4v9LCF=ryer;#H5oM;0&JK*`ySmhxquktGX1msy^=Yim%va2Cp&kiH!# zcreLU+WNu1EMm({=6(i63EuoKVcg}?97TM54l>oP(gm<>Is9*$D`_DUFZCBCQw&6f z(7TW#1By2hO`&=_6rLijy<87=*+N}9to55gYIu;q`@lBA=~pxrwjKzZ_9yFN{eLO9 zPGSi0SC#PZu}jcEwV{#eS9}|0ZRo@L_OkKQ93_(Iw%q|+mB1+69paGw`=kyBAC)t| bvHhjS!)p$OC6VyA5Vol)Ybq5eTHg8>I2C11 literal 24111 zcmb@ucT`jD*DVT)0xG>rFH!{rrI*k_g&?BzqM-ENYY^!jq<0ad2#5%T4$?bFN03hF zy$FE-=Lx>=_xrwc#~JsGJFb63i90)cKYKlE%{Av-ggk>P;@!G?3j+fK@3E5Ha}11Y zoER9GD4c8H9lL3gr{FIRCwVO=V_Q3StG6aj7>aLg-a5Q?di#dK$eqE=$;nQXm)Fkf zwT+XrwH1%Ctu@gD5gKrY8uJ%gPXE4-feB9Ime!}PqPp>buuiwzXTsPMWAYgbDXxE9 z!t;^f;PM~y`e|=sb%b4Z6|*ksYO@0NtLj=}tF*pLl!-kwFf^BqFU1S`{EC-+KBY%9 znCy8=4o)!4TE&pvYiv|aTl{(bk;8t-Z|Wh;F;mCb6gL(Sfx6N&oKAg6jMLXYT?iK zYbZT^!N6r^z89|G%I|`A{r#ukxT-!sZ=TGXT)*c~R3PpBsz|}tf*OxN$qvzT28mQ>JJvYBR4QEIeZ%tP)HaJ){-p*1$+8ha7?#4?qGmz%+O)v(&R+Q zt-)c#rRZ^Y!GoSR_?D)vjfcC!H2!{z^X@xDPsbYrhB5tH%pL_RFbA{!x+eZn)nnKR zUq|O9LtL*<{O?okwOHMnTszx<*LVtZuJ!UmK~~?z{6n-J-jnxhj-Cq2x!x418?P9pd1Mgnai z28Qn0V>#&;Zj)Q}1X_A8Ggi_o3#nuXNp#*&r3l3~p?*;gwl94)PYvPeQ)W>z#U)9k z(ClkjevNp1FxE=>E7DxWlrJkNH@P$Qv%9=}cno38=hBp{vZf?CIkCXCu3YmGd*yJV z7!}n(XQp)M0Ss|SV>=7UgOa_JCp$V97Z=almN@?H*uRr8dyLwac%;fJs`tgsm5>yjr z*FAf_{o)ZS;;=#u*4({N4n8T6?@jw5j$Tx(n}CsKB#yqD01-sa=)W(E%qq)95jIP! zT>axG6Sz=zNZoUDA{pU)4v3RW9K8$s?B#9@xfpO@!z zbT(VEJNY(LefSsB!v0-`s#0)C{se|@aJ{y+Q2pWa^tEUTLlh&Nxzwl)kKSvy(m~AO zQ*Ewe%`bm+`OEZ@>wAcC*2|M@l>g?F_}F*WQ`@t3NSkVkCkMQj{I4Eua97?Z zQ{0*;UF(3%FuK0fNcgdkB-_b7QR{u?^GO~jnFBd^xbm6}vcN0&C(6o|N;mlNw)gS$ z4d=dr%jE%|i@FK?dE`6+{iFCly$@%74hJ3F8L&7+kX&4vJROS4^(wFje0w^VP!&@l z+OS3bh^VMo&&$)2%kF%>Bqs4c6F%gM^?!7FYMfm=^>Lgc>3t?Cw_ow@gQwG5g-SxX zV|7M#k%&}05EEO>j>5m~JYBUnWhA!TIVOdjmFc7^OnC5?=4d_$($azGxpW%HxWtaN zEUDP{un?_X>m;}cSGer(IsEN1#qBVjA~1V0S|W;2D<~Kbdq<}6?naCj+$!e5sx9o` zATDmhj7X+g4is0i1bQLPpK-qG&j4S4uhlv^a(Ui(_K=@^<37#FS);AugQeMmwN2DFVWw0R z%JyP%T z9W-7Zdu+^zZEFQ!u#1Vcc#z^Fqb}{O)dm|Meii=F0hV`BM-J>g%;?OdB;L!zyvq_d zaf~QqtJfg7A;=}N;0_S-iZk0myghyAjUT0qFt7cg6B+VH9$lUqDq0V`d|5#h8qgeb zk?2G9!*?Ioza%fW>|kmk*q6{Ll(3`fxbeJli)TC$c9A}rvSX22|e{_mj{5*pi zOgF3jzez#e1l(8IN!C;m$>{Hwf=*|se{$GW>JrXxQHH^evgsG%26uxJp8t<@I2@m8 zc9xAk-tHOp8+6ufMCRG2nJ8S$Oq`r2iwfw_R%lvaaMMZMRw$Fl!Tt4djK1tt)&a`m zLWPwut|BLUo4A?pDUz|A0i8)#+MXs8${@22n67?4xRFPHcIhS{_Xzw*Q|35P>`y2o z1JW)b`jY>B$wGNy((5UTULB+7reY>7(|IRcyPF{E1}-DW*SrSZ{)F8G=@eH#XY$VZ zZ72niv-lfp+=Cn)9aWOZ;4oqu?ab8c?E4itf)W#rVc$XPdcpt^5fORmPiPcm$L{a; zw(0#1$sU5Q-@?E<*m_u*VT82)AJHW#j-GhvGxpyk-}|F^<6}?Mk#y|rc)9qPrO2>| z8hg*9{mrt0+}zyu)>e=)tT7>p6~`F;6;5kIe%J&a#i}PK0#i1Fje@zerFqbFnY_Na z_{CwVk0`?Kfi%J>QRBdwnVGb-wB5warKLAgqaq^%rC`yNR8-ovF2y|vkMpDP z(W5Ms(x1&=i!QYsptcZy-Y3!cPDEtWR$yoLa0rxDRk_2|NEFtYkygEjv#P3UlAbTP zYUG}~|Erpa$H#p_v*Cvm6B8;z(=01NZ0!N`$tAz$YBgfo)rlY7`@-zv`RZnHqB5=WF9gd$k12e)<@WQc~z5o;ZOK=6&0~>wazHgxi6J=(8)60yN3v%FtDo;JhbK>QWf9B>OM}n z?2KX7*4F0Z<6CtZ4%iz}SLN3Fh7r)IBSO_VvKu9Nii7>-_IMH88aFgddrUe{_E9Ry zozv4(c8Hz_;r0NG&SRA^XJca{;Mi-o2|EJW)6+vuO&vkvm{*E-%$Bx8ZCmHtT3k8v zh3avB`VEVt^}>?7`el3s76DGOve(lBB2iY8<&R%H?;LF*fD{>K?!b;~mBvR$n|c^9 zGb1C|G1ur(bJ$9=?knj^Ez=D8y1JIS4WDGA1IvaB2+}GD#iuyCtWMn4M>t{v<9#j; zGN(Ax=;7q%C+deR#GgNZ4n^Ij;E`LXKb|3Rd1OFIAin#qO2?*Z{$_=*GG36)G^n8K z%>sqiy6+c^5}bdo5%w)(rBN7q5tZR7Pp_st)|&NdsiXG0Jl!biz_Q+3Xge|MjWW+o zN?NqgsAcIbifJz&9H-W{Fx{IEq7QwvoHr((l*uH)4{q#ZG81Gy1WPXp<~>&xu_EJH zs&;Z;*-eeXA}Jw(RrD5#g%t#bs#4u&p30Y$gMP9ud{dsTe7lUvH>r1x^c0>j`m-yl zUWtLwUm=vUt3H@W1<#c~x?uh%ov@She99=CTiTv4m5I_h$V`y!o0K!*T^Co^;ld9% zZhsEycf3_MD`tL_dMMEH(n}$6t?u@ZQ(kg-l?p`AecK}B^H#tLy!^zL>9zkDWms+& z1Dj_(|G6of$BBHCZ9X|^36tYAO2GJ9yDiE{-Eg<_g=3W_Bbg+bjjnacJ>dw3bo@ap z=&MCKC#b3_fj(yl2GV_}{`i3+*Zupg{G-?vCuNb?;{1F* z)SGxx82m;=9}BSr6Ms7&)ifki&GPy{Kl0?aE=6(lRC;&ZK&afk;-}Y1SBRtB*EcrW z{K&CHjcPj9 zWC0uLL7B7>2&+)^hI9J7^{Y%V$NKhh!6f{iTYZvs>Z>in^kpo&V(I74i7SGh5-^#p zHSH~S#B}!ele@Q)+z!`GZY(V&8b*#5YKBc6u2baV8p6#+|4n~^d_$@noakb z51@GPtF@PwxkD4_ezsMqW~wZh>A5Yk=#s}z#3G$fxJ1QOu(mxYb&}|={-Z>@G38*w zI4Wc(n<0WbB`dy)&GDd5$koZ|q=xW!!IO)wsL?TtVH!~<9D8EDGB22oCr_1gFxQjF zkdUT$6Fp6HiS90`cJZf)JQB|9aY|F<+v*k8Pj@}|rUaWN8Abc%b*}TPFuj^TvN}(J zE@>p`D_(ORh?u-8q(RSNN=iib1mo3eFq3#xE0joj@xcOT?^kBb5YK#xxJ;^OdKslS z*)7J0WrshKQbAG6LjC&!N(-n>ahVto&x2n1gOcU%(A!UpOuitNlDSPRBO_wM0YSh0 z40dgrXmN4Trc#W9gM*coRYJmh{Hy-DLwkF>Ri&qR2FU6kK74pr*QG{ogkrj3v3k5C zNKQ`v=g*(+sE$b5sc)~Zqw6U$xipfUo7>vLqQ(JnYlwt|*_7%AVhklwk+%DRr@7x@ZH z1Ll7%nX_mBZarmTVG%vi=;NcWuivWQt>+=7R37n5yJ7b(pR}`c#n#s_3f|?Fm6que z?XsI|k6Vf#6zX(tIqa;ini&|-;SconkyUOcaO=9vH{W1srPjN7^JcUH>GwBX#NK@B zY16?KBG1O;!^r1z^vG3lLU2r~QEB4t{1Orp+}wRzF0Rhb^~mK^2TghTfGSz%xgVID zM*^oDyu7yN=E-Ru_sh+PSy@0x6S*@Cf(@UN!f6z75TK8VL{5AbLr!q(lLLdq@n@B)#v(L^~a}AHz;LA?;}kzJ2jr znjd0gvlMV~ah1S|&qLxK-}l(|c=GA@ly0UQ8{khtH!3X5a;5hr=s zdwJC<1xL$2Jo$DjaD*&N$jBHhU31a<_t$cW&NqKotLF$HOhum(fJf09>Q}5=DJrpj zYa-O)Z5fym7=W#b19~l{$k*BUM^}K#2y(N%gxJMZ2aj5U_gFMF3L@u4omrDNAX}F6Vm8_c*0noe+eR`dN+lT zco}>}&%o*``vQTSI8P!M2S>q|FBV2dGMjg)sb$|?tMBUUq`Z5Vm^v&h3}jActn2qy zmX{aoN2ls8UXL8-hl(raP#T~pkAJ|&7(YZSUa*y8Q&-Smxtk>$?(kl$>!*0rSMWRXXY9^E$R6hsEuw%jho z@{r&Vbw_l^|8rQgB`>Z0LTgA}op?;-A;tIPI-jbpfu#}EnXDGDZP6S-tWBkEUA*b@ z#Q-aKKW#{EE-Q^0N{wbwb%!DP$&)7**47p0_k@z+4b%O>r!SrNRb1OMYZX4#*Gus6 z4Z@A!qk;lsDNU#N^<6&a?pY@%Zk_NNk-omZ{l3Q_bFf3yxFO&`F;1+pI?5t`V~=%UsWwKcWHvaUHQ5?_>`v}K@f zTufE8^D{NgV~z!O12y#d>H6{>M739)Y+4y$X3RyiFD3BgrCHnkeY*@oY- z>L~@EBx);Mmtlp#cYexJw++*_N`ddwYRpFSE~Xk?su~&zZbR7zGJ<=AbnbAgU!OQ2 z&%Q@S;K}y6y^v*elY~dmRdeOKAjd~%!aMFTn-Kntsm@3tR26f zKBYsR#8;xwF((dUj+Z8F_-k)(?{J=~I_y23k$_W0taHuf8~t(mk}W9j-7nL&)OYOX z1sG#)P#Adb1ZQN>ZFs|9G1!L5v#q|#oAvT@Zk^4lf}E}G*An!dJ+yov2O@vJCDA(e zhi=EADPADxo7f>SoDfu4RpTE>jS5RYt*8w8n1@paRZ`7%4nHptQ=;Tvg=4eAIng1|!-c^4dIHA5 zrZvF;fr)PIu3r8&go-!Ntss<~hMwNFcKY$-$DkEhTU)Dj*?OM%P*BjMc6xEIwY@zd zE)Hucef4{^$jxUhV5yv8kS8wHdLX)~1+6G)Y15%yr+FEY-z%@X1*~cgtlh=YetFcl zQsE5hTt_K8qoia*dAZY@H!KV-EDZ2vkP=$n*2YA}-r!qGNlE#tSN;C5wN-AEG}Hm) zaO=w3)>R%y7C#V*HgZTi66mc*=^xa#Kmh7eP-W~hdwMD?b^P-cc{bB+VH7>x-GM@ zk{_RPNu>Kf963J3Y~Nezj-0|LAN!fpL7!xTjRN6?J@;r zTm&sao)}}huaKx{N@8MA3xJ+&TPbgQ67;L=FhjDk7~fD+QWCmC67Fk%;w6S(pGu-H z8v(^uWV*nSlj`9r$Ual3t*xzHf%a>~64!sG_4X=T$=Us!oa7Y4>{IMIJKjk&EtjD2 zJl>j0%*e3$@~(oKOhedi4&$1fFrhnJH(Wy)oy((1$k%pugq)T1^{4yO#RN()qdNTR zL8V}3UR)i_UcW6X_;ZpyCabj6{sM(u-2&ZbOAtZc=g;mH*`r?V`U;MX2mNA|za4J( zvx}Lwg_1FSYmhAnd-+a+9*XdTMKXecc0~LB!Y4&|7BLiR~;V zHY^X-4Gd<6hM2@ha#d2J^NWi7DHJ||B@hs($j!aEh6V)$A5pKK+%{bzWfPh9l8qO% zov})t)%j>W@Iw}G3XFq;gP8Q3j$`BFi*|2fO`X;!;Ba_PB5x|^t1Sh2Hh|LS^*W=} z^DS?zM9R6k3h`6+_+wa2*Sx&EE0u}!a${?&!XmGr8DShwx%kLWg?vH?fKGPVIUOyP3xEAK7O$y2S5^zEWR=MCwxnr z&@tGESHC7llE%}TK3c0*ZSWD19#mPmGYwuLbE9ab*&vCo%z06vWL!#GTH4&4qrBs; zf|i(G#z=wM5SFNm6MtqbUFUJOlz#f#=^o7Va|Hda;7CQ?zGeqm3AvFEv&+H&gM&DVOB{n+Z$-M zpE~_^g>l5^f>e`Ub6qa?=fm@0?@9?0ZJTM3A;PB2zOBv8eWcGVwT^3`JyJ1|V7_Zv z^XKDlDk>@i)z^1fQ}S+0fSfe;!!jw?ZJ(~jrUfkLvKj1v^am$Z$0lSb#JTM!2Nm(U>}E@ju&vQ|2Z61&CAWTaH$$sxFHAoNzh*AbGnwd5YC_Dc{r%3nkM{O zoSfcubK0K`P>{oaCtwEtKcK-Y5kSf(!B~@)dbhG3Dj%u>5^;$DBv5&jZ$VlQqWMwr zmoK-zKAF*!e)oPbJ|>1S$jsip)Wi3?Jwn&Uh6i7471tdD*Cw>xp!7qUCc%Y)Z4=7@OwGRAIF5uPP2XOB}rsT@*yu`7$8$ z54Lmfa~KY{l8o=EG*$sANpi! z86FKIF6R+#8$Q)}@}~MU=t>uyx$kGUqIF?N%QZ1nI;UH5j_{!pLACFEMG3w$y=o!( zdQ=}Y(_U3a-hGmAJ5!f#f~#9&V4T11bM!-~c$ti3t&H2{8l#$k$$3oXjoakE=}b_F zw2~Pp6V+eD98H{gRakLxYWB-qcV@(ziC1@NSw5iaa6jK|?ZFT9l2xpLdSy{EPJo0$ zaTwp+1jOb4>|*~L=PJmA2%1ffaspsO`2SJqoechVvkmY5;sSu;<545y;|Hz=xL_r3 zP0xBQ@)`iXHrdy(j(k5hHdb98e4!u{3nSLaK?D8B4ikX)zi`sL%rfYZ2XJ-t&C6Az zlzrAPuNG}!|KWMSxO5^t(yI&M(3dV&`R&{8R^{xNh+O?NM^wK7Ei_#e@UqVhtp)pH zPsb;cYov|}DWXp|NQnb!$#N2)L4br11UrnCbs0WolS(T=o}n4gbyRFn?_p-vS-#;% zAo-)?c*axJrb$r?8+VNj@SZ_Y?9yIxj3-GSJ^;40?+*YYG9J@4&hxww&p(^?wbOB# z?|M{iZEc}YC{|b0EE&G8YjRrJ{ko~u)m5u_`s~qbqY8#4Sc~q4Zu2*Cc4{_wVo|SZ zlL#ph*mT6WR&u8&dy5X5hK8BFo1yPWZrY9{fK!*4)31GX|SwKY#chc9V9b%&U zQe51Y(B1tchu#Ne-#t*i)L~8;d`O}y?h7>pywC3LH>S%NX2tK{zZ)4DJ-ffmnx~qX z;F;;MUH~HMBOzZK#$Os(!htCRP-0Tv;hzx*#I)Bz-^t0KBK^G7U-}Pooat*ZV6Dg# zvkSDevC*7dB+-fV$VJPY#X6zClX1c?ECA;l}ttJ`bnATM~gWG_w{Q?FI6+d zIc%ImxpOCt0wZU1tL)}uRVB^E!)oT&#o0;7o<4qD@b#Apn%|Pm%wS4X<6u%jULftXuYjfkm856* z`t@s&>m<>O>jR2Cu^pVBnD~PqeiL~)t@+^4tDmEz0-gzUost2VfZQg^*=ablNq-$; ziBBQ=m-O9LQK|z$Rnl@D)-*zo{3;8Okf+2g%HxB^mY;_-x z^Ijm8)6*Bv)v8PYPxa%7^9$n_)B75K>2BlJwdf!^4;*?l%RTOfX1V3#6BC651)1yz zB#!f~^UP8>9x9Ziwaz!1eJp^Ralm(`DhX=C1M;cA_B#SvG?ZWdX7Z(_;o0368Wve~ zmD?4{x+EJt1fqI0I31>ae)x(tSy|FfiJC}ziwdZB!`bu>Q^cAetEkeeawFy^Qg;vR za{ze+UBludspOt5@Em0=b9uj9QEV(wV*TPpOTlbfw|zJ7vg%LD#jiWwJDy3OoA?$Q zcO#e}<>W&=F8V&_N6*G$Vu-AKd5n1B%+~2zSpGmxqk6cLk&&VOeRuK%g%^9H%K6R< zu!tA(dCb1qq{vbVTxBmiS_TWg?KUixoY#<2T6n7>DHkiV9Zl}T(5QSC9*=+G&<~4R z4ck)j7j!9IRIW@!Aq_mJ;75Oq1S{S4jJzI^2c&FFfY?*lQU2cAcZg;xtMdxrpN@YL zzWAXPj1IYAmexN6k|#eN@hZ^xnX~4OE;x+d2cdSle*k#(l&{Z?>9lnRFAWeK1*4)z zM6Pa2)&5)*+myb%x8(3ZHEtDn_l{sHiXLLa`TL*sxv1D0wJ?~<&ks~@6p3FXec)SR zIi6xe2WU&BUgYlOT$5k>K3|BmPH+MxGtT>f?a(l8w@JSVt~ZT$DTY#QCGnmn!)X)v zBb4Oz_7)7jMC0WR**V|k#hD5t`*E{5>f}Y0bwz!`+=!Ye?WbQf%+Mf=-nLeS4HaA! z-PK2Zlz1&MS=UVa(}DSr}B5feoaO%)^CoajhU4Gof+{g#$Skgo^ta>)K1>CjUG zFv3(Pi^a;iImFGQQ{K6D?O@;y?9NNt!nJa>755~l2plOvM<7Ga9;&s%>WaHYTw|2- zYTD=WTq(>vnKEkO<~tlI#CcTkSO!%Bz5D0M{>`P{YsR|XA+I?7G4y`t6RAJUOIr}4 z8?nK^NhtIt{nl~CCzv&8rNsbY(hNQn_TGKo%m)3-lZf zfsv{_%pET=}g)FV3(o_il+aHWH^pc;#o{=b%`ZH+TCUDkhcD!j=9xqRUQhUeK=Q*V9-~L5j zh#OvOQ@5wULe6Voh{KGFtECH2#^iY^NMMJTw}}kYKXbtx`p+B;pK!-p|YrJi-jICMlx*bakt7 ziHVDgAPKh`s}3K)vFY3}9KBe&yStq-&e>B~ZC-}V4GFi~1y{h0@Fzw_SVE7wBBAa- zvH0G}H!nSlq=)d>1Wss7z|lL|aAmzbpT$FpQpv`Za~K3DouT-$^g-Lf9XI7t+E~{Z zJayauCBe!~Mx9hJNY(S76l!v!>LLw-Q1RKDA)LtVYGUAvPI)e?b>Q3>>3 zfHLR4bNKdS8*`ivl5s3IG<2QonX{fmlJG}_#sNK`;2(0F@FG#a{}4ANb)@Ov_m-x# z)GK)_aZ6q4j&bicpSeshGX)3L0m%A?BL(XJE(+#2iAm&ySy7-SveYFQ>Zm+H$JC(v zZaqYN0p3v6im-a>WeXLDsw;k4&wWiCq+dDhl8RHkBtcGPo1CE5RLwj|V#(w6JzJgb z5hEKL_A*J2EpvoTuGgM#t_ z*_e-wj%pt)hVmD`Aqta6tZz{nS`q^118yTEOUfVLNNGT6V@}fSwu`M3f8PqaSt;Nl zI8c;JUci>O+NryND@P2VMDwqG~|mjPS}?PZ{r}?e$u}7w$412hozJt4{-M!HvsY7 z#QqzOs{s%%nlcAqG&m##P%+hj2?bZR;^Hm9cf*9h+M!J!Q9XzcwSW^^$RP{~3HeFKLKv5iz$W^j z^rI{ORdS;FY4}OifK6KgJ~nbmCN{ts=@7^yYg`un%LLkcFIo&Q{af) zTs=xY^lC#OK4xZ0FmcCaWe^aNs8^@FMgc(H(mqDF_T@N+PM-)D8D%v6XM!W#-Q5A9 zapT4fCFav#)sA^e*bh!ibGtX?20xtKuP%aehW1E?MhG8PJq}TM-(K5-_|YkKv+2V4x4I*oX4wI80M`O(jVwA z?=A>sdhc6;Rv|7kKQ?oXMBDHq?m$m@<5EnNp?erm34nJ`7KvNn?D24C%iv@yk)nkD zifvVc0WNbl;I@g8v9WuB$^k!_kG>)I$*(UFyzM3Mo3dzq>xv)!u%w8hMePgK-IVo@ z)*jfbMz%KwpLwovxii^s|0Kc(3?3vLP^GNcLCeM4$_59J#!D8!UjIXyiaZj^lMq~4 zHS{uF+NEx+-Pd)&2N&C!09r46AEa847}$(4HxlW*=I7i5{L4^oc&=bqGTG>${;K=RmWBP_GpgVl1_e!kg!A(~5Kt(~fWB}=$~qa;#c{id zXCx&hiKv6vq?Pb^^rix6Il}|t9zcG)B3idL(M0P{0?>(EkZEaYO-xO3LFmL>tXcx` zpenl+U#=zg+_;`KF*vA}o|`P}b0HonkGr?G2l{i+rNe)ALPGtObLJz~PF5BDul$Ykhp*jQP=NzJ-G+{-G6SIy|rcyNsiBIEm$?IOO~dJ3MBa+l@L&a9!PCI!2ywZtvP zxRUx?xYAN4lU1I_Hd9ZVue8zUFs8u`hj{^9ODpf7v8-nJ_!zK73#*>vAt2 zPlFGLSm`%fim$UZ;ezb5W*m(K&jX(X;8*QGK6A>1DuU{ELUKOw^;S!c z!Is~M$b1%>p~S=SyZTa#rU!I9zqy?FL`0}E9-)(R@Ae-3#k9vP2c^)g-3-WqW&@#Q zT!+q@xBOx^IhKBQk&iVGZskJ3e(LP(-1P_rdJi-4{*e88HUvEOFZB{-U6mgsBSZwaO?W%nZD9q0n2JGZYP-VqygOX_a5)rk{-Q5Sj6Q;Hj zrsh`>-YiqZBk$ETG*~tGK3?qOKC!m8_V5tBxGkZCPVeU3Q3XgzG>Z?U?pqxZ3XGHA zcsM3Abdu-k4ihJyjagF*RKYz~3_jqt^M8`jvjmKtx{$!}6prGjK zAb{V?zhUg;^hijEB8XujCd0|jj*zOg<=QM(R1!-Rz2oT6OajkT9Gc47XWE@ z{nghgW=_f`&(@PjOGwO80?d{~#!Vz_&?N*Sq>Y>LviV40kn z0+tbLE)!jAv;u5pNW}(IVUv`k`(A7Vb##2_!x(YpjE{q`adGvW9GIopLLVaSH&R!_ z8_U-A+`duZR-p0?CGuc-Y3f~}p})2T(E9eDc>8>JZ*K{QyFS{ZaFJCDC*cxhK%`wm zlbNFA;bCD9FNPYHQ-_sdzfW03H^jyWYx5C`=j$RzVu-XS8ZUZXN(xHH?!ZgJpF*MJ zjq~1XL-(8=!*_aFSttz>Zn&*!cK+8HcNNB7JLp#9pF3PuQY6a=zRx3gT40jtgR7IQBl!DDzHOlt|X$4_vh#5jJ1DEmcoQz%R`}& zTKrI@=sF$>gIJ~SuK@24%$*#NrkkaAYf=*L;eHwajEV;Tp{g2nB&qu#y0YyPi4(ui z!9ULuZL#epz)iVf`X`E&0eFuIfRl2qx9A4&IQYYQL4`xRn69MSD^CK@(LvmU7c?P@ zR0HyH3_jX>Vr`rNIv?)NjUWIo1VFDtMUL8xH9R;)BNilMR@q$Vz|0B#?1`$h0AC3j z8#&R}#Wmu%-lRvRCK4au*L?RoZ^5cVpZx?E9U2O^L+w^RZcsCx>C=Gq&pNIF`f2kG z%PMfXH{{^ubyGkF>|%-iFE~nGHO+VN0cyI$BcY&M(pRWYrmv%<&e>}k{wHZ_(zx;Q zRmU>6kKVMAqwmLF1=q`UTJ2w*cltdO<);wbIfA_T4! zaKk_TuQ&XEq;6pG>iv=Q`JYo{9V~oC1pymQ?eum|P7bIPhlhvywJrrcRMgaE)7yJ) zASj+bd18mD+6e#uN#V{#1E6Zut4=_D>sLG6SObmB7YRkWaDD+ZS?hGvy{8GQVg6e`1eZJRI}d)6hq|t( zJ!!uqDk(W5=;P@r<`q#YWjTQHi0$D+5^5rvayYyjDs`Z?e{p_>pt2_DnDIXQ^Eof? zO6LtA!>18`he3odHxu%`m<(D2#HuBK`O~vA724;Tnw5?#&_2r_i+it1dJx^6ox&c6 z`KQg%V1Vt(8#61bG~gifs4Xiy*qv_yg~edYOd!)faqu_VcgJ1%`~Ac2e(h4D035UM zBE9M%3!=6MgAn~yV6OPK3~&_XE6Np{%G$!h4Pfhzs|B#_Bc9Tag4ZP_gQ@^7K7AUt z`|MvpWYqC7}XM^k}+6dJT*DVW7uGIrRk?X)6fWXWQ0JozLKSh5l>G~cfPt$a_yV* zhQ_%ihBwoT;>ybLFX_4XiFi&Hq+hXCG1q~d8Nn{s5vg$?vW7YNJ}p`|ZCWswNGiUV zOyds8`7mg-N--VZxOFu*%iK}QB4ps;c+B$Pn)>tSUh(w3uiI#SP!y2_>N@s*4f*+I zXMeVmo-ym|>D^(YltRWFXqSlG%jZmMJ2|QPdl$x@T zRw5`~{CUcLHC;8hA@B_R(b2iJRT@Y%hM&mhY!Ua!H!2J$8~Kb=LN0eDH3kG| zk$cQ}*VtB!wy3I7fg59$YXLWTcy(?h$a71}*nB;g%5ie+NydoNNgy6|w2Hr2%1e3V zg5X^%QDNcLAag6~&$f!*vDyQ{?I~X+*j*JA0>{;p0z;P90O8iqd{V$9dPROlt9X-C(o*d``{Yz1N+j-`uh5!qFy(b^e>-mR3Yb178YMj zf;_EXNG)NXzY;$~!^wVj*&0Q9@59y*l6K=gdtfsiQ)dqhxmc6gaRz;)f5i=8uHRT+ z2PO|Tn{S)bwVIM4fN>$50>SjXtPJs@e~mSs1Cy6E1q1)K zMb288I32+?R#c`vvUH1T#M;e`QnK?T3ERB%g9XsH*IUxpji(}@DkM?XIyyRN*H=Cn zC&OUs;$a|$wxmk}mEh+LGDX>uv;FB=eZ%*(b37_7tqXS+< z`WPwCh^LHB|6Du=1fXDHQPI$o>*^wjJptxAFJF@B#n+X*$GDLVY$~$&g9{5rN7vpi zc2|NlLlYUSh`-je`MPL`HN(w%ZX=Q(or<&~rdD!{BfZxY?~;#)ofEm7{3PfYGb zmgMgON%-{YNI7r>YMBD1km=6zktgCmw7`D<7Er3}raYp3aQ7XN8V(QL(qQI47mveu z1xpuyl>%6brxKffT|q$sG)l&KRX|W`ZFx9#OAL=B6Uv4JbyXc;%RwHnX7ijpt{H+oJ65d?f&N*%_w%k|T@8_)L_j$|v&to<0f2X@c^B8WWgD8KQKYO@Ng z)o@(%HVkgD7`9hf+(ZcKxC{<%0xG!C&+sB}9Ib(ybRqg77kj*SE)7nfWGJPHfkp=S zE?G#2n`93(TvXJ~j8Fggb{q#RybUzen07qCJ&A z@5phtb3gltu08+dn9k}~bK`Ja!$i|ArpRVX-2y2rW?~9ngWiFGpjv=O@rGgpbzhG; zEV)zzEBhl}UV1_ir?oEmc8qwX7u+L8Dxii z_-l5;#;38Xoo-No067+PGO_AW0fN?({&H{MB0xQhF#{x_ib`=ofpkrEHPsXVbcA52 z$k?ls7tV6;UPvqG_^(oMCa}5@Da~I3mX$4Udow;RjzR6ki*oNX7p#{`px1JBbyWv* zQY@3%^iwZQ|9X9vNeQd~CHf&P?T)|`B_%0Q9v*x^SDvT>h!w5P4hj8?og46`-Smv< z&EVmMMpN{^9c^Jr2`@Xl)ZM`WkT~RZ)L~VJ?+A(S84K2s8+eM-dBs`IgJK8%?-)66 zZ4G?V&r1P)& z#CDQrOXDD*uJ||14Vvn6h!3h;7yVA=FIEUpB$pF8E+&m_&}^Vj<6;NAiSS=rrO_2_ zI5bWGLsA>~G0Jon0U}a{6SA*Ldj~yE7z3IG{79eI7svaVUr9n7gCgzNiDi(l4`ttr ziMXwC`CN227Vh5XfVuCT8g83|_W@D={>pSI14EOUwdcN($XSmq-piiOc_El9q7qye z(XIajk;pX1UlE9=8m|C94+`L0GzrIV$@^BOb2ITD`^eJe`FP5o&_EHGJJRoYJB03Yy&~eJ( zgn0NLheIzM`Ze>k-$QoJr1K0}{G;=+FCJKA4)^v>LD7K@h7i#%7J%Q4P3e4l5Gi7= z+dw1;3Jg@Pl?2m1US2>6Un2KXR#A~(01guJ9Uekl)0`WcE$JiXa0Kin9y4M3g&#EdWR=b<-M1F^eBvw zzEmM*eND}v^57r~OUu&FfVNxfp!X3MHI7NPRx?{t%oj#a6b{_^gH-NPU8Sz|I9dnd z`-AF!FnXBzu`{a9m(_)goO~I0_n9x4xeq`NVD^33t)!&Xe|of0dJm+lH9)`)uRTD@ zPsDuq@LXTtTKou%SlKf`C2VKvK!OsHCU+MV6U(wrk_>y*V+l(;Urx1c?=VCyPzck0 zZv(TGPQxmnB-`(8LFH*h99X!xZY#bPdm`+8TAn-Z_~jcPfhePx*x96IiT84{=?Eg9 z+Opn{6Q8g8rZUmj>gBsMTr2%TGj7bE$n6(jP<|>Xz+D4YFp8-`wUh{X43=D41JB+U zvQs+ReWtBraVh)Ea0nRCT0MAKr~%rS4a&U`whXD^y5&mis)@6ICk=zouv5m1t$hLT zAqst-2`yuU-+Q&$!n8`?E2+}<#BtRrW)|bElhGPl6m@3h| z%NRwv7^MB{rk5f5+B%p3}_l`XB&XugG!ZKvKjYQ6l%AH zfO-Ri0trSDpQx?~Kon|siw(~v$LE|{y&V_J*h}&ZfMzlTKAR!D14b9uV^LaMR=s|9 zm}|gRcgKwu*ffg%wQ1~i%H87pv@PhX!C^}qS}9)yoD}S zgEaVV?&r^0^%IdI9B_IF>^B>d$ojCL0~k6jC7BIW3KGEVTpIMzSRs)bUOEbnN8w4N za%w+4NshDSE35hUY5%~Q2-eX)>mzN-ukGBqc~jPN7n0J({2_r3%*7}hXR#Zlr~baj z!<)rbr4gPucYZt@bqF%cp5bqqCMz6szN`ciCg$C{@I^T*A)XfXD6VvrgJ{(0;n5ND z1u1K)3`^@UFW>$Vg6*4mUiVJL1>U0R*0Oa8bBFH~^TJRz3>9o16b51CcCtnSzV_p2 zlT+(LTbQ-&m`HJ?rIl5A2K2tcyD1s3SNETwYW&RiKUXa{?}vH5V@gd50~cmuF#x#~ zmm3kTE?dtx!7`38dDg&zmfA`ppUw71|mh9UYd%+-H;C8X@Bx*=)#4WAU$_5-B zn}*k~pVb9TlEm!$g|PyDJ#oJ)mMx-|@PwJsMuPf)IW=uD_}g7(z3zd+k_HP2eQoXJ zXRI;bB=OzBP;u@Y}bj1PZ53p>#nu)+VHS7om+Og650Cq!ro~c09m6{6jRhQahFu7em zJ{~xcz>smiIaR6pqW4s(3$8ZCvtQ!tJ4z?H6QYES!Hl+h?2uR`udb)k!`I;Nx55Wl>XAi1%6b>9sFmGaF9@MqoI3X%C<2*K0W}K`HSi4OuTh6G-5P{ zUitPM^Rf!&>>F`8FQOGNg-tTcA+2+tdXPhJVjE=V1g@WTf0CfZr2C6+o_HghaumHv zb;LjHS+vRq;m?Ox!4aZ3Q;xp`y+p^0E2P3f&4Y5B{>lUQj1l(HxO7m94~RKiOqX{% zD{g|{U_hnyA*$!l$m!u(gKsO8H7jVqlgVLCI=Cs$2aGo+S%hGBg8KH^G{^J}8;OPL zBc3{ZKHj?BW-!6-|APk1zBwY#9Mqm}k{&W_zFDBxvm`V?Hy?x7fdwZ9Md^s1{2%0k z{J)tS{tJKn^J@yS-~VT91x+c^Ms+Y7e8gB-Scs&Pxcgl_Dk1{;V&LDIOH6ezb9wF7 zY17WmN7DZ>bLpo};#K?wl`Kh#(-s*Sx&ElH6+m_S`%i+SqIkKvX{o46`vfH=w_R#u zKYaK}{dbJzmK)=#GbN4`Xb}_MX?!{q{ch{oKd1ZAE>9e(;D@hhz@KjximNaxh$TrJL<530lO`|dc;K4SmRn{i<+UAku)nl z8=3qq?24e&KOxULgm~s1co^ZeQ&6xO$QI_=vuAy3Js_@2TM`2a-n<;RJ>&dljK-L= zWfpm)o=ckLUo*oTHDZgM>0Q4 zhus8fkgKJrjLl2R+>Nb+)Yo1al+zkBdBtgIn%x@8ZeIfdYz5o$v5TK$=Sg-iD=oEp z>1XyM9sD;@&`AecQ(RS_Uu3!PTH;rxH1p{{XG$e{=Ca2xIyD=9engzVnR&6N(rm2p zLt+E@%a>?w@=cK-BJtp8&M_Z7J#+Kzd7z2Tz(uC~Mq!yZ&BhTTj;(`kWEbQhZ^`U4 z9D}n&dxIK?H;2yNI$>L=bTP$)RDlNurwd`BFxsSzp1GrAmbdirwdU+N(wtcY997bz zc}x!-+kyW`%Rx5WrUBO{7M=e*;p=!Pf)=DnnMb zKH$fdQK@v>Zoe9z;^N{|RmoF%?>nr9+wPw>!UW^Wp72Gv9Pp}!o{~P)LXxo>2J6^9 z6w22_OHPx&Cv#@>xC2E>QAu`ggjPAw)Y#bh%)`;dq+?;C%5~J!S^wyQOOB+bYi%7w zFPn&LtgJCQ)7Ni>m1Hg&I1XzV8b+F4LZ*>j;^>hiBVD`@Lmi(#gfY_Gww+M>Ig~Dr zOWwCLJo=Bz-=93#Fe7_D%kL9U%%AVJX+WP`@`*&5{F-jb1{*Cdjdwq5<#ctbskFvm ztv%96y}fqcJIcGH%4LcA05~$k@{TR%dTf(hI&dBN7LI=3 z+CG1dl){OZ<|D`G9DFqW*)uXp*x5*5|6OZqmFI9=lZu?&adY!M)`C~R_|Juk1+|}e z=n}%eC68z+jS@CCHcn7LJ=vTxf}A^GByrjO9q&Xy_h0b`LZQ|WXs#0=(CBRf9YMLH zo4<$cZq(%rw|PY;m%Wb`l@b#ZVfvw5SilkxA;X;%`(?@`ndkoA0Er|ZasA?*GW~z@ zPm>M;w#+7)gL^(P`R65}Hck@sens@mupnE-XO~I7z{9SE1&N-^y?Ud$fe|({xCnGNb7$|X_6@RwWP#}#8352c)p17e=nI6jgedD_M*o%$n_P~=w_EYU;% z&nvzicAem$Rp{WB!nQk5zm+Qa6>&gIY#)V(S3CGRg;yIo^vN_H&|yN@<6Ly#BQ?w| zNW0EHjO-iUWH039;kFkPruj*rdkCh5%aqH=*VEdHGTqL1BR)9J%hOZHPNlYNN8!Zs zvC2>Nvu8A4BXQYT;^%R|pGG(-!IK9Z-P~eY@xMN^A|Xph2M0<xbP&3)TJxG5n$@I4d%zT|5ZiAI<*G)OBn%~8(0_y6ja ztlP^^Po=DfFT`@geBx3%3s6X4`F-^`K%$6226IG011FMAm3PuWXe7EAN{4eA)>uG$ z+3;fXm<}+|&=s84jdX_jKL}|QOj&2&7Q$enX%}6?==NC%osD0a{<6^+1Cjt?Cm`4p z%RWDrC|~f|s{6%JSOawgR%$g=-TY&vzC&yF`N38E+Ka;t1>B894yxV=xz2{G29F^S zbPFner`xQuejp(BNUYMH=!*o`=3z-0Mwf_waoq{Ti>@w%A0_hfbBoYI&Y$VMZb}(ey z!B?qVg%*2mSwJKbIRy4cL`C7UPA{D>LX*4JS8`3~4tjDB@*e*jnCh&U8f>MQ-rkHx z+nAdx%gJ%Y*m0>_oWM8aF=~OOI1Gh@o{@pUNyF3z7%n+Kw>nn?aR3Hsth-2K@IANV z4~K6@T!85;siJ{;8~(h!Fc#OL-thRbtD4uSv~{KFsZ;w^R3a2^s=K;Q)I|yuUHVxA zUs}lBEBl&h;43X{H7RpZeIktEX*k)deusVE;b@+DmYY|mM1qoY5T0tqq1RC4P;Vdr z(JaAAJMeYCvE7xiv4%?KIa}K+aAesl6`k5ccBfWSdOgK!4^nk>b*E@FH*XL>iB`qV z41Yg#f zJ}h-ymvk#JF*K5SlevY(8S#hn5sEFR+uoW2QqIIp3o+@ zvxzUPr(CnbW6XDaJ{Ey9Oq60Wmp+{rg)|u;>?` z$z-1&)ZUU3g~H1vNv*_p=f_zq)jU z`vT&(^`aOguH#8X&Jc4xLxmy(r?WtPpjWFqrFVb zUu?MNeJY%^35YEmET~wbW>mPD{-457m{qr;*A?0&^}W41G_tM4L`KBx&`>0a+2qE& zNM_S7so5!vU{wAoNE{-kpy}Kg@LbjD?$cKfy@2SG0Bta#Lv*Pv=yUc5)*x3kxH}Ex?r~O9}4o+&TOzTEk`18`yL4HQtyN1 z0W{2AQ*TuT1yMEOYqQ(qHHTJ#{2~FCHwlWfLEkF8{OV*&zq?)1DPvK6TD$f+-Y`2$tC%0F7C6lc- zPf++qpJ_)XYXRWuTQozwA3;^>eHp1rwc1V(2FAw5e*E~6v>TErYLnEavT9sSZC2Z1 z1kGTmh`qzi!e~dc;!fK}o?Zzag9qHHqpe8x;V3@D3~y~t&2P!58nP~N$-Z2Lqwmci z>>epxy5)8YJwJDArI-qjLR*v;JYbnIQ8ba|pe^i|hS#&Sy}KWV6I)*kjYwH72uu$C_>3 zMES6N9ObtfJ7$)R)C57$Z+Fp=aK z{>*Ot(|DseptrQ10P6uMc9 z?~0Vad$FcHt03C*)8Dn{GP=K+f_3FpxskxA-GeOo;bu+V8I)rMJBb-f)`zaz%QopL zzue?{%{3Lq9iC!(u+=9cGZVs{LZ7?4BgSr{SN7^LC+5?4Bkiwnr)c4e#1H`mjF;rN zxW3#)QRuGKS&;|_R>{k;Fe3vMy1HILBwMorYK-U4@6jkJtI-yn*hT2cL+Nfvgwh*^ zd(ioTM5PtU#l;m96&_v_Ai=$izg|9GN`63(os$D?AV9Yi$*~g%>T5s6k1P5@KVaOc34A$IlOE zzzB04Y~@-@@K@!5mGa-g3Z!Yw*1tDn^9M)sRSFBnuFor7AFr3t8n?F8hQy16j979O zD{1)?mB}xcA=@5&VQv(qh5Wuh7^Mzo!9?T2!g#j#WK!2QI_(toge%MXXJd-|R@mYD z;EpS@rzYN2f{PA50hH4d5EyLbc44ufTLQ)crkkO9ox8$$AY=&EeTw$gDZ>4h`!LOe zs&9kc>e03GV<0nnd3NTD)w~qQxbTl!L6X=^H)J;`q8w_JM=5qgy_ve$M6*#9lHUYH zYz%oLU2+K3TU^b{L*SiYz@Qh}iVPMBD3sMa+`}gX;SARc49q+CwskL_o}6lEiXw;H zMoV`^{%@o2WTo$?;C_+{hlQNNfBra%|1BO5zApgT-)sfYHvq@Nn`sHB+Y7Dy$QX;h oW%cihZ~9FFe#l+@v9jKyr}ukiEjtseZeMr!5Kg;5(>Cb80IH2qi~s-t From 346056a393f1ff4e4eb4a17b6a00a44addc5e92f Mon Sep 17 00:00:00 2001 From: Ananya Muddukrishna Date: Tue, 16 Feb 2021 15:29:05 +0100 Subject: [PATCH 11/17] Update to reflect latest implementation Signed-off-by: Ananya Muddukrishna --- .../unique-network-flows-arch-options.png | Bin 24960 -> 0 bytes articles/unique_network_flows.md | 128 ++++++++++-------- 2 files changed, 71 insertions(+), 57 deletions(-) delete mode 100755 articles/unique-network-flows-arch-options.png diff --git a/articles/unique-network-flows-arch-options.png b/articles/unique-network-flows-arch-options.png deleted file mode 100755 index 067e5c7d46f731db28b347cccf9d78096a451a25..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 24960 zcmd42byStx*EVbef^esiT{_gFVa3 zR}L6#{6ydiT{apzZvPy=cOP8F3)-!&q`bn0-K^gi;MH?_Z#I}BQ?@v6)Uhrf1IH%t zs}XdSy2k1EQmn-7^$Zeo$gTxiQHxHc@T-M^^NSaRh3_2riXZz{Ad1%eg-DTeD@MNA zu%gi`w0=1Ao3Bx`qI2b;8l#PVz1^TL{@3G@wPKSe_R8wssTYqJs0pndhCVKQzRYzf zMecUdxUV~N1+m}>tJ-tQ5RpJ3NR?dX7V2PS_nN_K<)|`!5X|I^j~?r3akQ4q5H?N5 ztZEa7Q(f)#BHI_^kH>v+Rg$7uvj9&D^!@o!RQXjM^|zOaP0t?}ve2-N3Z5tCoTzqi z7bkVUJttw?*4Fc-5g*b$Bti#KYYZ10ZrWs39iVL-Q{=z(vVV=8dm}h? zyLcY0-$YNV0145}L!K7Ux}9G3=8AmrYk6V`Lt&~VBnm^yREVT|g`H~5{xJy~)!Uyg z_x1EN$&vvnc1PS_$IZm%aOrL#Q@_ef)5NzKl8eA~VK`s5ZGj2Al8lgkkrcx^v z73#FkvP@2mRh+-}Z34T}JS&FW=IaIr1u}!Jg5A`E$8uk?wFG$+2rixtY{_N$^R7GJ zyJsvRCnc`oHM7%lU&Bx{_189Iehn-f5^~6MKK;qRe6cCM!!WDV>_d(UBrW6*=lbqa;)05Y0}( zQ87j!I9hSYg&bQ(0@Rk|ZZqfDhr`6td`t|jh=TPgJ0k=3Jd2koL zb_Fu{R|N`wp+{no2giM+gxnrqi27aj>2vPvW!&7r-nD3%jlnp7QUohbeUI*}M>d42 zwAB_pwVYYph_AE!O-+)9o6(C)Zr=F95&PS`M)&5cRR60@MhDL2y;uH+FC*0zb)mTA zRF{;zwkwW>u;9j3G$g4^O-dH&nqVqj9*vd$F2l*uqDXBagk3SGiEi*E!+4T_k_ zrwXNmGpJSDw-jV$~ZMg07?PdD3c|GW$x@DI>@@y)<| zXz=DK7Th)u=*rt)%~H73oVsOob0~`y(=K zDzoueAy-$w1jliIOOxmY$H~5%GfDTDi%OKgy_|aMB>Db5ql9`mYnza&#_=HSr}M-g zBEHh{zSk$hB8zqN#|4Fjs|~+yNX-*pW}2U8^GWYuYO4oF6Yd$;?UK~s2^`1X@|kdN zm((rL$vFJ1JpFO&7d2Dk|C*X9h9Gn%s3nc}RuJ4ZJTg%xFP#rY6+lnlmNRd$^IS1* zuW64Z+(WK+ujOJLxstf@c!gBdnsMi+Rlxc5OYSYP@0ynDQy*9i-^HfX&NiF-T-Ic! zrr2QTPW$EIWFr~F-C$jY!zO9wmfQZ;AyxZMG z<(tKn6og{UcJ(nFh%|KHiu-c$+VISwZobf?MCTyA?II20uHkWdg894iX*n5Y6z(A2 z+nX`bmXGh>F>ytdAE;(P@#?H9z&97*_{)dTS>^9}QA$;O-Qh{v0%1Ja0TejAcGbCUk zc(wh{gs%c_E(ECjZ;pteEa;ePvp!mR=_$w*VqlIYtyZ3dlsjsB{%Q_Sblux7+Y5AM zV$_duxj9z8m9&Nj&LHew{BW+49s043@A;1#`)nbBARF{~J>Q$0R@AT>9hUDhQN7rcksf8~&(ufgr z4lPnf^1IT|rOaZ(qp5)F1iilZiAQBr*B~Ck3F=DF6?5wBb&G`YxQv_APJ3lk{u+Y{ zc31K3S{oPt*@ANq$0OyC5bvREr|)gob&UP}w>9p>k3>FvxDlCB#sq7wo{yR~;Dq%C zcKf~ub6;k%^x&fP`lnTynBFHZnDY3`Cti&(*zLnhs(y;{v;AeV1m)Xf;mD&B@08eKXunlCJ9pi3x}LaZk6OBUq!rzIY#1t}4_M@-CDm$@!L& z-jw5ptY8xmxMpf@<*FdUNBR?%BFSWY!#rz*k9y78zi%&NN-5G;EKyQeUgD7{4U&;k zLC4ia)<=(sF^b@UxOfGSrc|NU|H@574e(6V9f5n7rLq{m@L_b`eeD1<4}Jkv=*Cle z2t4I>MaRYWbNyjqWkpL%yXZ0NU@Dqib#`yaeI=TAcd<>>PBqFyw)CZqk~eDr-PR_mZxkSaD9TW@iozJg>bQ5#-59GVs=(v zLt}N$AJ&CbW5#An$!YHFVkcc3p>AnO(-&|y?KEs~tA4;Jc|w09x1N-|Qp)D*PRj2J8FOYN z>We-;YdveJ-{571VQ)s6UL39o3k$QmJXQl$&zTj%JLmKe^u8;;H_`> z^7dqj*X-g zh1X~E+9016>sGPRI(?(4^_gLK_r%C`keg3y@aL<>O`2o;!az4eLt>7)>T0sI{;h7jtC?5$KTA& zr8y`;6FL8U)~7S(6a!0DhVqehlxsxR?YVf>PombNV496mNvW0#Xa zVHqoQ7=Cu?cl&2irm>;|Lj)1Y%g(zY8P#a&z`oj0)yj4zd5YQ=EnLOT{HIGZpZB`1WI3Lcp_Mjd~r zGU`GaOuEXa#dK0!*nU{&D2z7~6%89<(|r+`Zwi6LxM9<*O#6ey{JBf6X2dT%^^IMT zc}==}lx}=G&W6X+y;`k_{RC3qcrkRO$L|>mTEbZ!o_TV{l`N~!)cgMD!Dq8bUBkTW zAyi45OC4`D9z?36!>Go%xD6B1T#kPIm@?Yc^Ca?cejs|@fo5KIG#S^xalgiD3x>T= zt%!>KM0rP1KikMIEL;hrWx4Hmbxc{*fxGBz)<@n@=rmLReNg2HwgWMnXco~&$v4}j zO%N$+orgbdQGYQ?U}`MS*dj1>^fN!LLp_)#V!-OQx$NEyL?CG=7xdgsigm%0W)6>2 zIQ@zyime@z3FAR&p6fZHgP_X}?Dh5avHg_U=MoYkvbQKsRKoG{K6}TIdW?U~M@c>) zCVII+nHTH&fxeB{EDggK`{9Hbz4~B~MLeTLZTP(rZcxE4PGNHY;sjUJF*6S61Z4)xHtCj2X zg8IE1iw+x9frOQto4fZR$s@VSmZm2C%6ac!>E~WkhMv7W3kycEAz!^sUG61E>OH20 zIBGssfQe<6}J;g_QGyw{PDvYL_a0e1zmzZMch=$M(ml)vsYLUR3mg zThZL(59+!Gt`w1=2TDX|r$jHMvOHViv-j)+F&KTHQz1p>e~I!g(b}7OWnl|K~<8?Tw#o4^+`fnUGzq+%owc33XzLXtF{p{ zIN^N^Hf?8VHU~zey6pFsR{VyImxUZix~3k)Nd2xfAW-E3o=YHpWI2aMAX8BIg3shg zX#b;JcYowlx-K9FHJaR%G=8kv8Mf(C0tr84YsPKUll6K-#B$4@tT zzIyrcgl%ylRc-COdH0*Ei-ZKs@Qz3v#<)-AbLOtL5i_NyF)=Zh zej-Fh1_n&L5Am4w+;->qsaCo9A3l62OBtkF@096TzzLm?tl}S@zc~7}^2K0788L#G z4&w6!0l^jgxVT*T2WV?gftQ%+F)=yW*HZTS&#CFF@_2P6rNQ0#CR!R&nG(Y$+=!Fy zDN0-MtcC`DK|y_3%H*V$eMTgow-qQca&mIk96b*K4#Y+v`($lv%fQGe`I++aJ_iQ} zmGkCkPH}OuNjJT)u;HhJrCTV!Yuv`~-{6>3R&%r8(fT0b_qfvhe5acrm*B3=@9~s; zSHqc-L}&mystY+EuF6TN%0DS)wS1-pE#_YnA6e^9=H%qe)U2}l@j`%`8|OAtzmka3 z-1*Q;AMF=26H_n;vu>qS$vZM|fwAhsn0|G^jo?A`yqp|4iB96!z$kS{(pLEM-WM4^ z!q7{qtFdjQAo_KV@2h2;Q~QF#FE4#`5e|kW(Hev;qc*)C;z^+=gg;DF=snpcwzIWe zs{8@v=VbT(c-D7L+Tr2hxU{sW z+brClKYtE0#KOks=c7%KLCbXWv&t%&#gKvH-6gA0bBaPQ5_tt18$INbgov0}`gf2u zJ@}!dd^wh@w6d~NX=iVb((!DZj+OP5mzNHYmWIZcFJHXkv62i74BmakU@j=zMSb>+ z{4J9lK0dzK<#>Vm`e^4N^_*srmM$qbv5*(K{$HrlknnFzJ5u9}f(}qDI}5}QO0oym z4b^J4y&=LeJd&Oue07Am&xIBCG0$$!ZU%i1^f(m9b(1 z6yEjOE{D&%vwVSFR5ZZLi|@Ws5AQ)+D>8fE1A_f88>=}A$uBy=BE@$r!Oo*USvpwh zc`cIhW#!u4kH;aL+c`Dm%0}Fc4?F>A9O%W+F0cK)w?~7+rVeZjMKPI~nfZnXk#d3>C`H&YP&E#9t_Q|ZQ$Rq{pX1{Z+mtM3~g--Pu>%( zPaQNwB>IpiF28>p?mGbc^13dw^D0ZCF5wx@*lrE?iC1r*I-vg{C?c|}3PtCVy_n3G(2qF<6NJLb@Gcb$=5shtpE`Ay^7{JH zbR6Yfa(WV>E-oG@WO{fgAfo&HjSwM^{XQLJxFIG6m8v+*x__i~b~tibI%G0oq6wXn z95Ta8cX@rYGh3&u$l}SxYh`Usw`v9NTgFf@Y567%n>>`fe2Fudo{4o0&&+J8LdkH} z*498whxd-ZaNwWuB-CSX&R4-PdyQvC)g|}miL`DC@2iF8(^C)de)ROJi%vl5bVXGK zAqpzS90!!lbZHFVu#@>vNKu2aaT%!hUHoFN4Poz~= zgHk+$3Axpzxuf0;6_zd_#KAy}o1wE%$C{v>9v}1VElEFc|LR~F1o8Oe zAgm*EgZky>#+a$~S9|fI=My!My1@e|KxLV9dQN|#d;#4d9VCJtauwCocDmT8glUCJ zlndHNXL&Ja*vj#FBy^_DN55xIbfeG6dn>|<$|dVDQiOOtS? z_e`v|hATS{)VoHstD28rk{aHP!lZI_q4*5of?0g=(?OY-pyelE`0q=UMJnbMcn(Y# z(BW>({4eX;ndZNr^uMm_181M+70V8gyTA(tt=#JB>fGF%!))!xegZ*;L)H)ji~HQaH#gaTof)~{GZK(%B!^Zg?zww#x{Vjw*7&QbsoC0YZA}z` z?oiTt9^^S^CV5!}g`&d3=L7^u{W^&KkJ}$(sD~iOuMqPhn-$moUyABJtK(&o7wk); zn_gG%;IL{@VM#4rb9VOE;((dSbkTs;#>UIz%`w>(fxVp_!z#;XJdoE+!D|ne)LcLc zDQF)onWl*a*?=5Hw&b6h3o&tyl)Aup%1$|sGh~L5@mT*E;=o9@j00iat3{HA3NiTf zv*|b;z`EVt-DeC_3_m0#nRJ!vR7hB+Zt+CT?%-{Q6B84kT=?-d*VpUpVleAfZ_G8g z%A%m5QO>rvznOFStNf1nJhJmw^#AoEW6uK(onU5mcG<0%0LuJ0-^kxzgoh$@y2>in z7CSZZ!-rbWJtKFuo#p8!*Iz^69Ypjt8BM1Fgp!=>x%wT$ss#GlVq*)`XY!!nWRIjlaGOHxG~FeB(rR=HN-4u5r!GPrBZ6yQxxs*Y);C>FMcVsKkA} zz3lk?mlz#bJ_`-~w%2D1zI&7Uueu2|e=&iY8AQ&gkj&zq+RKL9oB^n6XNu*DrKKgZ z7c@k{i+g)pTRwnJ7%!cf(W67a+z*{5v|rUwSFiFtb98n9HgM=`U_=oKn(=w+LtyZ)+64f0aZNUY#Glzx`?_V8XWU zMaP^9@_0#kIXcF>pE_C=ofcJ&^8!_RpP9cSv48tna}alL&_iqbuQ6de0MNa_u8*$% z>U^%UlH;BE2wGfxJUJ4|{2fX`Qh(`<7G~QR=gormIBcFQxV2eyl zrDSIkrup)1YDk3}Xn3R{(DNL?4Ar6|FO7>x8MlH{O zQfWN(y`H}`p3VRKZfzuHh%>9A78#}SQ*tuiK6Y)z(t+DZ7)%NitDbOnKy`kSS9#!J zBW;dA){9CE5!5lYfv)%$tJUt?T5`VFd2h}FB(P{56p)`jRcR?LEd@M-*qW3Hxv~XY z2rucbkl^L=N2^c0Lqi^xlc(?XuQ944!ZeY$sx>Eu)+}E6`ue)N*PQ%TFK(;7eaOSW z1Pk^r|8yP2+8W1M=(JzaLn7#ZRez%6H&)y9d=^}kr|G{LkAV9y4i5Ce8H0|y>KGHuY z^>7}o#r`Gm`jQJ8e{z61S~+aD@+}%Zft{;{g*!B*Gp#N~Z9*(FExu==&j?X8rAxZ# z&rVK$e&P?jy1G)&gLuQ=uH7F|lk8jyi0y%ZK!=aV^y3K7NEC)yRVMiX>?T4Hc&dkI z?L~mRpe*A|K_&DqCWgYJ1BIQQ9;vELDUGM2ryz|V(UFbm=+6=Yoq$$ZCqHK+briV%hP9%Ip3yDX34=<|{xm16ByhZ+Nkik*}Hxg1wx%ZmKVXrZJAY@br zPb>j-4LG{G9{k0Yt?jx*FTh2^o3~>MAJ93H2Yb=NVAU^sVg$`FzyBeo`ULQ|+)^OY zPIxijfwRCJ8i(SZN1cJ&nW4fpzdMl1rJeZcHd^&{HM|9SqRX=$KlcS$P`%Cd2_NPCmv6dDVK6`{P%WPr`df?bf3q z!OV%Kva(lzp{1jure6Exy9?)KAJnSL%FB1E0#n zp-g6}3)xslr=Lxsgd#gZ!k(&r<;1O9pcTrGLAujby>YE8!KOuFu1x z*Rw(Q9u*mBPw32KzIifbEb4d8R_MIfa^};i3cU! z#5#2+OM)o&7Gy3LEX~?!+o+$yJ^=4~D~n!NpD?Y5rX77&)L>#{EGjJ>O1(|@TStzkJ2O@@vP}>sDL5;#mVLEC`3|P0iF z>(7ab90}*##+2&uS8)@MbrckkN82tZ%F2m}*SJ2``VP{#dIa^@%|=Z;s2x#xR_E`^ z*uwv7IMvcGO>F9*VAKxWYGTkiQh8mhC1*20%Ard9P!1yi zeC(!JsDNm(_ZUhq_DgD?>^Dg_(z^HYh)jIzuTQ?sD0&9*_)latG&Knhzq*ni-2B;V zyWA?0_1o58RP!bES_;IXh3Hs#?ab70G@9p&9)72s9Q|y{ftYElCMS7Eec#uv(vU2) zV%D1IPyjS?3{IUSiT}B&pS`UuX2#*tTQr|Go?;uxG5e3~wG)rfZ)-Az^dfiQ^zN`R zj+tXU|Bsfl1x5?`C+~057O!Zybm(&$^GM4TOCnmLumlWHgZ16k#ltA)=jNE^aa5pl zdOJBHi?=st7E&*t2S0kj$dmQx3wyE*oZr`A!L0IfxNw}eovGgdn}th+bSTF4;|}aJLRw*`4f7X&vi->D2G*w z$<@(_mnbOrwXu@;!#5dhMl6DQoLPc3N`?q>QGh^CZVOTXzH(3d93Nzz_+ z@1WZU2ESyr3_`|bYM0rY3gvdNPGycZC0nFUPRr+bRWqUGgZ51PadAirq2th7I;q;1 zK^9G&7!OS&Bk3CqN=f?1Q>UBwUSA#b)E14W%1mfoihky6sF_iCy}A%Dkj&@fJc*d9 zN@kmqm`FKwFG$*66l8@Gr@CG4w1J;U3uQZRw2-G?NOI3z4wewrnIN}Y%{U4HT2;(4 z_`EIP*5B?T1xCi4K=bm^{p$Qm1N(S&nl1-OL&Rsk-c`t8wuSTetBa$W&~a4ES+n~U zGVRnjmLW$R({o$dqv0;*Lf?fAFmUQdrM{H-6qHagQ5n(Xch*dx>ICV_c*8XlXNWp9 z?B2kNdG?VNV))Z|3~`-9%;eGwF=01@dY(@yI&EYaq^FBm54gs zNj_9}wBFx-+0tfKQ{#mu^@`)CHbk8tz0TXIoCeu6-=`uqna9T)^;Zs&!wrB?QV)g+ zPfVmeBFZDSrgM$Y76U{_IcyRS=y=hnyfn`soxZ$3a5<&rpm3olqGm07E=$!@^^3a5 z)xP8=6ABye+xx*kHmnEAEH=Mubjca^%;;vOzBgG-V6(h4PDe`lwEw6!)<{sI!3t2^eo z__}U@Zeoza!6DuGZE|SST0HF!I+ah&g;({3)NP6rZyWugG^hUQz7fGHt<31ojAJFC zMgy7lv=)UX(sL9dSy3ujij-2N)r^I6jZ;y*L4$iEwnj7f#e5C#S3Vd^8BX%)c+P{o z1Gs{6F<-)A2We6wFR%BTcx~q)oR*PWYAyqA5T&5jau#ZBWotGBsjSAv86Wj-k35N8 zdO=^>U}}$iG>;aLA`aWXdN`A5;31o0n%2Mhz^qRb#bj~OHa0LLcwW|Wn#wIuE6go! zP4~C15G0h+VIW?(?8lb<7xfOG2T=Uaz0UVdz47gQhkiBl4E%tiC{-D0Ok(6)LbS|p zrbkC9x}R9c)CpM!8MG&qH>3fO((8ch5K?1dtMvO?fHyBEVdFk~99$#OFKXEc6f%-H zTH}9uRp7pv*cct~k1k@h>^Q&OZQikdX3fp&a7+Q2?mo@NCXC z)(7R)V5~iZd6I8R3Rz*tgQhU;k1j4Q)qU7Gtjk$7DOtISLBYrIHCWEVy}aG4s?pZm z9B?6f=wrO=jyxhPnBaaXv!VdHfMwvhq}=|Qr3a-v0|IMN1BK7;clZ7zH%eVG=!FSMwJHQ|$&q+N z$sV@v%&oG6dUg?M6pV*ekRxU1HGp$;W;Ez3~A-iH?CG=)%0Cw9)wZ>gY$d z@A4b_DAr<}$qxnG6=&OX&<`h(8}~JSoVE}+d~_^1rGS(Z(}-2k1g`6%#h1MhzBymn zDs9!vwfkD)+mgbhgXLQdUqa3nlN`@pudQEFd0_@>H?3M0FfmrG_25|O8&e`1Oy~tV z%@0cMWfBg$a5{(@b78eoL(Gn5;>y4B+>f-}UNW#o_ShTf+TjOAivjDf@CI56jSCyx z9pM(lKsxi;S_i%+AqiuUSRFcc(<}N`J$PBQy5x3GM)assr$Xm0n>KUPRxMS>y$-tZ z0MxUDKF6VBVVPWjKy7MG&W5y*89B{^tu1Hh4S<&u9PR5(n%1Ixe> z@wK<-meYY?>dt-v5_^rIW_v$f>EDzyM$CP<4sz);d}$8URMBr&Bx~ zNIm!93PKMI8z~_%(bvbP)Sv-#O}!_CMiq*<+^z`B2fns=$gnR%vd&LC07d$dpAE~8 zzCj$iYRC)Bp;N}!76O((JKvG~as8`WXP1g6QH9VxuYiMZ5GWJ6GU2OKgxie+xmzgZlRL^l0iy zkg{UDJ3&_^R;B#8adeTInkuiL@DCvn#_r;h`_}e*wN4y52PcAM5G(cG-d@=A*ZUm^ zvbQ^nH_VSd*VRo$@d+OfO9GrHYd>8s?w}FXSwd)MX9r}KqaW_pc8p9+Ag4@BOn_G^ z{vfCKzf=`?2M>rQ$s!6WNG7+r9+NC$g!q)a^HWK*+PbZd!xnc_p$!KNqR>V14ooPif$h7?ZVrmoAb5D zo@l(kC;ldpk?0^8uWgeP5<2bNBB$!E++VjGC-+ZYF*!7F#=(LR;iPmp8K(qv_0_F* z8x~s}@Lf4!{;DOph3)R;lwD9cueY0`N=qP8(sEp@hdr$N;AvTU{z}v@HU7>h3V35w z1Hqum1D0kxWS6^K3o`*r4aw0EC1ntyij1A2H`|- zLFvwcb+@Xere5TlbO5;(kjq6ha!N|VE~~QRwt%o)7v|jhn2?~Rs~h11xgIZ!D`NUg z>f_@h!B`IXxVk#+*FrwW8-PuoPer%a4Egef3Lmi>$eunnsnKAy!gP}2?r1k|4vwrr zj{}%=+hRt1eBi?Ozl7eNPgHQ18dBQh<*4P7B$?gczoEju9svOXZnp0~e%utUqU!-& zZ6^!Fu>}Jgw1^9^?kH8^i+aBcm+(M+EQun9_6iW1mfmBhKY)T-R8*v#ArgHIG|0?s zyUF5?7gmrBEzn^ck_n55wAx?-#u;eVvKq{Yo)MeP?#RbB%bg)31)3`UN{qD2IJ6_Q z$8y#dj&t=Dt*s;xfJov2A=n=)7%Op(V_zm=fwdS8oCt5~JVo+#+!34n$ryd85cfPvuT<;B|!3h z^ks(JUU*p7Ro|0f>*4m7oE&~nUHxFTT|gE~FK<0rN}yJt!mq3Zx$9Q}#w}Q!kDK)d z@+d2SO5c^iM-LodYYXG8um4>6w+TENbE=oMi(CFsRZ_Aunxg0U3TsJ^N6l0B~12nMFn0(Yy~y zrA*AyzGN8cN8edpS65GZC$gk_JwGr8hCp3i0#xi4v@JKV zs$|b8X?Ns#9zC)|vlv3U)zsETSQA6c!Fl3ShH@{rrSjSy`9pA)kaI=oqHK z_|%h@g2KIe%E>xw`38NeAWB;g1-jnw146deaeiiE;-mwC)cQKL_ED9Ub|IxL72#_U z(%h?@oE%w7mrgnA9aU&7PbjG0zZxIGY|@QnLMnnhMa~iT)iDi zcM8-RCtLqEV}~%5d_j%705CLUwqsNYI9bij%p?bad@IddSyh#(Nz&ardG(nOZ0 zQ}Hs)$L!a&8`Y^HBZ$$)$JAQG{>V5($Q=kg>p+-n%f-oi!;it)VN7WT;!bOpk%3|S zB^vzg5D*hHv#@+45C4#qLHzu=#1q{FHa505#|7$n134jU3iU-=C6yw!0$%%Mwe{&5 z`bX$pa`s_5c{Mc?GZgkedV4`tZ*W=;>J#GNPy?nnj$Ys=r0sv6o+mbsZbzXfFE1}a zRc$jyqew5oM+`U4!fmSvG~E$ScM5&QJv^zdzkH@3w}R4)iS$2c!c>xTKH-G}>yCwm zh0#$p*1f?aNRn}Z$@SLz2Gsb{luW`uudi?~xhlb{b0F zTDoLtVpihBHpAV$VO{Jdu=9fzkqqV0`-^FKf2;#~DPDBhm>u%oa}f2A;R9v`Y}%On z&%k>$;^4yAXz|9{6;;{sL(4qj^+gZ1FgFLzm!6%lUTRbSgU=VyL0ciglZ&V_SuHJ~ z^EcFIcc_62>BFsTD*nlI%*@ZS933Hcc8mcLF>qO)=wTr9oiI-z5W6>6(t%#_{ zU`3Yf$?u;p2&E-^wuTK~gbH^FTRL;Jc@1B^5dSYxUi?pI`@n$WMfwRc9!pBv%h$64 zcP3z1T|HidrV`G2E zq5&_^)prjn{_wLa92n`KjeNk`?|vJ~`_y*P93KgO>hRv#tHsvWW-blc4g8Xll78z_ zAUYmBiE#&(9ISadpECK<57T|Bj+5+6@D~4 z(aeODoS+Wndm^M;o|o6-jlhbTPw2>r<;D@99}9n(!GGJc&0J6v_ehn*fEWDANnEO# zvjrEpt1<21_8KHWI%B#51|Cj61AuX7yZY-BmoWUxUms%tNCCQvkp?!8J$nyEsqML%Me?rK?!m3!;Ye<52Jsna3V*K}aAuuVkAXnk*t7UDt zKHp`w$~9MBw_T@AQr{SUDSju}n?(T+!uE7w^iFSGP2Lp=+`OVr6a^f)U4jr~frs>` znB<&2H7@l05$?$HWYSnz9N7l&G!*uj@)8*Xj<&iZ{u^N8Vup|Sj>rG_1}-B|#ee?G zqz>og;eImU3)l>d9b37R1K^%^_^Q71t^?Ek%Qrwe0DkpMy?uaZ{M$q+DJl8+`FVLg zK|8qa^6lHV>RCVvgRlTrE$92nEwy*RD-|!;zAI95%B3Gvl_qf@$>T= zqY8E5;N+~CmF9rB-1+|?>7%ojzE|j#Q>^kjfHgJ>?63^5D9FpR znQ0g6kaH@zr*-d~M!9$W%Tae0j{CQxE;KAG-oeB~%&~#VpnkY%+(tNvTa=~^G@YJJ z2;nHeY~&;`i8A!zfd;FnC@CR^B*=U++)oGP6clEgJj(}RmBZ}>ASHD1PKA75gPFoC8c!FFR$mA_uT%m80!xaTLm&ea z#n^9saU=#^vB%~p^~Z!L%j472NYd)%-;9z9070wXU+wW8)C}2{fkUeACd^oe`@nDK z&HI-$UNcjo%EmkXz#VjcQjcEpe9-*p^s`vMZm1a78renSowJ975S3U}Iu6gf$9s9% zAZtL_PC|b$Gctaa#l5}V1#Ak{t$+K&PsyCY2j_DVr10nePlT>77s+# zH)kl>d<8a@`VcE%N~l?*t>N-%%f1-Cv4HA{4jP+)l{NKc^<%7}0r}^n|*cq)52w>Bu-Se+gf~?Aytw zgx`YB#z0YzKm9PeE~ubi2Z)#WZ@+PFmusOcuNYvf?7xEh zQ^h9K&eK7p=F@kgz8~A#90=1_4$fMAg7;j6i_5(9y?}K4Wf63yurPHR)wsonyHYM6 z`}DHuq^(v}oS;9OINeI>r2`(moA4b(w13&LvPlO~J=29tnT0E6;{X2?^qp7l(r@)+ z;I#z753nPpr@P1R>C8{};QrLp+S*!RgXWghsM8wQMcP#+(GXJ|rwj`X4V{}axRPlE zZPh1Ky|s~_?w+2M)19~FQHixVd{diadCTvLW0T`PgcuR>I@ahg3f2VUn1xDjE^ zc?6hUdF`i7@)>NYAq{0^KTZdbR)77<5OD9y&U`p&JbQbekQ>KYEoy26X|n1oi!70_ zkA|>NV$SNJ|~(zfmdLPBo4PA+mTn0+`;2!`z{5emJVGkg$G9kZSWkO%$O_OQPw zN96x#=Ks&NHg%HZH}J^;3EMVde0&`Ah$&GZdR-ZT6q*BUMog?(T5#js2{6tr#m2-B$E9-TlZt{4B?-u@P8vU;T!%WoJaY5wnrcSOaQ*B z2w*4!4DVw`2xzP^gO}m)#j<%8mpDcaZ_Z3#RcMU)b7mTeOITFN+~z$p?~;rn6eWx>zK574mUb zcV5-x0MGYSLqvQ7nX-tct5UhHTl=F^H~a4c3};))VSuT;JxOdL;Kz`_)PjOmlBKHv zkeIErq#G$m2xYI&L}Q2q4$}JDa)FuE0`vzcyRpftOi}7!Z2?Y^2Z0qhSChU^)p2S7 zyDJ=se|P&goftelB>2C@$F1kte}F=R19Kn zLJ@Eu3Y?{I_QYTwl)qZ-D{?Cy*9FY($8T zDS%ti6^q8>VK~Fe60ks0>Whs6_$V?mk_VjgQKV}{CUwu~YmrvmQ z+|MpbW6|}*wZTB;W>Ua&=VLZ2LEn%Du2CXhD6Z*=cW9crcm&4v7qRCjP-M!ncn4qzb7 zczvqt*Y&Nyz#Hs$;b=KA3MUGR>u=m{+%8Qd54@Evx)Qw{iH*~#b0i*(j*ga9V44^n ze#5*yRVH&8LHjA?qw~OVEu8?2>Ur$lwo#kkN1K>=8@wL>Y#U8ihgwGR%{zU4wzW%e z4mMjJU=Wr}bs)1gJMGj=vFO(-QZQ_9c{zMUfAafs*RyT=b#nw?fxc^H5n!F`>+wnN zoHqcSGH@i)$v@iMZQA>W3;0tv8JVyhw-i1~^WMI`!lI(8PnTf(Tvs3;d~@jMK3f-N zZvjEUOurN#h0nNbqC@^*N7d2_h=aFd21s;@kEKmuhgg>bu>moaSGJ{qZud&=PL%ZcVcxoM6PU6gg^-RPjZlD*y1 zZtL`LOX~SX;gujK*G}SUR63-`KWHUwCm##@LM(ni`g~knIn(C;ksTq6FZ(=H(39n9 zye}F_nZ83{hTkDlBuB>3OUY_#liyz{Hb@7Q9J(1?y*9Jr2=S5FQ0O7*IJDEfJo!)z zMX(d7)Uc@-OY<$qV1}si>a(dxPiLRY%{=Mo;wJz7x1^tG>OBr_K@@9VIREU3?=6Kp zvXtHwb>e!N|L*qqlJ0m6jK#$HfQ;pyidHZNJioQFX);6;-!`fY_+(Y8;vR4&DE82E)WlFM=nnO zYftL)Oy(;@%yKfgVFlw4#p_?`o`v*}vy2=rp4?Z#vsDq!EqOoYr`4P?>KC6T6UX^% zF;YY(tFW*y&Un`8=b!e6Xiz@qPJxkQmw8*{;GucfAHwTn-uB_Oj}$S!n5miCRrC=V zHPWp<2_+u*ME)zB8%z);?fJnQSa3li*m$>y0ya**1I^Ex+k?8mwP2+fK`#Z7 z0N<^8v?;_RrT;+jO?3YIszv*e*oG(xM-!?fq}yyt=7?r8iCj$ETDol*`n7c#Qc0Vu z@A?;PmZg>zK=GP~F61{wRVuKEs1h5z?fmf`e7mdu%a-w;=SljkBn_+REpkr9Gz43P zNB;WCU#Rm*T#Mc>C!Cte$08F{=W%`+yr_AW(dS;oj9#jB<)xB0#zSR-m)? zDK`l`<@r1tDN^0(EugCw82a6nTQlj(C(Hq6iH~M<*gr4bOVfg(p_;PR0=BKj>1=lm z7gUmDlUh*)A37{&>wk3QJ|0!&pwwcIOzrLCdO$iNo!>j5ZOrVFV;H)kRWCIP1Y0AY z0fiD2*lKr@rL#*6Srvxn$&%dQFBcxa$f=SNv2$JT-EEIX6G9l1xGoi)v&#n}y@y}_ zK%cpn+WG544SGK68y z1G_xgwzzb^y*?ux%5gZ?DAl(L^l!WErOj}gCA0@=2>`Hx*c`JV8Qqx|{Qsx5GY^OA z@85XKP$K(QL_{b{*>@#EWX+acB4ingP$bHlCCN6jB>UKsT}mTsWC@KeQ*9eeprrC$_8#J*Q!eSqhd@PlbQc}0hN;LvMe z9XnlRXkvwq9I4kZHb~FQWdQzD*URP<&=pG_&$HSW^x(~3oeZ?G^h($ia(giO7AW|t{|oDdvY45=w8q&_A} z*f=d@y&*W(q2(JyYH!>|cZt$99=D5&P;oz7gzS^)DcT~dNYww?L$I$eZ zRgDzL#g&cfRXQh=SZma-Ee+i9rOf5P(^}G8$-nMpq0gTxNjx8~R`&!atS%^rEI(#_ zD`<+FuoDk)F!c})Io#>jE~>|<)P_mB?P>*zUwmfs1t(nax1?`LGd8q!a4~^}0^lvK zA{@Bkh9s9~`(B^aj?A`Lix>&#jt+0tJltt_-rxkT9GT8ScFfJ69WbKFD#F|W_qSf8 zyTN`+YZfsn$X1_YJ`qG07b6gbH}J;)2S~bCbj8!N)ra>4&_dCNRMI`Bii?X=x6I&u z4-D_;D&~)PpHv9^*$`J!QZhVjzBo78`6$n@(8E!&NMFBBZTHT!ey}x#&OSje%^0(W z(ROup<>uzX7*hI9(7a}Hc~wMM_zeqWIN%(R`)~|TYw&8?+}UA4q4G%9&CYHDWS7;| z)kp42u)e}w0a=nw1G+askRE8 z8HaAH%wL?GOcZl0>P(=!2NL=%S=GqMNS8#e@Y_V;``Y~X_;!=>6Df>>rp(Dtc8%RD zb)z*$)80@_2{ai0JNR2+TC~6$Xz^m{Na^+$7^e?Qvjb1#0tHCVXlYShfQ=D|(;O`U ziTDe;rn=8Tpfawe@$|K%%Z2}VzWdWu)nvI#csiQ-MR7o9dvxK5StnhtH zY*tG?B{*3Ku#|xSOVol}z*5tyuWMIGvy6^rrI>26M;M)5-3yXOCa$`()Madw`IXpe zibemcaB4Bt`2vc}h_;ySdUdBOh!3=lM#fcFau#}mHvoDqHB>j+GiPq6h~7hiqyd~6 zMLUcj$nl#KEOP1ZPB8YraDwA%+`0z`+7kz=TCwtz9->Y|Rab3QBL8gzx9o<86k-EY z$Ny#nyM_fR9>w`2+s%U&=C#G3;bm_N@HaIzMWJLh4GbQ%WM+13cXvoBymb4d$*q-a zu;KaGH!kOLH)(Vt_Vc87GsS6l?mt4kMU4?3=E{8k6R$H?*}kVCV|6 z3JNG&{Hf0zr9f;^YdO=OhtEMyhfdIqNc-qUZhT#DPtu95zy>WaGBPSAfW}KOi;Zpc zq*D$ss(M#%ELkc*>An=k3=0 zCrxeR=&OyyIRG&yM0fI{_O(C$IGz}EP}9Nb2LL+x0{9B_2i!+bY)`k0F^RmaEI)hn zi6E0c>uh0A7Y`O&hk-f)`_5Bx#pLS9;{kF#BzMR;2?qkU{9)f+TmEod#y|MObmV{d z!(ps{_lISNt1vBbu7=q27jN`jflT>jv`kLfI7e_yKc&?fN5{OU%KZ;xV!(%S1B@)z z%f==H7@Of)pOG8O%_XNN__QT(iMy|WUQCUNC(VaWc7(ysz|&&Ak*~U`PY9 z!KcLIr62ib1t~QlhML;-J$v@#*M0ji_f|<<=*#uDF^jv#sei^@i zx-Inv58WoU-J3$)BfiV*F6D&23YN1(*%66*b zH3Q8*yAHcI)A+se2N0A9>z`kITkk|6?TsN)_BK4AXc3*^27A(AI%JXQb_5}sgJwTm>Y5Yv z{kQw5IF3t*@@ifYq~{@eypwq3&OomIWb(#sh3t@31vDD*Kv1Rc0agP6hwHN`LRton z4Q1gi)`vDk(tkk0jY|={vh+EsECtD>Mym2GdgW0SR=8B)Q05qqthtOt;Dso_)&aYk zAa*VtF|9*g&;$^_vo?Gp=YlNA)MWcw9)G+dIJcMoJE-X?&farlET4TpIX;V!YUrU- zq5d;jmnOal ziIV*LMO??h82ht7MQWqJKiYZ|xaY_B@d$ag6@0l;b<6i&8a3}u zs}c>r=w0|EDeaMge%@jXxx&eqnUJj~4$Ld-LS4Ds~q-%TQHp&y3ntXIKl zx*bFcGqW48lw_cjmzS0bjGvIQWXUtam#5Ub6aOE;;GA3({N^0F2XgrOhhH*{juA)24T1%gY@-%bhSbaP{4<;lf z$sG6UBlm&x2YIvzRw+V$EYQ(rwQzSUMSZ26Vv7UFxEZLnD7vBHkr9{aY;gFoCgt9u zrIqXA(|HCmEZqA%X7OTn?X2;sPo6A5zY4fNq!)}8IIk&o$aorKH$b2n@}9;@KAOTn z)=c%7%|%#MHAVJqnwRN&Hzzt)R#se7V8GYaH8V3aDE_&Fhgu`@u@gInseJXt`9Xbf`E_cWT7MS$gNpu)Q@#il(5X+C?~{D+$H-1@~sXjn~c ziXM=8U7Jh`s$r3~T*Iz`KiNWXbhOwYUbjHR4`j|W3~ zj*Q!Rv~Kw>?55Ej_6&+ubTG8WnDq(S&mYy=Sd$AlAX;XGE;(C^)bzZpe96@IP>#jE z&3umCx%JaLFvx7~nictE=LBw#O zOAaZawuyAu*xAX5IIgx}*LUM_FLOz*;&UA)=1a<9TtO(ACB97ymVpti zs9@Bw4gz)4K7C0jb^ZIu`4SvhX=K(6Y`g^o42Z`tVn$a>%XrLaYu$wF>Y@)^hE`T- zi5$5mM{So;%RE#^QhBGM;Y^%zb{1SeWM*MOrL_w?KRE+q1kL49Kd5#Z(8Ww6p~cg^ zjq0kciiLMsBoe90AMTFv%qdH%a6pcFzT(pI=OL&~Id+cWDQ(suMALo7Q7}VkB}33F4ToG(?xoi)cb?4zhu@a2JJDR1AUIg<0Km#8hiT^+= z2W)R~g7F-Oxw);D${`~7UGl2Pt`~dnTr@X-xPJgb|M;;Q&1Ej~0awX%?lE$3!K18P zU0to`$K|_Pl&fMHd*WPAAw)h+@N>QkKSl-zFPk7eE3zndVljYv9KZ1`w$p6b%?|;> z^06229N`KSKBv!ydEe7D+ABP&)kz(RT&7MNyDs+tnPo#;3_-oT#TjIF4am%C-wp{N z$N}-i3<6{||3=fqs=S!G=M%k!?~)fivbNGGW~MN&IPaZfW?0-vdY$rY-ax4XS;}FF zbtQc3g1VHp?~ei&4EIXREYhkXOA|U|g@;x(KP2YI>dJ<>$NQ;tRTR?3vYUe=H0Gx= zClZ+#3|1{g*DI{gxp;3N-WCasm8;XYFLQ#LGbqJO1-B~Sh7P}p$c-P(L$p$8J}EPn zMlu1_rqYn^jym5y|I&fTI0a$1Nt}z|hIBYmAKG^sjtM0(iAmG#N5+_fyRe>;IlqB& zmQcO;O=FvjtK};bCoHgS+W$CxZ)UQ#rtjNLzfIqpjbwk-hyv@505NTDZ=YKfKqfvi2 z$zXQTXW{61c=vF!O*2k**WV_uZMwrGMOG*oBA?b?lL{<(7W5foqy8rjGSa!~tTo#f z@r)QO!V#0;e*ai1_GPG=>5nxcnLwWia64M=SmphigQ=3nacoTw!ow-lOslT&9X)!~ z2n~uz+(C+siM9l2;_4_WDWMJ7N@^ecmDHxsdV}yF(}2bU+yR(#N5!MLTLQ|VC^2Tu>S?+~rc zppgQlLBETw!TyzCFcQi`C!QqOm|zeb=)bU)iO_}K^{|-HoMap)-la0)?p=wRq{sQCFq|>P*?vhG>YUQ^@d0W8^vy)94FjZTh$C$4KD0oRNe~x1slp8GSQ$M)b*sF?bP+Rwo8y@jg zne8a3gt!J#b_!vDxJKNwr67t7I7P2_qf06k)*Bi|S|#qQ6E=$%R4MA=?xW^0KRrh) z)oiiqvNo)4v9LCF=ryer;#H5oM;0&JK*`ySmhxquktGX1msy^=Yim%va2Cp&kiH!# zcreLU+WNu1EMm({=6(i63EuoKVcg}?97TM54l>oP(gm<>Is9*$D`_DUFZCBCQw&6f z(7TW#1By2hO`&=_6rLijy<87=*+N}9to55gYIu;q`@lBA=~pxrwjKzZ_9yFN{eLO9 zPGSi0SC#PZu}jcEwV{#eS9}|0ZRo@L_OkKQ93_(Iw%q|+mB1+69paGw`=kyBAC)t| bvHhjS!)p$OC6VyA5Vol)Ybq5eTHg8>I2C11 diff --git a/articles/unique_network_flows.md b/articles/unique_network_flows.md index a398ff95f..8f1bdcf46 100644 --- a/articles/unique_network_flows.md +++ b/articles/unique_network_flows.md @@ -1,8 +1,8 @@ --- layout: default -title: Unique Network Flow Identifiers +title: Unique Network Flows permalink: articles/unique_network_flows.html -abstract: Enable unique network flow identifiers for publishers and subscribers in communicating nodes +abstract: Enable unique network flow identifiers for publishers and subscriptions in communicating nodes author: '[Ananya Muddukrishna, Ericsson AB](https://github.com/anamud)' published: true --- @@ -17,11 +17,11 @@ published: true Original Author: {{ page.author }} -# Unique Network Flow Identifiers +# Unique Network Flows -For performance, ROS2 applications require careful selection of QoS for publishers and subscribers. Although networks offer various QoS options, ROS2 publishers and subscribers are unable to use them due to non-unique flow identifiers. As a result, ROS2 publishers and subscribers can only hope to obtain undifferentiated QoS from networks. This ultimately degrades the performance potential of ROS2 applications and wastes networking infrastructure investments. +For performance, ROS2 applications require careful selection of QoS for publishers and subscriptions. Although networks offer various QoS options, ROS2 publishers and subscriptions are unable to use them due to non-unique flows. As a result, ROS2 publishers and subscriptions can only hope to obtain undifferentiated QoS from networks. This ultimately degrades the performance potential of ROS2 applications and wastes networking infrastructure investments. -We propose unique network flow identifiers for ROS2 publishers and subscribers. Our proposal is easy to use, convenient to implement, and minimal. Plus, it respects network-agnostic and non-DDS-middleware-friendly concerns in ROS2. +We propose unique network flows for ROS2 publishers and subscriptions. Our proposal is easy to use, convenient to implement, and minimal. Plus, it respects network-agnostic and non-DDS-middleware-friendly concerns in ROS2. In this document, we first describe essential background concepts. After that we precisely state the problem and propose a solution to the problem. Existing solutions are compared in the end. @@ -45,7 +45,7 @@ We briefly discuss two relevant explicit QoS specification methods for applicati - Differentiated Services (DS) [3] is a widely-used QoS architecture for IP networks. The required DS-based QoS is set by the application in the 6-bit DS Code Point (DSCP) sub-field of the 8-bit DS field in the IP packet header. For example, DSCP set to 0x2E specifies expedited forwarding as the required QoS. Expedited forwarding is typically used for real-time data such as voice and video. - ROS2 lacks an API to specify DS-based QoS for publishers and subscribers. The DSCP value in their flows is therefore set to 0x00. This specifies default forwarding as the required QoS from the network. However, DDS provides the Transport Priority QoS policy to specify DS-based QoS. + ROS2 lacks an API to specify DS-based QoS for publishers and subscriptions. The DSCP value in their flows is therefore set to 0x00. This specifies default forwarding as the required QoS from the network. However, DDS provides the Transport Priority QoS policy to specify DS-based QoS. A frustrating problem with DS-based QoS is that intermediate routers can reset or alter the DSCP value within flows. One workaround is to carefully configure intermediate routers such that they retain DSCP markings from incoming to outgoing flows. @@ -69,9 +69,9 @@ We briefly discuss two relevant explicit QoS specification methods for applicati ## Problem -All publishers and subscribers in communicating nodes have the same flow identifers (5-tuple or 3-tuple). This disables explicit network QoS differentiation for publishers and subscribers in communicating nodes. In other words, publishers and subscribers in communicating nodes can only be assigned the same network QoS. +All publishers and subscriptions in communicating nodes have the same flow identifers (5-tuple or 3-tuple). This disables explicit network QoS differentiation for publishers and subscriptions in communicating nodes. In other words, publishers and subscriptions in communicating nodes can only be assigned the same network QoS. -We believe the problem occurs by design. For performance reasons, RMW implementations are likely to associate IP address, port, and transport protocol to nodes and not to individual publishers/subscribers. This is true for all the tier-1 RMW implementations today (Foxy Fitzroy at the time of writing). None of the tier-1 RMW implementations set the IPv6 flow label to differentiate flows of publisher/subscribers. +We believe the problem occurs by design. For performance reasons, RMW implementations are likely to associate IP address, port, and transport protocol to nodes and not to individual publishers/subscriptions. This is true for all the tier-1 RMW implementations today (Foxy Fitzroy at the time of writing). None of the tier-1 RMW implementations set the IPv6 flow label to differentiate flows of publisher/subscriptions. ### Example @@ -81,100 +81,114 @@ Consider a distributed robotics application with communicating nodes N1 and N2. N1 contains publishers P1 and P2. P1 publishes video data whereas P2 publishes battery status data. -N2 contains subscribers S1 and S2. S1 receives video from P1 and performs real-time object detection. S2 receives battery data from P2 and performs non-real-time battery management. +N2 contains subscriptions S1 and S2. S1 receives video from P1 and performs real-time object detection. S2 receives battery data from P2 and performs non-real-time battery management. The link P1-S1 requires low-latency QoS from the 5G network, say a maximum delay of 5ms. P2-S2 requires default latency QoS i.e., 300ms. Then, by construction, since the flow identifiers of P1-S1 and P2-S2 links are the same, they cannot be assigned the required QoS by the network. Both P1-S1 and P2-S2 can either be assigned QoS with 5ms delay or with 300ms delay. The former case represents a waste of network resources, the latter case degrades performance. ## Proposed Solution -Our proposal to solve the problem is to make the flow identifiers of publishers and subscribers in communicating nodes unique using existing IP header fields. +Our proposal to solve the problem is to make the flows of publishers and subscriptions in communicating nodes unique using existing IP header fields. -We believe there are at least three architectural options to implement unique network flows. These are shown in the figure below. +We construct a publisher/subscription creation-time option called `require_unique_network_flow` as a candidate structure to enable unique identification of flows. This option takes a value from the enumeration shown below. -![Architecture options for unique network flows](./unique-network-flows-arch-options.png) -### Option 1: Unique flow ID from publisher/subscriber +```cpp +enum unique_network_flow_requirement_t +{ + UNIQUE_NETWORK_FLOW_NOT_REQUIRED = 0, + UNIQUE_NETWORK_FLOW_STRICTLY_REQUIRED, + UNIQUE_NETWORK_FLOW_OPTIONALLY_REQUIRED, + UNIQUE_NETWORK_FLOW_SYSTEM_DEFAULT +} +``` -We construct a publisher and subscriber creation-time option called `unique_network_flow` as a candidate structure to enable unique identification of flows. +Upon receiving the publisher/subscription creation request, the RMW implementation assigns unique flow identifiers according to the `require_unique_network_flow` option value. -In one variant, `unique_network_flow` is a 32-bit unsigned integer whose default value is zero. +The default value of the option is a `UNIQUE_NETWORK_FLOW_NOT_REQUIRED` which indicates to the RMW implementation that a unique network flow is not required. -Each publisher or subscriber that requires a specific QoS from the network is required to set `unique_network_flow` to a value unique within the node. This unique value can be computed internally in the node using simple UID generation schemes (atomic integer counters or random integer sampling without duplicates) or supplied by an external component. +The value `UNIQUE_NETWORK_FLOW_STRICTLY_REQUIRED` indicates to the RMW implementation that a unique network flow is strictly required. If not feasible, the RMW implementation must flag an error and not create the associated publisher/subscription. -The example C++ snippet below shows a node creating three subscriptions. Subscriptions `sub_1_` and `sub_2_` are created using subscription options with node-unique `unique_network_flow` values. Subscription `sub_3_` is created with default subscription options (`unique_network_flow` = 0). +The value `UNIQUE_NETWORK_FLOW_OPTIONALLY_REQUIRED` enables the RMW implementation to create an unique network flow if feasible. If not, it can continue to create the associated publisher/subscription and not flag an error. -```cpp -// Enable unique network flow via subscription options -auto options_1 = rclcpp::SubscriptionOptions(); -options_1.unique_network_flow = 0xf4688c52; +The value `UNIQUE_NETWORK_FLOW_SYSTEM_DEFAULT` delegates the decision to create unique network flows fully to the RMW implementation. The RMW implementation can decide internally on its own or be coerced through side-loading mechanisms to create a unique network flow for the associated publisher/subscription. -sub_1_ = this->create_subscription( - "topic_1", 10, std::bind( - &MyNode::topic_1_callback, this, - _1), options_1); +The example C++ snippet below shows a node creating two subscriptions and a publisher. Subscription `sub_x_` strictly requires a unique network flow whereas `sub_y_` is indifferent. Publisher `pub_z_` optionally requires a unique network flow. -// Enable unique network flow via subscription options -auto options_2 = rclcpp::SubscriptionOptions(); -options_2.unique_network_flow = 0x88181c45; - -sub_2_ = this->create_subscription( - "topic_2", 10, std::bind( - &MyNode::topic_2_callback, this, - _1), options_2); - -// Unique network flows are disabled by default -auto options_3 = rclcpp::SubscriptionOptions(); -sub_3_ = this->create_subscription( - "topic_3", 10, std::bind( - &MyNode::topic_3_callback, this, - _1), options_3); -``` +```cpp +// Unique network flow strictly required +auto options_x = rclcpp::SubscriptionOptions(); +options_x.require_unique_network_flow = UNIQUE_NETWORK_FLOW_STRICTLY_REQUIRED; -The RMW implementation has several alternatives to convert the `unique_network_flow` argument to a unique flow identifier visible in packet headers. We list few candidate alternatives next. +sub_x_ = this->create_subscription( + "topic_x", 10, std::bind( + &MyNode::topic_x_callback, this, + _1), options_x); -If the node is communicating using IPv6, then the lower 20-bits of `unique_network_flow` can be transferred to the Flow Label field. This creates a unique 3-tuple. +// Unique network flow not required, relying on default +auto options_2 = rclcpp::SubscriptionOptions(); -Else, if the node is communicating via IPv4, then there are two alternatives. One is to copy the `unique_network_flow` argument to the Options field. Another is to copy the lower 6-bits of the `unique_network_flow` argument to the DSCP field. Both alternatives enable the network to infer a custom 6-tuple (traditional 5-tuple plus Options/DSCP) that uniquely identifies flows. +sub_y_ = this->create_subscription( + "topic_y", 10, std::bind( + &MyNode::topic_y_callback, this, + _1), options_y); -Yet another alternative is to use the `unique_network_flow` argument as a key that maps to unique identifiers maintained by the RMW implementation. These mapped unique identifiers are written to appropriate locations in packet headers, including transport protocol port fields. +// Unique network flow optionally required +auto options_z = rclcpp::PublisherOptions(); +options_z.require_unique_network_flow = RMW_UNIQUE_NETWORK_FLOW_OPTIONALLY_REQUIRED; -Both DDS and non-DDS RMW implementations can trivially set fields in IP or transport protocol headers using native socket API on all ROS2 platforms (Linux, Windows, MacOS). +publisher_z_ = this->create_publisher("topic_z", 10, options_z); +``` +The RMW implementation has several alternatives to convert the `require_unique_network_flow` option to a unique flow identifier visible in packet headers. We list few candidate alternatives next. -### Option 2: Publisher/subscriber delegates unique flow ID generation to RMW +If the node is communicating using IPv6, then it can write a unique value (such as a suitable derivative of the RTPS entity ID) into the Flow Label field. This creates a unique 3-tuple. -This is a programmer-friendly alternative that modifies the `unique_network_flow` option to a boolean parameter. Setting it to `true` instructs the RMW implementation to generate a unique flow identifier for the associated publisher/subscriber. By default, `unique_network_flow` is set to `false`. +Else, if the node is communicating via IPv4, then the RMW implementation can write unique values into the IP Options and DSCP fields. Both alternatives enable the network to infer a custom 6-tuple (traditional 5-tuple plus Options/DSCP) that uniquely identifies flows. -For those publishers/subscribers with `unique_network_flow` set to `true`, the RMW implementation generates an unique flow identifier internally first before writing it to an appropriate field in packet headers, similar to Option-1. +A simple option that works for both IPv6 and IPv4 is to use a unique transport port and accordingly update the transport protocol port fields. -Generating the unique flow identifier internally is a matter of implementing a simple hashing functions parameterized by existing unique identifiers for publishers/subscribers within a node (for example, the RTPS entity ID). +Both DDS and non-DDS RMW implementations can trivially set fields in IP or transport protocol headers using native socket API on all ROS2 platforms (Linux, Windows, MacOS). -A suitable interface should be created for publishers/subscribers to obtain unique flow IDs created by the RMW. We prefer to use community help to create this interface. +To enable applications to discern (unique) network flows created by the RMW implementation, we propose a simple flow-getter interface for publishers/subscriptions. Continuing from the previous code snippet, -### Option 3: RMW generates unique flow IDs for all publishers/subscribers +```cpp +// Get network flows +auto network_flows_x = sub_x_->get_network_flow(); +auto network_flows_y = sub_y_->get_network_flow(); + +// Ensure sub_x_ has an unique network flow +if (network_flows_x.size() > 0 && network_flows_y.size() > 0) { + for (auto network_flow_x : network_flows_x) { + for (auto network_flow_y : network_flows_y) { + if (network_flow_x == network_flow_y) { + std::runtime_error("Flows are not unique!"); + } + } + } +``` -This is the most programmer-friendly option. The RMW indiscriminately generates unique flow IDs for all publishers/subscribers. Everything else is similar to Option-2. +The RMW implementation is only required to the return the endpoints of network flows of the publisher/subscription associated with the `get_network_flow()` call. ### Advantages Our proposal has the following advantages: -- Easy to use: Application developers are only required to decide if unique flow identifiers for publishers/subscribers are necessary. If yes, they can easily generate the unique identifiers themselves, or obtain it from external components, or delegate unique identifier generation to the RMW implementation, or choose an RMW implementation that indiscriminately generates unique flow IDs for publishers/subscribers. +- Easy to use: Application developers are only required to decide if unique flow identifiers for publishers/subscriptions are necessary. - Light-weight implementation: Both non-DDS and DDS RMW can implement the required support conveniently using native socket API with negligible impact on performance. - Network-agnostic: No particular network is preferred, respecting ROS2 design preferences. - Minimum change: Introducing the *choice* of unique flow identifiers into the application layer is the minimum framework change required to obtain bespoke QoS from QoS-centric machine-machine network technologies such as 5G. ### Limitations -If the RMW implementation decides to create a custom 6-tuple using the IPv4 DSCP field, the flows of only up to 64 (2^6) publishers and subscribers within a node can be uniquely identified. In addition, network administration processes should be notified that the DSCP field is re-purposed as an identifier to prevent misinterpretation. +If the RMW implementation decides to create a custom 6-tuple using the IPv4 DSCP field, the flows of only up to 64 (2^6) publishers and subscriptions within a node can be uniquely identified. In addition, network administration processes should be notified that the DSCP field is re-purposed as an identifier to prevent misinterpretation. ## Alternative Solutions We list a few alternative solutions to the problem that are limited and dissatisfactory. -1. Dedicated nodes: Publishers and subscribers that require special network QoS can be isolated to dedicated nodes. Such isolation indeed makes their 5-tuple flow identifier unique. However, this breaks the functionality-based node architecture of the application and degrades performance since nodes are heavy-weight structures. In the worst case, a dedicated node per publisher or subscriber is required. +1. Dedicated nodes: Publishers and subscriptions that require special network QoS can be isolated to dedicated nodes. Such isolation indeed makes their 5-tuple flow identifier unique. However, this breaks the functionality-based node architecture of the application and degrades performance since nodes are heavy-weight structures. In the worst case, a dedicated node per publisher or subscription is required. -2. Custom 6-tuple using side-loaded DDS Transport Priority QoS policies: Conceptually, the custom 6-tuple can be constructed by side-loading `unique_network_flow` values into the Transport Priority QoS policy of the DDS RMW implementation. In practice, however, this is difficult to implement for several reasons. First, it expects DDS RMW side-loading competence from application programmers which is inconvenient. Second, re-purposing DSCP values as identifiers is limited to 64 identifiers and requires careful network administration as mentioned before. Third, side-loading support varies across DDS RMW implementations. To the best of our knowledge, none of the tier-1 DDS implementations for ROS2 today (Foxy) support side-loading Transport Priority QoS policies for *select few* publishers and subscribers in a node due to lack of fine-grained interfaces. A glaring limitation is that this alternative ignores non-DDS RMW. +2. Custom 6-tuple using side-loaded DDS Transport Priority QoS policies: Conceptually, the custom 6-tuple can be constructed by side-loading unique values into the Transport Priority QoS policy of the DDS RMW implementation. In practice, however, this is difficult to implement for several reasons. First, it expects DDS RMW side-loading competence from application programmers which is inconvenient. Second, re-purposing DSCP values as unique identifiers is limited to 64 identifiers and requires careful network administration as mentioned before. Third, side-loading support varies across DDS RMW implementations. To the best of our knowledge, none of the tier-1 DDS implementations for ROS2 today (Foxy) support side-loading Transport Priority QoS policies for *select few* publishers and subscriptions in a node due to lack of fine-grained interfaces. A glaring limitation is that this alternative ignores non-DDS RMW. 3. DS-based QoS using side-loaded DDS Transport Priority QoS policies: This gets ahead of the problem by directly specifying the required DS-based QoS through side-loaded Transport Priority QoS policies. However, this suffers from similar impracticalities as the previous alternative. It ignores non-DDS RMW, expects DS competence from programmers, and is not supported by tier-1 RMW implementations. From 04fd6f76647f5361a9e7dce38d5fe3c485380ae9 Mon Sep 17 00:00:00 2001 From: Ananya Muddukrishna Date: Wed, 17 Feb 2021 17:58:21 +0100 Subject: [PATCH 12/17] Address leaky abstraction concerns Signed-off-by: Ananya Muddukrishna --- articles/unique_network_flows.md | 32 +++++++++++++++++++------------- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/articles/unique_network_flows.md b/articles/unique_network_flows.md index 8f1bdcf46..6677712f7 100644 --- a/articles/unique_network_flows.md +++ b/articles/unique_network_flows.md @@ -91,7 +91,6 @@ Our proposal to solve the problem is to make the flows of publishers and subscri We construct a publisher/subscription creation-time option called `require_unique_network_flow` as a candidate structure to enable unique identification of flows. This option takes a value from the enumeration shown below. - ```cpp enum unique_network_flow_requirement_t { @@ -104,7 +103,7 @@ enum unique_network_flow_requirement_t Upon receiving the publisher/subscription creation request, the RMW implementation assigns unique flow identifiers according to the `require_unique_network_flow` option value. -The default value of the option is a `UNIQUE_NETWORK_FLOW_NOT_REQUIRED` which indicates to the RMW implementation that a unique network flow is not required. +The default value of the option is `UNIQUE_NETWORK_FLOW_NOT_REQUIRED` which indicates to the RMW implementation that a unique network flow is not required. The value `UNIQUE_NETWORK_FLOW_STRICTLY_REQUIRED` indicates to the RMW implementation that a unique network flow is strictly required. If not feasible, the RMW implementation must flag an error and not create the associated publisher/subscription. @@ -136,9 +135,11 @@ sub_y_ = this->create_subscription( auto options_z = rclcpp::PublisherOptions(); options_z.require_unique_network_flow = RMW_UNIQUE_NETWORK_FLOW_OPTIONALLY_REQUIRED; -publisher_z_ = this->create_publisher("topic_z", 10, options_z); +pub_z_ = this->create_publisher("topic_z", 10, options_z); ``` +In the common case where nodes have a single publisher/subscription per topic, a unique flow is adequately achieved by activating either the publisher- or the subscription-side `require_unique_network_flow` option. RMW implementations can therefore decide to support either publisher/subscription-side option to enable unique network flows. Support for options on both sides is required in the odd case that a node has multiple publishers/subscriptions for the same topic. + The RMW implementation has several alternatives to convert the `require_unique_network_flow` option to a unique flow identifier visible in packet headers. We list few candidate alternatives next. If the node is communicating using IPv6, then it can write a unique value (such as a suitable derivative of the RTPS entity ID) into the Flow Label field. This creates a unique 3-tuple. @@ -149,25 +150,29 @@ A simple option that works for both IPv6 and IPv4 is to use a unique transport p Both DDS and non-DDS RMW implementations can trivially set fields in IP or transport protocol headers using native socket API on all ROS2 platforms (Linux, Windows, MacOS). -To enable applications to discern (unique) network flows created by the RMW implementation, we propose a simple flow-getter interface for publishers/subscriptions. Continuing from the previous code snippet, +To enable applications to discern (unique) network flows created by the RMW implementation, we propose a simple interface for getting flow endpoints of publishers/subscriptions. Continuing from the previous code snippet, ```cpp // Get network flows -auto network_flows_x = sub_x_->get_network_flow(); -auto network_flows_y = sub_y_->get_network_flow(); +auto flows_x = sub_x_->get_network_flow_endpoints(); +auto flows_y = sub_y_->get_network_flow_endpoints(); // Ensure sub_x_ has an unique network flow -if (network_flows_x.size() > 0 && network_flows_y.size() > 0) { - for (auto network_flow_x : network_flows_x) { - for (auto network_flow_y : network_flows_y) { - if (network_flow_x == network_flow_y) { +if (flows_x.size() > 0 && flows_y.size() > 0) { + for (auto flow_x : flows_x) { + for (auto flow_y : flows_y) { + if (flow_x == flow_y) { std::runtime_error("Flows are not unique!"); } } } ``` -The RMW implementation is only required to the return the endpoints of network flows of the publisher/subscription associated with the `get_network_flow()` call. +The proposed method `get_network_flow_endpoints()` requires RMW implementations to return a vector of byte-arrays/strings that represents the endpoints of network flows of the associated publisher/subscription. This information is custom to each RMW, similar to the existing `get_gid()` method that returns a custom `rmw_gid_t` value. The rationale to consider RMW-custom byte-arrays/strings instead of RMW-common structures is to address leaky-abstraction concerns. + +To clarify, the flow endpoints returned by the `get_network_flow_endpoints()` for a publisher represents the endpoints of the channels on the publisher-side only. It does not contain information about endpoints of matched subscriptions. Similarly, the endpoints returned by `get_network_flow_endpoints()` for a subscription has no information about matched publishers. + +DDS-based RMW may consider returning a string-serialized variant of the DDS-standard `Locator_t` structure when `get_network_flow_endpoints()` is called. ### Advantages @@ -180,7 +185,9 @@ Our proposal has the following advantages: ### Limitations -If the RMW implementation decides to create a custom 6-tuple using the IPv4 DSCP field, the flows of only up to 64 (2^6) publishers and subscriptions within a node can be uniquely identified. In addition, network administration processes should be notified that the DSCP field is re-purposed as an identifier to prevent misinterpretation. +- If the RMW implementation decides to create a custom 6-tuple using the IPv4 DSCP field, the flows of only up to 64 (2^6) publishers and subscriptions within a node can be uniquely identified. In addition, network administration processes should be notified that the DSCP field is re-purposed as an identifier to prevent misinterpretation. + +- Custom parsers built using RMW-specific documentation are required to understand the byte-arrays/strings returned by `get_network_flow_endpoints()`. ## Alternative Solutions @@ -201,4 +208,3 @@ We list a few alternative solutions to the problem that are limited and dissatis [3] [Differentiated Services (IETF RFC-2474)](https://tools.ietf.org/html/rfc2474) [4] [5G System Architecture Specification (3GPP TS 23.501)](https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3144) - From 997d6577b764f9b0128e97c466382d9208a3e2d9 Mon Sep 17 00:00:00 2001 From: Ananya Muddukrishna Date: Wed, 24 Feb 2021 10:27:19 +0100 Subject: [PATCH 13/17] Make specification concrete Signed-off-by: Ananya Muddukrishna --- articles/unique_network_flows.md | 104 +++++++++++++++++++------------ 1 file changed, 64 insertions(+), 40 deletions(-) diff --git a/articles/unique_network_flows.md b/articles/unique_network_flows.md index 6677712f7..0a0f1c92d 100644 --- a/articles/unique_network_flows.md +++ b/articles/unique_network_flows.md @@ -21,7 +21,7 @@ Original Author: {{ page.author }} For performance, ROS2 applications require careful selection of QoS for publishers and subscriptions. Although networks offer various QoS options, ROS2 publishers and subscriptions are unable to use them due to non-unique flows. As a result, ROS2 publishers and subscriptions can only hope to obtain undifferentiated QoS from networks. This ultimately degrades the performance potential of ROS2 applications and wastes networking infrastructure investments. -We propose unique network flows for ROS2 publishers and subscriptions. Our proposal is easy to use, convenient to implement, and minimal. Plus, it respects network-agnostic and non-DDS-middleware-friendly concerns in ROS2. +We propose unique network flows for ROS2 publishers and subscriptions. Our proposal is easy to use, convenient to implement, and minimal. Plus, it respects non-DDS-middleware-friendly concerns in ROS2. In this document, we first describe essential background concepts. After that we precisely state the problem and propose a solution to the problem. Existing solutions are compared in the end. @@ -87,43 +87,65 @@ The link P1-S1 requires low-latency QoS from the 5G network, say a maximum delay ## Proposed Solution -Our proposal to solve the problem is to make the flows of publishers and subscriptions in communicating nodes unique using existing IP header fields. +Our proposal to solve the problem is to make the flows of publishers and subscriptions in communicating nodes unique. -We construct a publisher/subscription creation-time option called `require_unique_network_flow` as a candidate structure to enable unique identification of flows. This option takes a value from the enumeration shown below. +### Definitions + +- *Network flow*: A tuple of networking resources selected by a RMW implementation for transmission of messages from a publisher to a subscription. The networking resources considered are: + - transport protocol UDP or TCP (publisher and subscription) + - transport port (publisher and subscription) + - internet protocol IPv4 or IPv6 (publisher and subscription) + - internet address (publisher and subscription) + - DSCP (publisher only) + - Flow label (publisher only) + + Network flows are defined for UDP/TCP and IP-based RMW implementations only. It is not a limiting definition since these are the majority protocols used today. The definition can be later extended to include relevant non-IP networks such as deterministic ethernet. + +- *Network Flow Endpoint (NFE)*: The portion of a network flow specific to the publisher and or the subscription. In other words, each network flow has two NFEs; one for the publisher and the other for the subscription. + +- *Dissimilar NFE pair*: Two NFEs are dissimilar if one or more of their networking resources are different. + +- *Unique NFE*: A NFE is unique if it is dissimilar to all other NFEs. + +- *Strongly and weakly unique network flow*: A network flow is strongly unique if both publisher and subscription NFEs are unique. A weakly unique network flow either has a publisher NFE or a subscriber NFE that is unique. + +### Unique Network Flow Endpoints + +We construct a publisher/subscription creation-time option called `require_unique_network_flow_endpoint` as a candidate structure to enable unique identification of network flows. This option takes a value from the enumeration shown below. ```cpp -enum unique_network_flow_requirement_t +enum unique_network_flow_endpoint_requirement_t { - UNIQUE_NETWORK_FLOW_NOT_REQUIRED = 0, - UNIQUE_NETWORK_FLOW_STRICTLY_REQUIRED, - UNIQUE_NETWORK_FLOW_OPTIONALLY_REQUIRED, - UNIQUE_NETWORK_FLOW_SYSTEM_DEFAULT + UNIQUE_NETWORK_FLOW_ENDPOINT_NOT_REQUIRED = 0, + UNIQUE_NETWORK_FLOW_ENDPOINT_STRICTLY_REQUIRED, + UNIQUE_NETWORK_FLOW_ENDPOINT_OPTIONALLY_REQUIRED, + UNIQUE_NETWORK_FLOW_ENDPOINT_SYSTEM_DEFAULT } ``` -Upon receiving the publisher/subscription creation request, the RMW implementation assigns unique flow identifiers according to the `require_unique_network_flow` option value. +Upon receiving the publisher/subscription creation request, the RMW implementation assigns unique NFEs according to the `require_unique_network_flow_endpoint` option value. -The default value of the option is `UNIQUE_NETWORK_FLOW_NOT_REQUIRED` which indicates to the RMW implementation that a unique network flow is not required. +The default value of the option is `UNIQUE_NETWORK_FLOW_ENDPOINT_NOT_REQUIRED` which indicates to the RMW implementation that a unique NFE is not required. -The value `UNIQUE_NETWORK_FLOW_STRICTLY_REQUIRED` indicates to the RMW implementation that a unique network flow is strictly required. If not feasible, the RMW implementation must flag an error and not create the associated publisher/subscription. +The value `UNIQUE_NETWORK_FLOW_ENDPOINT_STRICTLY_REQUIRED` indicates to the RMW implementation that a unique NFE is strictly required. If not feasible, the RMW implementation must flag an error and not create the associated publisher/subscription. -The value `UNIQUE_NETWORK_FLOW_OPTIONALLY_REQUIRED` enables the RMW implementation to create an unique network flow if feasible. If not, it can continue to create the associated publisher/subscription and not flag an error. +The value `UNIQUE_NETWORK_FLOW_ENDPOINT_OPTIONALLY_REQUIRED` enables the RMW implementation to create an unique NFE if feasible. If not feasible, it can continue to create the associated publisher/subscription and not flag an error. -The value `UNIQUE_NETWORK_FLOW_SYSTEM_DEFAULT` delegates the decision to create unique network flows fully to the RMW implementation. The RMW implementation can decide internally on its own or be coerced through side-loading mechanisms to create a unique network flow for the associated publisher/subscription. +The value `UNIQUE_NETWORK_FLOW_ENDPOINT_SYSTEM_DEFAULT` delegates the decision to create unique NFEs fully to the RMW implementation. The RMW implementation can decide internally on its own or be coerced through side-loading mechanisms to create a unique NFE for the associated publisher/subscription. -The example C++ snippet below shows a node creating two subscriptions and a publisher. Subscription `sub_x_` strictly requires a unique network flow whereas `sub_y_` is indifferent. Publisher `pub_z_` optionally requires a unique network flow. +The example C++ snippet below shows a node creating two subscriptions and a publisher. Subscription `sub_x_` strictly requires a unique NFE whereas `sub_y_` is indifferent. Publisher `pub_z_` optionally requires a unique NFE. ```cpp -// Unique network flow strictly required +// Unique network flow endpoint strictly required auto options_x = rclcpp::SubscriptionOptions(); -options_x.require_unique_network_flow = UNIQUE_NETWORK_FLOW_STRICTLY_REQUIRED; +options_x.require_unique_network_flow_endpoint = UNIQUE_NETWORK_FLOW_ENDPOINT_STRICTLY_REQUIRED; sub_x_ = this->create_subscription( "topic_x", 10, std::bind( &MyNode::topic_x_callback, this, _1), options_x); -// Unique network flow not required, relying on default +// Unique network flow endpoint not required, relying on default auto options_2 = rclcpp::SubscriptionOptions(); sub_y_ = this->create_subscription( @@ -131,48 +153,52 @@ sub_y_ = this->create_subscription( &MyNode::topic_y_callback, this, _1), options_y); -// Unique network flow optionally required +// Unique network flow enpoint optionally required auto options_z = rclcpp::PublisherOptions(); -options_z.require_unique_network_flow = RMW_UNIQUE_NETWORK_FLOW_OPTIONALLY_REQUIRED; +options_z.require_unique_network_flow_endpoint = RMW_UNIQUE_NETWORK_FLOW_ENDPOINT_OPTIONALLY_REQUIRED; pub_z_ = this->create_publisher("topic_z", 10, options_z); ``` -In the common case where nodes have a single publisher/subscription per topic, a unique flow is adequately achieved by activating either the publisher- or the subscription-side `require_unique_network_flow` option. RMW implementations can therefore decide to support either publisher/subscription-side option to enable unique network flows. Support for options on both sides is required in the odd case that a node has multiple publishers/subscriptions for the same topic. +A weakly unique network flow is created by making either the publisher or the subscription NFE unique. This is sufficient to enable network QoS differentiation for the common case where nodes have a single publisher/subscription per topic. RMW implementations can therefore decide to support either the publisher or subscription option to enable unique NFEs. + +A strongly unique network flow is created by making both publisher and subscription NFEs unique. This enables network QoS differentiation in the rather odd case of a node having multiple publishers/subscriptions for the same topic. RMW implementations can decide not to support such odd cases i.e., not support both publisher and subscription option for unique NFEs. -The RMW implementation has several alternatives to convert the `require_unique_network_flow` option to a unique flow identifier visible in packet headers. We list few candidate alternatives next. +We list few candidate alternatives next for RMW implementations to to implement the `require_unique_network_flow_endpoint` option. -If the node is communicating using IPv6, then it can write a unique value (such as a suitable derivative of the RTPS entity ID) into the Flow Label field. This creates a unique 3-tuple. +- If the node is communicating using IPv6, then the RMW implementation can write a unique value (such as a suitable derivative of the RTPS entity ID) in the Flow Label field. -Else, if the node is communicating via IPv4, then the RMW implementation can write unique values into the IP Options and DSCP fields. Both alternatives enable the network to infer a custom 6-tuple (traditional 5-tuple plus Options/DSCP) that uniquely identifies flows. +- Else, if the node is communicating via IPv4, then the RMW implementation can write unique values in the IP Options and DSCP fields. -A simple option that works for both IPv6 and IPv4 is to use a unique transport port and accordingly update the transport protocol port fields. +- A simple option that works for both IPv6 and IPv4 is to select a unique transport port (UDP/TCP) and accordingly update the port field in the transport protocol header. Both DDS and non-DDS RMW implementations can trivially set fields in IP or transport protocol headers using native socket API on all ROS2 platforms (Linux, Windows, MacOS). -To enable applications to discern (unique) network flows created by the RMW implementation, we propose a simple interface for getting flow endpoints of publishers/subscriptions. Continuing from the previous code snippet, +### Get Network Flow Endpoints + +To enable applications to discern NFEs created by the RMW implementation, we propose a simple getter interface. Continuing from the previous code snippet, ```cpp // Get network flows -auto flows_x = sub_x_->get_network_flow_endpoints(); -auto flows_y = sub_y_->get_network_flow_endpoints(); +auto a_nfe_x = sub_x_->get_network_flow_endpoint(); +auto a_nfe_y = sub_y_->get_network_flow_endpoint(); // Ensure sub_x_ has an unique network flow -if (flows_x.size() > 0 && flows_y.size() > 0) { - for (auto flow_x : flows_x) { - for (auto flow_y : flows_y) { - if (flow_x == flow_y) { - std::runtime_error("Flows are not unique!"); +if (a_nfe_x.size() > 0 && a_nfe_y.size() > 0) { + for (auto nfe_x : a_nfe_x) { + for (auto nfe_y : a_nfe_y) { + if (nfe_x == nfe_y) { + std::runtime_error("Network flow endpoints are not unique!"); } } } ``` -The proposed method `get_network_flow_endpoints()` requires RMW implementations to return a vector of byte-arrays/strings that represents the endpoints of network flows of the associated publisher/subscription. This information is custom to each RMW, similar to the existing `get_gid()` method that returns a custom `rmw_gid_t` value. The rationale to consider RMW-custom byte-arrays/strings instead of RMW-common structures is to address leaky-abstraction concerns. +The proposed method `get_network_flow_endpoint()` requires RMW implementations to return NFEs of the associated publisher/subscription. The data structure of the NFE is specified concretely by the `rmw` layer. -To clarify, the flow endpoints returned by the `get_network_flow_endpoints()` for a publisher represents the endpoints of the channels on the publisher-side only. It does not contain information about endpoints of matched subscriptions. Similarly, the endpoints returned by `get_network_flow_endpoints()` for a subscription has no information about matched publishers. +To reiterate, the NFEs returned by `get_network_flow_endpoint()` for a publisher represents the NFEs created on the publisher-side only. It does not contain information about NFEs of matched subscriptions. Similarly, the NFEs returned by `get_network_flow_endpoint()` for a subscription has no information about matched publishers. -DDS-based RMW may consider returning a string-serialized variant of the DDS-standard `Locator_t` structure when `get_network_flow_endpoints()` is called. +DDS-based RMW implementations can obtain required information using the DDS-standard `Locator_t` structure when `get_network_flow_endpoint()` is called. ### Advantages @@ -180,14 +206,12 @@ Our proposal has the following advantages: - Easy to use: Application developers are only required to decide if unique flow identifiers for publishers/subscriptions are necessary. - Light-weight implementation: Both non-DDS and DDS RMW can implement the required support conveniently using native socket API with negligible impact on performance. -- Network-agnostic: No particular network is preferred, respecting ROS2 design preferences. +- RMW-agnostic: No particular network is preferred, respecting ROS2 design preferences. - Minimum change: Introducing the *choice* of unique flow identifiers into the application layer is the minimum framework change required to obtain bespoke QoS from QoS-centric machine-machine network technologies such as 5G. ### Limitations -- If the RMW implementation decides to create a custom 6-tuple using the IPv4 DSCP field, the flows of only up to 64 (2^6) publishers and subscriptions within a node can be uniquely identified. In addition, network administration processes should be notified that the DSCP field is re-purposed as an identifier to prevent misinterpretation. - -- Custom parsers built using RMW-specific documentation are required to understand the byte-arrays/strings returned by `get_network_flow_endpoints()`. +- If the RMW implementation decides to create a NFE using the IPv4 DSCP field, then only up to 64 (2^6) publishers and subscriptions can be uniquely identified assuming all other resources in the NFE remain constant. In addition, network administration processes should be notified that the DSCP field is re-purposed as an identifier to prevent misinterpretation and erasure. ## Alternative Solutions @@ -195,7 +219,7 @@ We list a few alternative solutions to the problem that are limited and dissatis 1. Dedicated nodes: Publishers and subscriptions that require special network QoS can be isolated to dedicated nodes. Such isolation indeed makes their 5-tuple flow identifier unique. However, this breaks the functionality-based node architecture of the application and degrades performance since nodes are heavy-weight structures. In the worst case, a dedicated node per publisher or subscription is required. -2. Custom 6-tuple using side-loaded DDS Transport Priority QoS policies: Conceptually, the custom 6-tuple can be constructed by side-loading unique values into the Transport Priority QoS policy of the DDS RMW implementation. In practice, however, this is difficult to implement for several reasons. First, it expects DDS RMW side-loading competence from application programmers which is inconvenient. Second, re-purposing DSCP values as unique identifiers is limited to 64 identifiers and requires careful network administration as mentioned before. Third, side-loading support varies across DDS RMW implementations. To the best of our knowledge, none of the tier-1 DDS implementations for ROS2 today (Foxy) support side-loading Transport Priority QoS policies for *select few* publishers and subscriptions in a node due to lack of fine-grained interfaces. A glaring limitation is that this alternative ignores non-DDS RMW. +2. Custom 6-tuple using side-loaded DDS Transport Priority QoS policies: Conceptually, a custom 6-tuple can be constructed by side-loading unique values into the Transport Priority QoS policy of the DDS RMW implementation. In practice, however, this is difficult to implement for several reasons. First, it expects DDS RMW side-loading competence from application programmers which is inconvenient. Second, re-purposing DSCP values as unique identifiers is limited to 64 identifiers and requires careful network administration as mentioned before. Third, side-loading support varies across DDS RMW implementations. To the best of our knowledge, none of the tier-1 DDS implementations for ROS2 today (Foxy) support side-loading Transport Priority QoS policies for *select few* publishers and subscriptions in a node due to lack of fine-grained interfaces. A glaring limitation is that this alternative ignores non-DDS RMW. 3. DS-based QoS using side-loaded DDS Transport Priority QoS policies: This gets ahead of the problem by directly specifying the required DS-based QoS through side-loaded Transport Priority QoS policies. However, this suffers from similar impracticalities as the previous alternative. It ignores non-DDS RMW, expects DS competence from programmers, and is not supported by tier-1 RMW implementations. From 5de98d7ca1687e88d8a89038b34067915edccbd1 Mon Sep 17 00:00:00 2001 From: Ananya Muddukrishna Date: Fri, 26 Feb 2021 18:50:28 +0100 Subject: [PATCH 14/17] Clarify API for implementors Signed-off-by: Ananya Muddukrishna --- articles/unique_network_flows.md | 54 +++++++++++++++----------------- 1 file changed, 26 insertions(+), 28 deletions(-) diff --git a/articles/unique_network_flows.md b/articles/unique_network_flows.md index 0a0f1c92d..eb47a9b22 100644 --- a/articles/unique_network_flows.md +++ b/articles/unique_network_flows.md @@ -111,41 +111,41 @@ Our proposal to solve the problem is to make the flows of publishers and subscri ### Unique Network Flow Endpoints -We construct a publisher/subscription creation-time option called `require_unique_network_flow_endpoint` as a candidate structure to enable unique identification of network flows. This option takes a value from the enumeration shown below. +We construct a publisher/subscription creation-time option called `require_unique_network_flow_endpoints` as a candidate structure to enable unique identification of network flows. This option takes a value from the enumeration shown below. ```cpp -enum unique_network_flow_endpoint_requirement_t +enum unique_network_flow_endpoints_requirement_t { - UNIQUE_NETWORK_FLOW_ENDPOINT_NOT_REQUIRED = 0, - UNIQUE_NETWORK_FLOW_ENDPOINT_STRICTLY_REQUIRED, - UNIQUE_NETWORK_FLOW_ENDPOINT_OPTIONALLY_REQUIRED, - UNIQUE_NETWORK_FLOW_ENDPOINT_SYSTEM_DEFAULT + UNIQUE_NETWORK_FLOW_ENDPOINTS_NOT_REQUIRED = 0, + UNIQUE_NETWORK_FLOW_ENDPOINTS_STRICTLY_REQUIRED, + UNIQUE_NETWORK_FLOW_ENDPOINTS_OPTIONALLY_REQUIRED, + UNIQUE_NETWORK_FLOW_ENDPOINTS_SYSTEM_DEFAULT } ``` -Upon receiving the publisher/subscription creation request, the RMW implementation assigns unique NFEs according to the `require_unique_network_flow_endpoint` option value. +Upon receiving the publisher/subscription creation request, the RMW implementation assigns unique NFEs according to the `require_unique_network_flow_endpoints` option value. -The default value of the option is `UNIQUE_NETWORK_FLOW_ENDPOINT_NOT_REQUIRED` which indicates to the RMW implementation that a unique NFE is not required. +The default value of the option is `UNIQUE_NETWORK_FLOW_ENDPOINTS_NOT_REQUIRED` which indicates to the RMW implementation that unique NFEs are not required. -The value `UNIQUE_NETWORK_FLOW_ENDPOINT_STRICTLY_REQUIRED` indicates to the RMW implementation that a unique NFE is strictly required. If not feasible, the RMW implementation must flag an error and not create the associated publisher/subscription. +The value `UNIQUE_NETWORK_FLOW_ENDPOINTS_STRICTLY_REQUIRED` indicates to the RMW implementation that unique NFEs are strictly required. If not feasible, the RMW implementation must flag an error and not create the associated publisher/subscription. -The value `UNIQUE_NETWORK_FLOW_ENDPOINT_OPTIONALLY_REQUIRED` enables the RMW implementation to create an unique NFE if feasible. If not feasible, it can continue to create the associated publisher/subscription and not flag an error. +The value `UNIQUE_NETWORK_FLOW_ENDPOINTS_OPTIONALLY_REQUIRED` enables the RMW implementation to create unique NFEs if feasible. If not feasible, it can continue to create the associated publisher/subscription and not flag an error. -The value `UNIQUE_NETWORK_FLOW_ENDPOINT_SYSTEM_DEFAULT` delegates the decision to create unique NFEs fully to the RMW implementation. The RMW implementation can decide internally on its own or be coerced through side-loading mechanisms to create a unique NFE for the associated publisher/subscription. +The value `UNIQUE_NETWORK_FLOW_ENDPOINTS_SYSTEM_DEFAULT` delegates the decision to create unique NFEs fully to the RMW implementation. The RMW implementation can decide internally on its own or be coerced through side-loading mechanisms to create unique NFEs for the associated publisher/subscription. -The example C++ snippet below shows a node creating two subscriptions and a publisher. Subscription `sub_x_` strictly requires a unique NFE whereas `sub_y_` is indifferent. Publisher `pub_z_` optionally requires a unique NFE. +The example C++ snippet below shows a node creating two subscriptions and a publisher. Subscription `sub_x_` strictly requires unique NFEs whereas `sub_y_` is indifferent. Publisher `pub_z_` optionally requires a unique NFE. ```cpp -// Unique network flow endpoint strictly required +// Unique network flow endpoints strictly required auto options_x = rclcpp::SubscriptionOptions(); -options_x.require_unique_network_flow_endpoint = UNIQUE_NETWORK_FLOW_ENDPOINT_STRICTLY_REQUIRED; +options_x.require_unique_network_flow_endpoints = UNIQUE_NETWORK_FLOW_ENDPOINTS_STRICTLY_REQUIRED; sub_x_ = this->create_subscription( "topic_x", 10, std::bind( &MyNode::topic_x_callback, this, _1), options_x); -// Unique network flow endpoint not required, relying on default +// Unique network flow endpoints not required, relying on default auto options_2 = rclcpp::SubscriptionOptions(); sub_y_ = this->create_subscription( @@ -153,9 +153,9 @@ sub_y_ = this->create_subscription( &MyNode::topic_y_callback, this, _1), options_y); -// Unique network flow enpoint optionally required +// Unique network flow endpoints optionally required auto options_z = rclcpp::PublisherOptions(); -options_z.require_unique_network_flow_endpoint = RMW_UNIQUE_NETWORK_FLOW_ENDPOINT_OPTIONALLY_REQUIRED; +options_z.require_unique_network_flow_endpoints = RMW_UNIQUE_NETWORK_FLOW_ENDPOINTS_OPTIONALLY_REQUIRED; pub_z_ = this->create_publisher("topic_z", 10, options_z); ``` @@ -166,11 +166,9 @@ A strongly unique network flow is created by making both publisher and subscript We list few candidate alternatives next for RMW implementations to to implement the `require_unique_network_flow_endpoint` option. -- If the node is communicating using IPv6, then the RMW implementation can write a unique value (such as a suitable derivative of the RTPS entity ID) in the Flow Label field. - -- Else, if the node is communicating via IPv4, then the RMW implementation can write unique values in the IP Options and DSCP fields. - - A simple option that works for both IPv6 and IPv4 is to select a unique transport port (UDP/TCP) and accordingly update the port field in the transport protocol header. +- If the node is communicating using IPv6, then the RMW implementation can write a unique value (such as a suitable derivative of the RTPS entity ID) in the Flow Label field. +- If the node is communicating via IPv4, then the RMW implementation can write a unique value in the DSCP field. This option should only be considered as a last resort since re-purposing the DSCP as an identifier is prone to misinterpretation, is limited to 64 entries, and requires careful configuration of intermediate routers. Both DDS and non-DDS RMW implementations can trivially set fields in IP or transport protocol headers using native socket API on all ROS2 platforms (Linux, Windows, MacOS). @@ -180,10 +178,10 @@ To enable applications to discern NFEs created by the RMW implementation, we pro ```cpp // Get network flows -auto a_nfe_x = sub_x_->get_network_flow_endpoint(); -auto a_nfe_y = sub_y_->get_network_flow_endpoint(); +auto a_nfe_x = sub_x_->get_network_flow_endpoints(); +auto a_nfe_y = sub_y_->get_network_flow_endpoints(); -// Ensure sub_x_ has an unique network flow +// Ensure sub_x_ has unique network flow endpoints if (a_nfe_x.size() > 0 && a_nfe_y.size() > 0) { for (auto nfe_x : a_nfe_x) { for (auto nfe_y : a_nfe_y) { @@ -194,11 +192,11 @@ if (a_nfe_x.size() > 0 && a_nfe_y.size() > 0) { } ``` -The proposed method `get_network_flow_endpoint()` requires RMW implementations to return NFEs of the associated publisher/subscription. The data structure of the NFE is specified concretely by the `rmw` layer. +The proposed method `get_network_flow_endpoints()` requires RMW implementations to return NFEs of the associated publisher/subscription. The data structure of the NFE is specified concretely by the `rmw` layer. -To reiterate, the NFEs returned by `get_network_flow_endpoint()` for a publisher represents the NFEs created on the publisher-side only. It does not contain information about NFEs of matched subscriptions. Similarly, the NFEs returned by `get_network_flow_endpoint()` for a subscription has no information about matched publishers. +To reiterate, the NFEs returned by `get_network_flow_endpoints()` for a publisher represents the NFEs created on the publisher-side only (local). It does not contain information about NFEs of matched subscriptions. Similarly, the NFEs returned by `get_network_flow_endpoints()` for a subscription are localized with no information about matched publishers. -DDS-based RMW implementations can obtain required information using the DDS-standard `Locator_t` structure when `get_network_flow_endpoint()` is called. +DDS-based RMW implementations can obtain required information using the DDS-standard `Locator_t` structure when `get_network_flow_endpoints()` is called. ### Advantages @@ -207,7 +205,7 @@ Our proposal has the following advantages: - Easy to use: Application developers are only required to decide if unique flow identifiers for publishers/subscriptions are necessary. - Light-weight implementation: Both non-DDS and DDS RMW can implement the required support conveniently using native socket API with negligible impact on performance. - RMW-agnostic: No particular network is preferred, respecting ROS2 design preferences. -- Minimum change: Introducing the *choice* of unique flow identifiers into the application layer is the minimum framework change required to obtain bespoke QoS from QoS-centric machine-machine network technologies such as 5G. +- Minimal change: The application layer is responsible for automating network QoS configuration. This represents minimal disruption to the ROS framework. ### Limitations From 5560e112fd549a1b6f494902e456a34334eb840d Mon Sep 17 00:00:00 2001 From: Ananya Muddukrishna Date: Thu, 15 Apr 2021 12:05:29 +0200 Subject: [PATCH 15/17] Correct typos Signed-off-by: Ananya Muddukrishna --- articles/unique_network_flows.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/articles/unique_network_flows.md b/articles/unique_network_flows.md index eb47a9b22..e371645ff 100644 --- a/articles/unique_network_flows.md +++ b/articles/unique_network_flows.md @@ -101,7 +101,7 @@ Our proposal to solve the problem is to make the flows of publishers and subscri Network flows are defined for UDP/TCP and IP-based RMW implementations only. It is not a limiting definition since these are the majority protocols used today. The definition can be later extended to include relevant non-IP networks such as deterministic ethernet. -- *Network Flow Endpoint (NFE)*: The portion of a network flow specific to the publisher and or the subscription. In other words, each network flow has two NFEs; one for the publisher and the other for the subscription. +- *Network Flow Endpoint (NFE)*: The portion of a network flow specific to the publisher or the subscription. In other words, each network flow has two NFEs; one for the publisher and the other for the subscription. - *Dissimilar NFE pair*: Two NFEs are dissimilar if one or more of their networking resources are different. @@ -164,7 +164,7 @@ A weakly unique network flow is created by making either the publisher or the su A strongly unique network flow is created by making both publisher and subscription NFEs unique. This enables network QoS differentiation in the rather odd case of a node having multiple publishers/subscriptions for the same topic. RMW implementations can decide not to support such odd cases i.e., not support both publisher and subscription option for unique NFEs. -We list few candidate alternatives next for RMW implementations to to implement the `require_unique_network_flow_endpoint` option. +We list few candidate alternatives next for RMW implementations to implement the `require_unique_network_flow_endpoint` option. - A simple option that works for both IPv6 and IPv4 is to select a unique transport port (UDP/TCP) and accordingly update the port field in the transport protocol header. - If the node is communicating using IPv6, then the RMW implementation can write a unique value (such as a suitable derivative of the RTPS entity ID) in the Flow Label field. From 37d444786932c06be71a54f5d8daad0806bd9756 Mon Sep 17 00:00:00 2001 From: Ananya Muddukrishna Date: Fri, 21 May 2021 15:55:01 +0200 Subject: [PATCH 16/17] Correct typo Signed-off-by: Ananya Muddukrishna --- articles/unique_network_flows.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/articles/unique_network_flows.md b/articles/unique_network_flows.md index e371645ff..248804742 100644 --- a/articles/unique_network_flows.md +++ b/articles/unique_network_flows.md @@ -27,7 +27,7 @@ In this document, we first describe essential background concepts. After that we ## Background -IP networking [1] is the pre-dominant inter-networking technology used today. Ethernet, WiFi, 4G/5G telecommunication all rely on IP networking. +IP networking [1] is the predominant inter-networking technology used today. Ethernet, WiFi, 4G/5G telecommunication all rely on IP networking. Streams of IP packets from a given source to destination are called *packet flows* or simply *flows*. Applications can uniquely identify certain flows and explicitly specify what QoS is required from the network for those flows. From 70f7a39cc90007c3fba9c40c1dd48ccff6453a19 Mon Sep 17 00:00:00 2001 From: Ananya Muddukrishna Date: Fri, 21 May 2021 17:05:46 +0200 Subject: [PATCH 17/17] Correct snippet Signed-off-by: Ananya Muddukrishna --- articles/unique_network_flows.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/articles/unique_network_flows.md b/articles/unique_network_flows.md index 248804742..4f423f10d 100644 --- a/articles/unique_network_flows.md +++ b/articles/unique_network_flows.md @@ -146,7 +146,7 @@ sub_x_ = this->create_subscription( _1), options_x); // Unique network flow endpoints not required, relying on default -auto options_2 = rclcpp::SubscriptionOptions(); +auto options_y = rclcpp::SubscriptionOptions(); sub_y_ = this->create_subscription( "topic_y", 10, std::bind(