Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PoC: use ip route to read IPv6 routes instead of /proc #3275

Closed
wants to merge 1 commit into from

Conversation

grawity
Copy link

@grawity grawity commented Jul 1, 2021

Note: This is a proof-of-concept only, handling just IPv6 routes at the moment (although it would be almost exactly same code for IPv4 too), and only tested on Arch (iproute2 v5.13, python v3.9). The intent is to check whether this approach would be suitable for scapy.

The legacy /proc/net interface has several disadvantages:

  1. It is very slow, taking about 2-3 ms per route (about 60 times slower than netlink).
  2. It doesn't support filtering by table, making it even slower and potentially confusing scapy when it sees multiple routes for the same destination.
  3. It does not include many parameters that Linux routes have (such as routing table IDs or multiple nexthops).

For example, my system has ~60 routes in the 'main' table and ~120K routes in the 'inet' table. It is useless for scapy to scan routes from non-default tables because it does not process policy rules anyway; it would be better if it only looked at the 'main' table.

With netlink it is possible to request a filtered dump (even though getting the complete dump is already reasonably fast), but /proc/net does not even have an indication of which table a route belongs to.)

Of course, it would be much better to use a native Python netlink module instead of shelling out to ip (especially considering that the -json option only became available last year), but I honestly don't know which one to use, and even the current approach shows quite an improvement:

  • /proc (all tables): 115690 routes in 304.03 seconds
  • iproute2 (all tables): 115651 routes in 5.26 seconds
  • iproute2 (main table): 32 routes in 0.08 seconds

(The small differences in route count come mostly from BGP announcements and withdrawals happening between dumps.)

Ref: #2133

The /proc/net interface has two disadvantages:

 1. It is very slow, taking about 2-3 ms per route (about 60 times
    slower than netlink).
 2. It doesn't support filtering by table, making it even slower *and*
    potentially confusing scapy when it sees multiple routes for the
    same destination.

For example, my system has ~60 routes in the 'main' table and ~120K
routes in the 'inet' table. It is useless for scapy to scan routes from
non-default tables because it does not process policy rules anyway; it
would be better if it only looked at the 'main' table.

With netlink it is possible to request a filtered dump (even though
getting the complete dump is already reasonably fast), but /proc/net
does not even have an indication of which table a route belongs to.)

Of course, it would be *much* better to use a native Python netlink
module instead of shelling out to `ip` (especially considering that the
-json option only became available last year),

But even the current approach shows quite an improvement:

 - /proc (all tables): 115690 routes in 304.03170132637024 seconds
 - iproute2 (all tables): 115651 routes in 5.262296199798584 seconds
 - iproute2 (main table): 32 routes in 0.07946968078613281 seconds

(The small differences in route count come mostly from BGP announcements
and withdrawals happening between dumps.)

Ref: secdev#2133
@codecov
Copy link

codecov bot commented Jul 1, 2021

Codecov Report

Merging #3275 (4373e44) into master (6eef12d) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3275   +/-   ##
=======================================
  Coverage   52.38%   52.38%           
=======================================
  Files           9        9           
  Lines        1365     1365           
=======================================
  Hits          715      715           
  Misses        650      650           

@guedou
Copy link
Member

guedou commented Jul 1, 2021

Thanks for this PR! That is a really interesting approach. I think that is something that we should also do for IPv4.

Following on your idea, it will be awesome to be able to load specific tables using something like:

conf.route.routes = []
conf.route.resync(table="mytable")

A last thing: could you add some unit tests by mocking the output of ip route -json?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants