Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

unexpected edge between "Inbound" and "Outbound" #1110

Closed
rade opened this issue Mar 3, 2016 · 31 comments
Closed

unexpected edge between "Inbound" and "Outbound" #1110

rade opened this issue Mar 3, 2016 · 31 comments
Labels
accuracy Incorrect information is being shown to the user; usually a bug bug Broken end user or developer functionality; not working as the developers intended it

Comments

@rade
Copy link
Member

rade commented Mar 3, 2016

screenshot from 2016-03-03 13 42 58

That edge should not be there, right?

Also, what is 'Requests' supposed to mean?
@rade rade added the bug Broken end user or developer functionality; not working as the developers intended it label Mar 3, 2016
@tomwilkie
Copy link
Contributor

Can we have a report please?

Also, what is 'Requests' supposed to mean?

@pidster decided on the naming.

@pidster
Copy link
Contributor

pidster commented Mar 3, 2016

Did I?

@rade
Copy link
Member Author

rade commented Mar 3, 2016

report

@pidster
Copy link
Contributor

pidster commented Mar 3, 2016

@tomwilkie issue #566 (comment) refers to "connections".

If I indicated 'requests' elsewhere, this was an error on my part. I think "connections" is more appropriate. I filed #1111

@rade
Copy link
Member Author

rade commented Mar 4, 2016

Tom reckons the cause could be related to #1122. But note that I did not make any adjustments to scope timing parameters here - the above is from a straight, single-host "scope launch".

@rade
Copy link
Member Author

rade commented Apr 4, 2016

Here is another, shorter, report.

This is from a vanilla scope, current master. I have no containers besides scope.

And here are some screenshots...

screenshot from 2016-04-04 22 11 21
screenshot from 2016-04-04 22 11 45
screenshot from 2016-04-04 22 12 14
screenshot from 2016-04-04 22 12 49

The 192.168.3.22 IP showing up in the Outbound connections of the "Inbound Internet" is not pingable. The two IPs (192.30.252.91 and 216.58.214.14) showing up in the Inbound connections of the "Outbound Internet" are pingable.

192.168.3.22 and 192.30.252.91 do not show up in netstat -an. 216.58.214.14 does.

Here are my machine's network interfaces:

$ ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether e8:2a:ea:ab:a4:73 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.65/24 brd 192.168.1.255 scope global wlan0
       valid_lft forever preferred_lft forever
    inet6 fe80::ea2a:eaff:feab:a473/64 scope link 
       valid_lft forever preferred_lft forever
3: docker_gwbridge: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:69:1e:d2:cc brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.1/16 scope global docker_gwbridge
       valid_lft forever preferred_lft forever
    inet6 fe80::42:69ff:fe1e:d2cc/64 scope link 
       valid_lft forever preferred_lft forever
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:99:fc:0c:79 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:99ff:fefc:c79/64 scope link 
       valid_lft forever preferred_lft forever
9: dummy0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default 
    link/ether 7e:c6:0d:5d:e2:f3 brd ff:ff:ff:ff:ff:ff
73: vboxnet0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
    link/ether 0a:00:27:00:00:00 brd ff:ff:ff:ff:ff:ff
    inet 192.168.48.1/24 brd 192.168.48.255 scope global vboxnet0
       valid_lft forever preferred_lft forever
    inet6 fe80::800:27ff:fe00:0/64 scope link 
       valid_lft forever preferred_lft forever
74: vboxnet1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 0a:00:27:00:00:01 brd ff:ff:ff:ff:ff:ff
75: vboxnet2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 0a:00:27:00:00:02 brd ff:ff:ff:ff:ff:ff

And here is my routing table:

$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.1.254   0.0.0.0         UG    0      0        0 wlan0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
172.18.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker_gwbridge
192.168.1.0     0.0.0.0         255.255.255.0   U     9      0        0 wlan0
192.168.48.0    0.0.0.0         255.255.255.0   U     0      0        0 vboxnet0

@tomwilkie
Copy link
Contributor

I believe this procspy rate limiting again...

@rade
Copy link
Member Author

rade commented May 5, 2016

Note that I do not see an 'Inbound connections' node in the Process view. Doesn't that discrepancy with the Container view point to a logic bug rather than data collection timing issue?

@rade
Copy link
Member Author

rade commented May 12, 2016 •
edited
Loading

The stray connections do appear in conntrack -L, so that clears up the mystery where they come from.

I was unable to reproduce this issue at work, only at home. With the same machine. Also, after rebooting my machine at home, and running all my usual apps, the problem went away.

I have a theory... perhaps conntrack entries are not removed when my IP address changes when I relocate from the offoce to home? So the 192.168.3.22 should be my IP address at the office. I shall find out tomorrow :)

This would explain why the conntrack entries have that as the src IP and are for destinations ips/ports of the services for all the usual apps I am running, e.g. spotify, xmpp. Since that IP is not on my home network, scope treats it as an "Internet" IP.

@2opremio
Copy link
Contributor

2opremio commented May 12, 2016 •
edited
Loading

scope treats it as an "Internet" IP.

It could be, but it shouldn't. Conntrack-gathered connections shouldn't appear as edges unless one of the sides matches the IP of a container, right @tomwilkie ?

@2opremio
Copy link
Contributor

The stray connections do appear in conntrack -L

What's the status of the flows from conntrack -L? Maybe TIME_WAIT / FIN_WAIT? Can you paste the output?

@rade
Copy link
Member Author

rade commented May 12, 2016

iirc it was 'ESTABLISHED'.

@2opremio
Copy link
Contributor

2opremio commented May 12, 2016 •
edited
Loading

Uhm, I think that the problem might be ESTABLISHED flows lingering because of nf_conntrack_tcp_timeout_established which defaults to 5 days.

For instance, in my system:

<flow><meta direction="original"><layer3 protonum="2" protoname="ipv4"><src>127.0.0.1</src><dst>127.0.0.1</dst></layer3><layer4 protonum="6" protoname="tcp"><sport>43606</sport><dport>4040</dport></layer4></meta><meta direction="reply"><layer3 protonum="2" protoname="ipv4"><src>127.0.0.1</src><dst>127.0.0.1</dst></layer3><layer4 protonum="6" protoname="tcp"><sport>4040</sport><dport>43606</dport></layer4></meta><meta direction="independent"><state>ESTABLISHED</state><timeout>431913</timeout><mark>0</mark><use>1</use><id>913591744</id><assured/></meta></flow>

However, that flow disappears when the connection is not established anymore.

@rade
Copy link
Member Author

rade commented May 13, 2016

So the 192.168.3.22 should be my IP address at the office. I shall find out tomorrow :)

It is.

@rade
Copy link
Member Author

rade commented May 13, 2016

iirc it was 'ESTABLISHED'.

Here's an example

tcp      6 431924 ESTABLISHED src=192.168.3.22 dst=94.125.182.252 sport=53912 dport=6667 src=94.125.182.252 dst=192.168.3.22 sport=6667 dport=53912 [ASSURED] mark=0 use=1

@2opremio
Copy link
Contributor

Can we get the xml with the timeout?

@rade
Copy link
Member Author

rade commented May 13, 2016

Can we get the xml with the timeout?

how?

@2opremio
Copy link
Contributor

conntrack -L -o xml

@2opremio
Copy link
Contributor

(if my memory serves correctly)

@2opremio
Copy link
Contributor

Also, the output of sysctl net.netfilter.nf_conntrack_tcp_timeout_established may be helpful

@rade
Copy link
Member Author

rade commented May 13, 2016

The timeout is the third column, i.e. 431924 in the above.

@rade
Copy link
Member Author

rade commented May 13, 2016 

# sysctl net.netfilter.nf_conntrack_tcp_timeout_established
net.netfilter.nf_conntrack_tcp_timeout_established = 432000

@2opremio
Copy link
Contributor

nf_conntrack_buckets and nf_conntrack_count may also be helpful

@2opremio
Copy link
Contributor

The timeout is the third column, i.e. 431924 in the above.

My bad, thanks :)

@rade
Copy link
Member Author

rade commented May 13, 2016 

# sysctl net.netfilter.nf_conntrack_buckets 
net.netfilter.nf_conntrack_buckets = 65536
# sysctl net.netfilter.nf_conntrack_count 
net.netfilter.nf_conntrack_count = 89

15 of the 89 connections have my office IP as the src.

@2opremio
Copy link
Contributor

2opremio commented May 13, 2016 •
edited
Loading

Either I don't understand conntrack and flows are supposed to linger in some situations or there's a conntrack bug. I think that the next step is to ask the netfilter guys. Maybe there's a problem due to the laptop being suspended ... who knows.

@jml
Copy link
Contributor

jml commented Jun 10, 2016 •
edited
Loading

Observed on my laptop:

20160610-173106_3196x1717_scrot

Scope report for "The Internet (Inbound connections)"

OUTBOUND    REMOTE  PORT    COUNT
The Internet    ntt-4.lastpass.com (192.168.3.173)  443 1
The Internet    stackoverflow.com (192.168.3.173)   80  1
The Internet    ea-in-f188.1e100.net (192.168.3.173)    443 1
The Internet    192.168.3.173   443 1

Scope report for "The Internet (Outbound connections)"

OUTBOUND    REMOTE  PORT    COUNT
The Internet    ntt-4.lastpass.com (192.168.3.173)  443 1
The Internet    stackoverflow.com (192.168.3.173)   80  1
The Internet    ea-in-f188.1e100.net (192.168.3.173)    443 1
The Internet    192.168.3.173   443 1

conntrack

$ sudo conntrack -L
udp      17 106 src=192.168.1.83 dst=216.58.213.69 sport=53905 dport=443 src=216.58.213.69 dst=192.168.1.83 sport=443 dport=53905 [ASSURED] mark=0 use=1
tcp      6 63 TIME_WAIT src=192.168.1.83 dst=52.3.9.140 sport=49144 dport=443 src=52.3.9.140 dst=192.168.1.83 sport=443 dport=49144 [ASSURED] mark=0 use=1
udp      17 39 src=192.168.1.83 dst=216.58.212.129 sport=49108 dport=443 src=216.58.212.129 dst=192.168.1.83 sport=443 dport=49108 [ASSURED] mark=0 use=1
tcp      6 86397 ESTABLISHED src=127.0.0.1 dst=127.0.0.1 sport=55322 dport=8000 src=127.0.0.1 dst=127.0.0.1 sport=8000 dport=55322 [ASSURED] mark=0 use=1
udp      17 18 src=192.168.1.83 dst=192.168.1.1 sport=47997 dport=53 src=192.168.1.1 dst=192.168.1.83 sport=53 dport=47997 mark=0 use=1
tcp      6 86398 ESTABLISHED src=127.0.0.1 dst=127.0.0.1 sport=32806 dport=6784 src=127.0.0.1 dst=127.0.0.1 sport=6784 dport=32806 [ASSURED] mark=0 use=1
udp      17 178 src=192.168.1.83 dst=74.125.206.189 sport=48670 dport=443 src=74.125.206.189 dst=192.168.1.83 sport=443 dport=48670 [ASSURED] mark=0 use=1
tcp      6 22 TIME_WAIT src=172.17.0.1 dst=172.17.0.1 sport=37468 dport=53 src=172.17.0.1 dst=192.168.1.83 sport=53 dport=37468 [ASSURED] mark=0 use=1
udp      17 29 src=192.168.1.124 dst=192.168.1.255 sport=137 dport=137 [UNREPLIED] src=192.168.1.255 dst=192.168.1.124 sport=137 dport=137 mark=0 use=1
udp      17 17 src=192.168.1.83 dst=239.255.255.250 sport=37886 dport=1900 [UNREPLIED] src=239.255.255.250 dst=192.168.1.83 sport=1900 dport=37886 mark=0 use=1
tcp      6 86392 ESTABLISHED src=192.168.1.83 dst=192.184.9.104 sport=54108 dport=443 src=192.184.9.104 dst=192.168.1.83 sport=443 dport=54108 [ASSURED] mark=0 use=1
tcp      6 86397 ESTABLISHED src=127.0.0.1 dst=127.0.0.1 sport=55320 dport=8000 src=127.0.0.1 dst=127.0.0.1 sport=8000 dport=55320 [ASSURED] mark=0 use=1
tcp      6 12 TIME_WAIT src=172.17.0.1 dst=172.17.0.1 sport=37462 dport=53 src=172.17.0.1 dst=192.168.1.83 sport=53 dport=37462 [ASSURED] mark=0 use=1
udp      17 113 src=192.168.1.83 dst=172.217.16.174 sport=36031 dport=443 src=172.217.16.174 dst=192.168.1.83 sport=443 dport=36031 [ASSURED] mark=0 use=1
tcp      6 86359 ESTABLISHED src=192.168.1.83 dst=54.67.97.29 sport=37744 dport=443 src=54.67.97.29 dst=192.168.1.83 sport=443 dport=37744 [ASSURED] mark=0 use=1
udp      17 119 src=192.168.1.83 dst=172.217.16.165 sport=53792 dport=443 src=172.217.16.165 dst=192.168.1.83 sport=443 dport=53792 [ASSURED] mark=0 use=1
udp      17 13 src=192.168.1.83 dst=192.168.1.1 sport=32828 dport=53 src=192.168.1.1 dst=192.168.1.83 sport=53 dport=32828 mark=0 use=1
tcp      6 32 TIME_WAIT src=172.17.0.1 dst=172.17.0.1 sport=37474 dport=53 src=172.17.0.1 dst=192.168.1.83 sport=53 dport=37474 [ASSURED] mark=0 use=1
udp      17 42 src=192.168.1.83 dst=216.58.208.163 sport=45338 dport=443 src=216.58.208.163 dst=192.168.1.83 sport=443 dport=45338 [ASSURED] mark=0 use=1
tcp      6 86386 ESTABLISHED src=192.168.1.83 dst=192.30.253.124 sport=52492 dport=443 src=192.30.253.124 dst=192.168.1.83 sport=443 dport=52492 [ASSURED] mark=0 use=1
udp      17 168 src=127.0.0.1 dst=127.0.1.1 sport=50282 dport=53 src=127.0.1.1 dst=127.0.0.1 sport=53 dport=50282 [ASSURED] mark=0 use=2
tcp      6 86397 ESTABLISHED src=127.0.0.1 dst=127.0.0.1 sport=55326 dport=8000 src=127.0.0.1 dst=127.0.0.1 sport=8000 dport=55326 [ASSURED] mark=0 use=1
tcp      6 86397 ESTABLISHED src=127.0.0.1 dst=127.0.0.1 sport=55330 dport=8000 src=127.0.0.1 dst=127.0.0.1 sport=8000 dport=55330 [ASSURED] mark=0 use=1
tcp      6 86384 ESTABLISHED src=192.168.1.83 dst=216.58.213.69 sport=39006 dport=443 src=216.58.213.69 dst=192.168.1.83 sport=443 dport=39006 [ASSURED] mark=0 use=1
udp      17 15 src=192.168.1.11 dst=192.168.1.255 sport=57621 dport=57621 [UNREPLIED] src=192.168.1.255 dst=192.168.1.11 sport=57621 dport=57621 mark=0 use=1
udp      17 18 src=192.168.1.83 dst=192.168.1.1 sport=14482 dport=53 src=192.168.1.1 dst=192.168.1.83 sport=53 dport=14482 mark=0 use=1
tcp      6 86374 ESTABLISHED src=192.168.1.83 dst=216.58.213.78 sport=45706 dport=443 src=216.58.213.78 dst=192.168.1.83 sport=443 dport=45706 [ASSURED] mark=0 use=1
tcp      6 86374 ESTABLISHED src=192.168.1.83 dst=216.58.213.78 sport=45704 dport=443 src=216.58.213.78 dst=192.168.1.83 sport=443 dport=45704 [ASSURED] mark=0 use=1
udp      17 10 src=192.168.1.83 dst=216.58.214.14 sport=36316 dport=443 src=216.58.214.14 dst=192.168.1.83 sport=443 dport=36316 [ASSURED] mark=0 use=1
udp      17 18 src=127.0.0.1 dst=127.0.1.1 sport=55460 dport=53 src=127.0.1.1 dst=127.0.0.1 sport=53 dport=55460 mark=0 use=1
tcp      6 86368 ESTABLISHED src=192.168.1.83 dst=192.30.253.124 sport=32866 dport=443 src=192.30.253.124 dst=192.168.1.83 sport=443 dport=32866 [ASSURED] mark=0 use=1
tcp      6 86355 ESTABLISHED src=192.168.1.83 dst=185.31.17.133 sport=52560 dport=443 src=185.31.17.133 dst=192.168.1.83 sport=443 dport=52560 [ASSURED] mark=0 use=1
udp      17 26 src=192.168.1.54 dst=255.255.255.255 sport=17500 dport=17500 [UNREPLIED] src=255.255.255.255 dst=192.168.1.54 sport=17500 dport=17500 mark=0 use=1
tcp      6 86388 ESTABLISHED src=192.168.1.83 dst=74.125.206.188 sport=52980 dport=5228 src=74.125.206.188 dst=192.168.1.83 sport=5228 dport=52980 [ASSURED] mark=0 use=1
udp      17 4 src=192.168.1.83 dst=91.189.94.4 sport=50647 dport=123 src=91.189.94.4 dst=192.168.1.83 sport=123 dport=50647 mark=0 use=1
udp      17 13 src=127.0.0.1 dst=127.0.1.1 sport=57993 dport=53 src=127.0.1.1 dst=127.0.0.1 sport=53 dport=57993 mark=0 use=1
tcp      6 70063 ESTABLISHED src=192.168.3.173 dst=192.184.9.104 sport=49744 dport=443 src=192.184.9.104 dst=192.168.3.173 sport=443 dport=49744 [ASSURED] mark=0 use=1
tcp      6 70060 ESTABLISHED src=192.168.3.173 dst=128.121.22.144 sport=59160 dport=443 src=128.121.22.144 dst=192.168.3.173 sport=443 dport=59160 [ASSURED] mark=0 use=1
tcp      6 86397 ESTABLISHED src=127.0.0.1 dst=127.0.0.1 sport=55328 dport=8000 src=127.0.0.1 dst=127.0.0.1 sport=8000 dport=55328 [ASSURED] mark=0 use=1
udp      17 40 src=192.168.1.83 dst=216.58.213.100 sport=55305 dport=443 src=216.58.213.100 dst=192.168.1.83 sport=443 dport=55305 [ASSURED] mark=0 use=1
udp      17 19 src=127.0.0.1 dst=127.0.1.1 sport=60258 dport=53 src=127.0.1.1 dst=127.0.0.1 sport=53 dport=60258 mark=0 use=1
udp      17 165 src=192.168.1.83 dst=172.217.16.165 sport=51068 dport=443 src=172.217.16.165 dst=192.168.1.83 sport=443 dport=51068 [ASSURED] mark=0 use=1
udp      17 45 src=192.168.1.83 dst=172.217.16.174 sport=35267 dport=443 src=172.217.16.174 dst=192.168.1.83 sport=443 dport=35267 [ASSURED] mark=0 use=1
tcp      6 86399 ESTABLISHED src=192.168.1.83 dst=52.5.255.40 sport=44314 dport=443 src=52.5.255.40 dst=192.168.1.83 sport=443 dport=44314 [ASSURED] mark=0 use=1
tcp      6 86374 ESTABLISHED src=192.168.1.83 dst=172.217.16.174 sport=60018 dport=443 src=172.217.16.174 dst=192.168.1.83 sport=443 dport=60018 [ASSURED] mark=0 use=1
tcp      6 112 TIME_WAIT src=192.168.1.83 dst=54.230.10.228 sport=41958 dport=443 src=54.230.10.228 dst=192.168.1.83 sport=443 dport=41958 [ASSURED] mark=0 use=1
tcp      6 86397 ESTABLISHED src=192.168.1.83 dst=52.90.222.61 sport=36328 dport=443 src=52.90.222.61 dst=192.168.1.83 sport=443 dport=36328 [ASSURED] mark=0 use=1
udp      17 23 src=127.0.0.1 dst=127.0.1.1 sport=52875 dport=53 src=127.0.1.1 dst=127.0.0.1 sport=53 dport=52875 mark=0 use=1
udp      17 23 src=192.168.1.83 dst=192.168.1.1 sport=35576 dport=53 src=192.168.1.1 dst=192.168.1.83 sport=53 dport=35576 mark=0 use=1
tcp      6 68232 ESTABLISHED src=192.168.1.83 dst=192.30.253.124 sport=49524 dport=443 src=192.30.253.124 dst=192.168.1.83 sport=443 dport=49524 [ASSURED] mark=0 use=1
tcp      6 86357 ESTABLISHED src=192.168.1.83 dst=216.58.213.110 sport=49636 dport=443 src=216.58.213.110 dst=192.168.1.83 sport=443 dport=49636 [ASSURED] mark=0 use=1
tcp      6 70067 ESTABLISHED src=192.168.3.173 dst=192.30.253.124 sport=43496 dport=443 src=192.30.253.124 dst=192.168.3.173 sport=443 dport=43496 [ASSURED] mark=0 use=1
tcp      6 23 TIME_WAIT src=192.168.1.83 dst=52.5.255.40 sport=44422 dport=443 src=52.5.255.40 dst=192.168.1.83 sport=443 dport=44422 [ASSURED] mark=0 use=1
tcp      6 92 TIME_WAIT src=172.17.0.1 dst=172.17.0.1 sport=37512 dport=53 src=172.17.0.1 dst=192.168.1.83 sport=53 dport=37512 [ASSURED] mark=0 use=1
tcp      6 62 TIME_WAIT src=172.17.0.1 dst=172.17.0.1 sport=37494 dport=53 src=172.17.0.1 dst=192.168.1.83 sport=53 dport=37494 [ASSURED] mark=0 use=1
udp      17 40 src=192.168.1.83 dst=216.58.213.99 sport=53210 dport=443 src=216.58.213.99 dst=192.168.1.83 sport=443 dport=53210 [ASSURED] mark=0 use=1
tcp      6 70051 ESTABLISHED src=192.168.3.173 dst=74.125.136.188 sport=59014 dport=443 src=74.125.136.188 dst=192.168.3.173 sport=443 dport=59014 [ASSURED] mark=0 use=1
tcp      6 82 TIME_WAIT src=172.17.0.1 dst=172.17.0.1 sport=37506 dport=53 src=172.17.0.1 dst=192.168.1.83 sport=53 dport=37506 [ASSURED] mark=0 use=1
tcp      6 13 TIME_WAIT src=192.168.1.83 dst=52.3.9.140 sport=49112 dport=443 src=52.3.9.140 dst=192.168.1.83 sport=443 dport=49112 [ASSURED] mark=0 use=1
udp      17 13 src=192.168.1.83 dst=192.168.1.1 sport=28811 dport=53 src=192.168.1.1 dst=192.168.1.83 sport=53 dport=28811 mark=0 use=1
udp      17 17 src=192.168.1.83 dst=224.0.0.251 sport=5353 dport=5353 [UNREPLIED] src=224.0.0.251 dst=192.168.1.83 sport=5353 dport=5353 mark=0 use=1
tcp      6 86365 ESTABLISHED src=192.168.1.83 dst=216.58.214.14 sport=53922 dport=443 src=216.58.214.14 dst=192.168.1.83 sport=443 dport=53922 [ASSURED] mark=0 use=1
udp      17 40 src=192.168.1.83 dst=216.58.213.78 sport=54402 dport=443 src=216.58.213.78 dst=192.168.1.83 sport=443 dport=54402 [ASSURED] mark=0 use=2
tcp      6 33 TIME_WAIT src=192.168.1.83 dst=52.5.255.40 sport=44428 dport=443 src=52.5.255.40 dst=192.168.1.83 sport=443 dport=44428 [ASSURED] mark=0 use=1
udp      17 3 src=127.0.0.1 dst=127.0.1.1 sport=55202 dport=53 src=127.0.1.1 dst=127.0.0.1 sport=53 dport=55202 mark=0 use=1
tcp      6 86385 ESTABLISHED src=192.168.1.83 dst=216.58.213.69 sport=38986 dport=443 src=216.58.213.69 dst=192.168.1.83 sport=443 dport=38986 [ASSURED] mark=0 use=1
tcp      6 109 TIME_WAIT src=192.168.1.83 dst=91.189.88.161 sport=39924 dport=80 src=91.189.88.161 dst=192.168.1.83 sport=80 dport=39924 [ASSURED] mark=0 use=1
tcp      6 70067 ESTABLISHED src=192.168.3.173 dst=198.252.206.25 sport=46238 dport=80 src=198.252.206.25 dst=192.168.3.173 sport=80 dport=46238 [ASSURED] mark=0 use=1
udp      17 172 src=192.168.1.83 dst=74.125.206.189 sport=60465 dport=443 src=74.125.206.189 dst=192.168.1.83 sport=443 dport=60465 [ASSURED] mark=0 use=1
udp      17 18 src=192.168.1.83 dst=192.168.1.1 sport=23347 dport=53 src=192.168.1.1 dst=192.168.1.83 sport=53 dport=23347 mark=0 use=1
tcp      6 68247 ESTABLISHED src=192.168.1.83 dst=74.125.71.188 sport=57710 dport=5228 src=74.125.71.188 dst=192.168.1.83 sport=5228 dport=57710 [ASSURED] mark=0 use=1
tcp      6 2 TIME_WAIT src=172.17.0.1 dst=172.17.0.1 sport=37454 dport=53 src=172.17.0.1 dst=192.168.1.83 sport=53 dport=37454 [ASSURED] mark=0 use=1
udp      17 26 src=192.168.1.83 dst=216.58.210.46 sport=49945 dport=443 src=216.58.210.46 dst=192.168.1.83 sport=443 dport=49945 [ASSURED] mark=0 use=1
tcp      6 86398 ESTABLISHED src=192.168.1.83 dst=52.5.255.40 sport=44410 dport=443 src=52.5.255.40 dst=192.168.1.83 sport=443 dport=44410 [ASSURED] mark=0 use=1
tcp      6 102 TIME_WAIT src=172.17.0.1 dst=172.17.0.1 sport=37518 dport=53 src=172.17.0.1 dst=192.168.1.83 sport=53 dport=37518 [ASSURED] mark=0 use=1
tcp      6 68274 ESTABLISHED src=192.168.1.83 dst=192.184.9.104 sport=41424 dport=443 src=192.184.9.104 dst=192.168.1.83 sport=443 dport=41424 [ASSURED] mark=0 use=1
tcp      6 86355 ESTABLISHED src=192.168.1.83 dst=23.235.43.133 sport=39608 dport=443 src=23.235.43.133 dst=192.168.1.83 sport=443 dport=39608 [ASSURED] mark=0 use=1
tcp      6 93 TIME_WAIT src=192.168.1.83 dst=52.3.9.140 sport=49162 dport=443 src=52.3.9.140 dst=192.168.1.83 sport=443 dport=49162 [ASSURED] mark=0 use=1
udp      17 107 src=192.168.1.83 dst=172.217.16.163 sport=36360 dport=443 src=172.217.16.163 dst=192.168.1.83 sport=443 dport=36360 [ASSURED] mark=0 use=1
tcp      6 86355 ESTABLISHED src=192.168.1.83 dst=172.217.16.174 sport=59962 dport=443 src=172.217.16.174 dst=192.168.1.83 sport=443 dport=59962 [ASSURED] mark=0 use=1
udp      17 175 src=192.168.1.83 dst=64.233.167.189 sport=40578 dport=443 src=64.233.167.189 dst=192.168.1.83 sport=443 dport=40578 [ASSURED] mark=0 use=1
udp      17 23 src=192.168.1.83 dst=192.168.1.1 sport=45049 dport=53 src=192.168.1.1 dst=192.168.1.83 sport=53 dport=45049 mark=0 use=1
tcp      6 86381 ESTABLISHED src=192.168.1.83 dst=192.30.253.124 sport=51806 dport=443 src=192.30.253.124 dst=192.168.1.83 sport=443 dport=51806 [ASSURED] mark=0 use=1
tcp      6 86355 ESTABLISHED src=192.168.1.83 dst=23.235.43.133 sport=39604 dport=443 src=23.235.43.133 dst=192.168.1.83 sport=443 dport=39604 [ASSURED] mark=0 use=1
tcp      6 86355 ESTABLISHED src=192.168.1.83 dst=23.235.43.133 sport=39628 dport=443 src=23.235.43.133 dst=192.168.1.83 sport=443 dport=39628 [ASSURED] mark=0 use=1
tcp      6 68242 ESTABLISHED src=192.168.1.83 dst=74.125.71.188 sport=57690 dport=5228 src=74.125.71.188 dst=192.168.1.83 sport=5228 dport=57690 [ASSURED] mark=0 use=1
tcp      6 86399 ESTABLISHED src=192.168.1.83 dst=52.5.255.40 sport=44402 dport=443 src=52.5.255.40 dst=192.168.1.83 sport=443 dport=44402 [ASSURED] mark=0 use=1
tcp      6 86398 ESTABLISHED src=192.168.1.83 dst=52.87.211.206 sport=49210 dport=443 src=52.87.211.206 dst=192.168.1.83 sport=443 dport=49210 [ASSURED] mark=0 use=1
tcp      6 86397 ESTABLISHED src=127.0.0.1 dst=127.0.0.1 sport=55324 dport=8000 src=127.0.0.1 dst=127.0.0.1 sport=8000 dport=55324 [ASSURED] mark=0 use=1
udp      17 13 src=127.0.0.1 dst=127.0.1.1 sport=59142 dport=53 src=127.0.1.1 dst=127.0.0.1 sport=53 dport=59142 mark=0 use=1
udp      17 135 src=192.168.1.83 dst=172.217.16.174 sport=35081 dport=443 src=172.217.16.174 dst=192.168.1.83 sport=443 dport=35081 [ASSURED] mark=0 use=1
tcp      6 86363 ESTABLISHED src=192.168.1.83 dst=216.58.210.46 sport=44324 dport=443 src=216.58.210.46 dst=192.168.1.83 sport=443 dport=44324 [ASSURED] mark=0 use=1
tcp      6 3 TIME_WAIT src=192.168.1.83 dst=52.3.9.140 sport=49104 dport=443 src=52.3.9.140 dst=192.168.1.83 sport=443 dport=49104 [ASSURED] mark=0 use=1
tcp      6 52 TIME_WAIT src=172.17.0.1 dst=172.17.0.1 sport=37488 dport=53 src=172.17.0.1 dst=192.168.1.83 sport=53 dport=37488 [ASSURED] mark=0 use=1
udp      17 23 src=127.0.0.1 dst=127.0.1.1 sport=39248 dport=53 src=127.0.1.1 dst=127.0.0.1 sport=53 dport=39248 mark=0 use=1
tcp      6 103 TIME_WAIT src=192.168.1.83 dst=52.3.9.140 sport=49166 dport=443 src=52.3.9.140 dst=192.168.1.83 sport=443 dport=49166 [ASSURED] mark=0 use=1
tcp      6 72 TIME_WAIT src=172.17.0.1 dst=172.17.0.1 sport=37500 dport=53 src=172.17.0.1 dst=192.168.1.83 sport=53 dport=37500 [ASSURED] mark=0 use=1
tcp      6 86388 ESTABLISHED src=192.168.1.83 dst=74.125.206.188 sport=52966 dport=5228 src=74.125.206.188 dst=192.168.1.83 sport=5228 dport=52966 [ASSURED] mark=0 use=1
tcp      6 113 TIME_WAIT src=192.168.1.83 dst=52.3.9.140 sport=49176 dport=443 src=52.3.9.140 dst=192.168.1.83 sport=443 dport=49176 [ASSURED] mark=0 use=1
tcp      6 38 TIME_WAIT src=192.168.1.83 dst=216.58.210.46 sport=44214 dport=443 src=216.58.210.46 dst=192.168.1.83 sport=443 dport=44214 [ASSURED] mark=0 use=1
tcp      6 43 TIME_WAIT src=192.168.1.83 dst=52.5.255.40 sport=44434 dport=443 src=52.5.255.40 dst=192.168.1.83 sport=443 dport=44434 [ASSURED] mark=0 use=1
tcp      6 53 TIME_WAIT src=192.168.1.83 dst=52.3.9.140 sport=49138 dport=443 src=52.3.9.140 dst=192.168.1.83 sport=443 dport=49138 [ASSURED] mark=0 use=1
tcp      6 83 TIME_WAIT src=192.168.1.83 dst=52.3.9.140 sport=49156 dport=443 src=52.3.9.140 dst=192.168.1.83 sport=443 dport=49156 [ASSURED] mark=0 use=1
tcp      6 86359 ESTABLISHED src=192.168.1.83 dst=54.67.97.29 sport=37746 dport=443 src=54.67.97.29 dst=192.168.1.83 sport=443 dport=37746 [ASSURED] mark=0 use=2
udp      17 26 src=192.168.1.54 dst=192.168.1.255 sport=17500 dport=17500 [UNREPLIED] src=192.168.1.255 dst=192.168.1.54 sport=17500 dport=17500 mark=0 use=1
udp      17 26 src=192.168.1.83 dst=216.58.210.46 sport=48290 dport=443 src=216.58.210.46 dst=192.168.1.83 sport=443 dport=48290 [ASSURED] mark=0 use=1
udp      17 3 src=192.168.1.83 dst=192.168.1.1 sport=27347 dport=53 src=192.168.1.1 dst=192.168.1.83 sport=53 dport=27347 mark=0 use=1
tcp      6 88 TIME_WAIT src=192.168.1.83 dst=54.230.10.228 sport=41870 dport=443 src=54.230.10.228 dst=192.168.1.83 sport=443 dport=41870 [ASSURED] mark=0 use=1
udp      17 8 src=192.168.1.11 dst=224.0.0.251 sport=61982 dport=5353 [UNREPLIED] src=224.0.0.251 dst=192.168.1.11 sport=5353 dport=61982 mark=0 use=1
udp      17 3 src=127.0.0.1 dst=127.0.1.1 sport=52527 dport=53 src=127.0.1.1 dst=127.0.0.1 sport=53 dport=52527 mark=0 use=1
udp      17 44 src=192.168.1.83 dst=216.58.213.69 sport=53970 dport=443 src=216.58.213.69 dst=192.168.1.83 sport=443 dport=53970 [ASSURED] mark=0 use=1
udp      17 45 src=192.168.1.83 dst=216.58.213.78 sport=51679 dport=443 src=216.58.213.78 dst=192.168.1.83 sport=443 dport=51679 [ASSURED] mark=0 use=1
tcp      6 86357 ESTABLISHED src=192.168.1.83 dst=216.58.213.78 sport=45910 dport=443 src=216.58.213.78 dst=192.168.1.83 sport=443 dport=45910 [ASSURED] mark=0 use=1
tcp      6 86372 ESTABLISHED src=192.168.1.83 dst=52.5.255.40 sport=43254 dport=443 src=52.5.255.40 dst=192.168.1.83 sport=443 dport=43254 [ASSURED] mark=0 use=1
tcp      6 42 TIME_WAIT src=172.17.0.1 dst=172.17.0.1 sport=37480 dport=53 src=172.17.0.1 dst=192.168.1.83 sport=53 dport=37480 [ASSURED] mark=0 use=1
tcp      6 68244 ESTABLISHED src=192.168.1.83 dst=128.121.22.144 sport=41590 dport=443 src=128.121.22.144 dst=192.168.1.83 sport=443 dport=41590 [ASSURED] mark=0 use=1
udp      17 40 src=192.168.1.83 dst=172.217.16.163 sport=44830 dport=443 src=172.217.16.163 dst=192.168.1.83 sport=443 dport=44830 [ASSURED] mark=0 use=1
tcp      6 86377 ESTABLISHED src=192.168.1.83 dst=52.90.222.61 sport=36324 dport=443 src=52.90.222.61 dst=192.168.1.83 sport=443 dport=36324 [ASSURED] mark=0 use=1
udp      17 3 src=192.168.1.83 dst=192.168.1.1 sport=47606 dport=53 src=192.168.1.1 dst=192.168.1.83 sport=53 dport=47606 mark=0 use=1
tcp      6 86390 ESTABLISHED src=192.168.1.83 dst=52.90.222.61 sport=36326 dport=443 src=52.90.222.61 dst=192.168.1.83 sport=443 dport=36326 [ASSURED] mark=0 use=1
tcp      6 68275 ESTABLISHED src=192.168.1.83 dst=52.91.153.80 sport=33454 dport=443 src=52.91.153.80 dst=192.168.1.83 sport=443 dport=33454 [ASSURED] mark=0 use=1
tcp      6 46 TIME_WAIT src=192.168.1.83 dst=52.72.191.253 sport=55974 dport=443 src=52.72.191.253 dst=192.168.1.83 sport=443 dport=55974 [ASSURED] mark=0 use=1
tcp      6 86395 ESTABLISHED src=192.168.1.83 dst=216.58.213.100 sport=54720 dport=443 src=216.58.213.100 dst=192.168.1.83 sport=443 dport=54720 [ASSURED] mark=0 use=1
tcp      6 43 TIME_WAIT src=192.168.1.83 dst=216.58.210.46 sport=44216 dport=443 src=216.58.210.46 dst=192.168.1.83 sport=443 dport=44216 [ASSURED] mark=0 use=1
tcp      6 70051 ESTABLISHED src=192.168.3.173 dst=74.125.136.188 sport=59012 dport=443 src=74.125.136.188 dst=192.168.3.173 sport=443 dport=59012 [ASSURED] mark=0 use=1
tcp      6 112 TIME_WAIT src=172.17.0.1 dst=172.17.0.1 sport=37526 dport=53 src=172.17.0.1 dst=192.168.1.83 sport=53 dport=37526 [ASSURED] mark=0 use=1
tcp      6 68267 ESTABLISHED src=192.168.1.83 dst=198.252.206.25 sport=55520 dport=443 src=198.252.206.25 dst=192.168.1.83 sport=443 dport=55520 [ASSURED] mark=0 use=1
udp      17 19 src=192.168.1.83 dst=192.168.1.1 sport=40655 dport=53 src=192.168.1.1 dst=192.168.1.83 sport=53 dport=40655 mark=0 use=1
tcp      6 73 TIME_WAIT src=192.168.1.83 dst=52.3.9.140 sport=49150 dport=443 src=52.3.9.140 dst=192.168.1.83 sport=443 dport=49150 [ASSURED] mark=0 use=1
conntrack v1.4.3 (conntrack-tools): 129 flow entries have been shown.

@jml
Copy link
Contributor

jml commented Jun 10, 2016

report.gz

192.168.1.83 is my laptop's IP address.

$ ifconfig
br-b339d37612ec Link encap:Ethernet  HWaddr 02:42:21:96:64:07  
          inet addr:172.18.0.1  Bcast:0.0.0.0  Mask:255.255.0.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

br-e937db6ee40d Link encap:Ethernet  HWaddr 02:42:44:48:11:2f  
          inet addr:172.19.0.1  Bcast:0.0.0.0  Mask:255.255.0.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

datapath  Link encap:Ethernet  HWaddr 82:34:ad:33:8e:0a  
          inet6 addr: fe80::8034:adff:fe33:8e0a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1410  Metric:1
          RX packets:1290 errors:0 dropped:0 overruns:0 frame:0
          TX packets:634 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:116302 (116.3 KB)  TX bytes:66639 (66.6 KB)

docker0   Link encap:Ethernet  HWaddr 02:42:13:cc:f5:2b  
          inet addr:172.17.0.1  Bcast:0.0.0.0  Mask:255.255.0.0
          inet6 addr: fe80::42:13ff:fecc:f52b/64 Scope:Link
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:12899 errors:0 dropped:0 overruns:0 frame:0
          TX packets:15128 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:938560 (938.5 KB)  TX bytes:1652517 (1.6 MB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:69491 errors:0 dropped:0 overruns:0 frame:0
          TX packets:69491 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:38934895 (38.9 MB)  TX bytes:38934895 (38.9 MB)

vethwe-bridge Link encap:Ethernet  HWaddr 86:d9:4a:bf:9b:b2  
          inet6 addr: fe80::84d9:4aff:febf:9bb2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1410  Metric:1
          RX packets:1268 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1291 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:133278 (133.2 KB)  TX bytes:134452 (134.4 KB)

vethwe-datapath Link encap:Ethernet  HWaddr 76:6b:56:68:d7:59  
          inet6 addr: fe80::746b:56ff:fe68:d759/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1410  Metric:1
          RX packets:1291 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1268 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:134452 (134.4 KB)  TX bytes:133278 (133.2 KB)

vxlan-6784 Link encap:Ethernet  HWaddr 3e:51:93:08:69:ce  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:1 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

weave     Link encap:Ethernet  HWaddr 6e:58:dd:4c:43:9a  
          inet6 addr: fe80::6c58:ddff:fe4c:439a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1410  Metric:1
          RX packets:1305 errors:0 dropped:0 overruns:0 frame:0
          TX packets:618 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:117714 (117.7 KB)  TX bytes:64927 (64.9 KB)

wlp2s0    Link encap:Ethernet  HWaddr c4:8e:8f:f6:8c:39  
          inet addr:192.168.1.83  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::c68e:8fff:fef6:8c39/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:387352 errors:0 dropped:0 overruns:0 frame:408561
          TX packets:296176 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:330979753 (330.9 MB)  TX bytes:78937565 (78.9 MB)
          Interrupt:19

@jml
Copy link
Contributor

jml commented Jun 10, 2016 

$ journalctl | grep 192.168.3.173
Jun 10 12:47:56 wit dhclient[10410]: DHCPREQUEST of 192.168.3.173 on wlp2s0 to 255.255.255.255 port 67 (xid=0x6fa9a714)
Jun 10 12:47:56 wit dhclient[10410]: DHCPOFFER of 192.168.3.173 from 192.168.2.1
Jun 10 12:47:56 wit dhclient[10410]: DHCPACK of 192.168.3.173 from 192.168.2.1
Jun 10 12:47:56 wit NetworkManager[797]: <info>  [1465559276.5506]   address 192.168.3.173
Jun 10 12:47:56 wit avahi-daemon[814]: Joining mDNS multicast group on interface wlp2s0.IPv4 with address 192.168.3.173.
Jun 10 12:47:56 wit avahi-daemon[814]: Registering new address record for 192.168.3.173 on wlp2s0.IPv4.
Jun 10 12:47:56 wit dhclient[10410]: bound to 192.168.3.173 -- renewal in 42624 seconds.
Jun 10 13:18:13 wit avahi-daemon[814]: Withdrawing address record for 192.168.3.173 on wlp2s0.
Jun 10 13:18:13 wit avahi-daemon[814]: Leaving mDNS multicast group on interface wlp2s0.IPv4 with address 192.168.3.173.

Looks like 192.168.3.173 was my old IP address on wifi, apparently.

@2opremio
Copy link
Contributor

Yep, the key question is: Why do those flows linger when the connections clearly don't exist anymore?

Maybe I don't understand conntrack properly but that sounds like a bug to me.

@rade rade mentioned this issue Jul 4, 2016
Closed
@2opremio 2opremio mentioned this issue Jul 5, 2016
Closed
@2opremio
Copy link
Contributor

2opremio commented Jul 7, 2016

I've run into this for the first time, on my development VM. I had to run sudo conntrack -F to fix it.

My VM uses a reasonable recent kernel:

$ uname -a
Linux vagrant-ubuntu-wily-64 4.2.0-34-generic #39-Ubuntu SMP Thu Mar 10 22:13:01 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

@rade rade added the accuracy Incorrect information is being shown to the user; usually a bug label Jan 11, 2017
This was referenced Nov 28, 2017
Closed
Merged
rade added a commit that referenced this issue Nov 29, 2017
@rade
Merge pull request #2954 from weaveworks/filter-internet-adjacencies
d5eab89 
filter internet adjacencies

Fixes #1110.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
accuracy Incorrect information is being shown to the user; usually a bug bug Broken end user or developer functionality; not working as the developers intended it
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@rade @jml @pidster @tomwilkie @2opremio