Target IP filter blocking all traffic

[mlafeldt]

Hey,

While playing around with Pumba 0.4.2, I noticed that the target IP filter is blocking all traffic instead of affecting the specified IP only.

I'm starting a test container in which I run ping:

docker rm -f ubuntu;docker run -d --name ubuntu ubuntu /bin/bash -c 'sleep 3600'
docker exec -it ubuntu /bin/bash

After injecting the delay with Pumba, all traffic appears to be blocked:

ubuntu@gremlin-1:~$ sudo pumba --debug netem --tc-image gaiadocker/iproute2 --duration 60s --target 8.8.8.8 delay --time 3000 ubuntu
DEBU[0000] No interval, running only once
INFO[0000] netem: delay for containers
DEBU[0000] Retrieving running containers
DEBU[0000] Running container: /ubuntu - (75176c1a20decd07caed2c18c06870196f45a94eba7cda0e4eb851d6d6abda03)
INFO[0000] Running netem command '[delay 3000ms 10ms 20.00]' on container 75176c1a20decd07caed2c18c06870196f45a94eba7cda0e4eb851d6d6abda03 with filter 8.8.8.8 for 1m0s
INFO[0000] Start netem for container 75176c1a20decd07caed2c18c06870196f45a94eba7cda0e4eb851d6d6abda03 on 'eth0' with command '[delay 3000ms 10ms 20.00]', filter by IP '8.8.8.8'
DEBU[0000] handleCommand [qdisc add dev eth0 root handle 1: prio]
DEBU[0000] target tc image: gaiadocker/iproute2
DEBU[0000] Container Config: {   %!s(bool=false) %!s(bool=false) %!s(bool=false) map[] %!s(bool=false) %!s(bool=false) %!s(bool=false) [] [qdisc add dev eth0 root handle 1: prio] %!s(*container.HealthConfig=<nil>) %!s(bool=false) gaiadocker/iproute2 map[]  [tc] %!s(bool=false)  [] map[com.gaiaadm.pumba.skip:true]  %!s(*int=<nil>) []}
DEBU[0000] Host Config: {[]  { map[]} container:75176c1a20decd07caed2c18c06870196f45a94eba7cda0e4eb851d6d6abda03 map[] { %!s(int=0)} %!s(bool=true)  [] [NET_ADMIN] [] [] [] [] [] []   [] %!s(int=0)  %!s(bool=false) %!s(bool=false) %!s(bool=false) [] map[] map[]   %!s(int64=0) map[]  [%!s(uint=0) %!s(uint=0)]  {%!s(int64=0) %!s(int64=0) %!s(int64=0)  %!s(uint16=0) [] [] [] [] [] %!s(int64=0) %!s(int64=0) %!s(int64=0) %!s(int64=0)   [] %!s(int64=0) %!s(int64=0) %!s(int64=0) %!s(int64=0) %!s(*int64=<nil>) %!s(*bool=<nil>) %!s(int64=0) [] %!s(int64=0) %!s(int64=0) %!s(uint64=0) %!s(uint64=0)} [] %!s(*bool=<nil>) }
DEBU[0000] tc container id: e3f56137bddc2f6ad234f7066d5036314e71dde660be1721751c149be33fa80c
DEBU[0000] netemCommand [qdisc add dev eth0 parent 1:3 netem delay 3000ms 10ms 20.00]
DEBU[0000] target tc image: gaiadocker/iproute2
DEBU[0000] Container Config: {   %!s(bool=false) %!s(bool=false) %!s(bool=false) map[] %!s(bool=false) %!s(bool=false) %!s(bool=false) [] [qdisc add dev eth0 parent 1:3 netem delay 3000ms 10ms 20.00] %!s(*container.HealthConfig=<nil>) %!s(bool=false) gaiadocker/iproute2 map[]  [tc] %!s(bool=false)  [] map[com.gaiaadm.pumba.skip:true]  %!s(*int=<nil>) []}
DEBU[0000] Host Config: {[]  { map[]} container:75176c1a20decd07caed2c18c06870196f45a94eba7cda0e4eb851d6d6abda03 map[] { %!s(int=0)} %!s(bool=true)  [] [NET_ADMIN] [] [] [] [] [] []   [] %!s(int=0)  %!s(bool=false) %!s(bool=false) %!s(bool=false) [] map[] map[]   %!s(int64=0) map[]  [%!s(uint=0) %!s(uint=0)]  {%!s(int64=0) %!s(int64=0) %!s(int64=0)  %!s(uint16=0) [] [] [] [] [] %!s(int64=0) %!s(int64=0) %!s(int64=0) %!s(int64=0)   [] %!s(int64=0) %!s(int64=0) %!s(int64=0) %!s(int64=0) %!s(*int64=<nil>) %!s(*bool=<nil>) %!s(int64=0) [] %!s(int64=0) %!s(int64=0) %!s(uint64=0) %!s(uint64=0)} [] %!s(*bool=<nil>) }
DEBU[0000] tc container id: 95cc004c3fcd8c1d3e644b3eef380f5da9b1dd546ba3b76348a86d2dcf264410
DEBU[0000] filterCommand [filter add dev eth0 protocol ip parent 1:0 prio 3 u32 match ip dport 8.8.8.8 flowid 1:3]
DEBU[0000] target tc image: gaiadocker/iproute2
DEBU[0000] Container Config: {   %!s(bool=false) %!s(bool=false) %!s(bool=false) map[] %!s(bool=false) %!s(bool=false) %!s(bool=false) [] [filter add dev eth0 protocol ip parent 1:0 prio 3 u32 match ip dport 8.8.8.8 flowid 1:3] %!s(*container.HealthConfig=<nil>) %!s(bool=false) gaiadocker/iproute2 map[]  [tc] %!s(bool=false)  [] map[com.gaiaadm.pumba.skip:true]  %!s(*int=<nil>) []}
DEBU[0000] Host Config: {[]  { map[]} container:75176c1a20decd07caed2c18c06870196f45a94eba7cda0e4eb851d6d6abda03 map[] { %!s(int=0)} %!s(bool=true)  [] [NET_ADMIN] [] [] [] [] [] []   [] %!s(int=0)  %!s(bool=false) %!s(bool=false) %!s(bool=false) [] map[] map[]   %!s(int64=0) map[]  [%!s(uint=0) %!s(uint=0)]  {%!s(int64=0) %!s(int64=0) %!s(int64=0)  %!s(uint16=0) [] [] [] [] [] %!s(int64=0) %!s(int64=0) %!s(int64=0) %!s(int64=0)   [] %!s(int64=0) %!s(int64=0) %!s(int64=0) %!s(int64=0) %!s(*int64=<nil>) %!s(*bool=<nil>) %!s(int64=0) [] %!s(int64=0) %!s(int64=0) %!s(uint64=0) %!s(uint64=0)} [] %!s(*bool=<nil>) }
DEBU[0000] tc container id: 708abf00ae86d36aa6f702ed014c1e1fc962a4ed75b0906d0ad2788d75bce959
^CDEBU[0046] Recieved signal: 2
DEBU[0046] Stopping netem by stop event
INFO[0046] Stopping netem on container 75176c1a20decd07caed2c18c06870196f45a94eba7cda0e4eb851d6d6abda03
INFO[0046] Stop netem for container 75176c1a20decd07caed2c18c06870196f45a94eba7cda0e4eb851d6d6abda03 on 'eth0'
DEBU[0046] netem command 'qdisc del dev eth0 root netem'
DEBU[0046] target tc image: gaiadocker/iproute2
DEBU[0046] Container Config: {   %!s(bool=false) %!s(bool=false) %!s(bool=false) map[] %!s(bool=false) %!s(bool=false) %!s(bool=false) [] [qdisc del dev eth0 root netem] %!s(*container.HealthConfig=<nil>) %!s(bool=false) gaiadocker/iproute2 map[]  [tc] %!s(bool=false)  [] map[com.gaiaadm.pumba.skip:true]  %!s(*int=<nil>) []}
DEBU[0046] Sending stop signal to runnung chaos commands ...
DEBU[0046] Host Config: {[]  { map[]} container:75176c1a20decd07caed2c18c06870196f45a94eba7cda0e4eb851d6d6abda03 map[] { %!s(int=0)} %!s(bool=true)  [] [NET_ADMIN] [] [] [] [] [] []   [] %!s(int=0)  %!s(bool=false) %!s(bool=false) %!s(bool=false) [] map[] map[]   %!s(int64=0) map[]  [%!s(uint=0) %!s(uint=0)]  {%!s(int64=0) %!s(int64=0) %!s(int64=0)  %!s(uint16=0) [] [] [] [] [] %!s(int64=0) %!s(int64=0) %!s(int64=0) %!s(int64=0)   [] %!s(int64=0) %!s(int64=0) %!s(int64=0) %!s(int64=0) %!s(*int64=<nil>) %!s(*bool=<nil>) %!s(int64=0) [] %!s(int64=0) %!s(int64=0) %!s(uint64=0) %!s(uint64=0)} [] %!s(*bool=<nil>) }
DEBU[0046] tc container id: 8d050df267c0ad6248796c3a6e7ce903e5816208f082e4713c18a6b6e9285f51
DEBU[0046] Graceful exit :-)

Also, stopping Pumba does not seem to revert the impact properly, leaving me with a broken container.

Affecting all traffic without --target works just fine though.

[alexei-led]

Thank you for reporting.
Will take a look.

I will gladly accept PR with issue fix, if you have time and desire to assist.

[mlafeldt]

We already found out that this should be dst instead of dport: https://github.com/gaia-adm/pumba/blob/90c1ab9516901c9e3e66183aa140d17201fb7b1b/container/client.go#L293

However, that won't fix the issue.

[mlafeldt]

This appears to be a general problem with Docker and tc. I opened an upstream issue at moby/moby#33162

[alexei-led]

Thank you, will follow this issue too

[mlafeldt]

You can find the solution in the upstream issue.

[Tyson1986]

Any news about this issue? Looks like solution found in moby/moby#33162

[alexei-led]

I've added a delete for 2 qdisc added when using filter IP. Now after netem command completes, traffic control settings are properly restored.