Docker and NAT to LAN on same machine using iptables

by twbarr   Last Updated July 12, 2019 01:00 AM

I've been using iptables on my lab server (Ubuntu 18.04) to perform NAT to the rest of the devices on my network for a while now:

-t nat -A PREROUTING -i eno1 -p tcp -m tcp --dport 23 -j DNAT --to-destination 10.0.1.2:22
-t nat -A POSTROUTING -o eno1 -j MASQUERADE

-A FORWARD -s 10.0.0.0/24 -i eno2 -o eno1 -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -d 10.0.1.2 -p tcp -m tcp --dport 22 -j ACCEPT

In the past, it's worked nicely. However, it broke when I installed Docker. This is almost certainly due to Docker rewriting all my iptables rules. By default, some of my rules survive:

% sudo iptables -t nat -v -L                                                                                                                        
Chain PREROUTING (policy ACCEPT 257 packets, 36440 bytes)
 pkts bytes target     prot opt in     out     source               destination
    6  1384 DNAT       tcp  --  eno1   any     anywhere             anywhere             tcp dpt:telnet to:10.0.1.2:22
  133  8676 DOCKER     all  --  any    any     anywhere             anywhere             ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 122 packets, 8474 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 42 packets, 3008 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER     all  --  any    any     anywhere            !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 21 packets, 2395 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 MASQUERADE  all  --  any    !docker0  172.17.0.0/16        anywhere
    0     0 MASQUERADE  all  --  any    !br-643d6580203c  172.18.0.0/16        anywhere
   39  2900 MASQUERADE  all  --  any    eno1    anywhere             anywhere
    0     0 MASQUERADE  tcp  --  any    any     172.18.0.2           172.18.0.2           tcp dpt:8443

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RETURN     all  --  docker0 any     anywhere             anywhere
    0     0 RETURN     all  --  br-643d6580203c any     anywhere             anywhere
    0     0 DNAT       tcp  --  !br-643d6580203c any     anywhere             anywhere             tcp dpt:https to:172.18.0.2:8443

% sudo iptables -v -L                                                                                                                               
Chain INPUT (policy ACCEPT 600 packets, 44910 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy DROP 135 packets, 27966 bytes)
 pkts bytes target     prot opt in     out     source               destination
  176 32752 DOCKER-USER  all  --  any    any     anywhere             anywhere
  176 32752 DOCKER-ISOLATION-STAGE-1  all  --  any    any     anywhere             anywhere
    0     0 ACCEPT     all  --  any    docker0  anywhere             anywhere             ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  any    docker0  anywhere             anywhere
    0     0 ACCEPT     all  --  docker0 !docker0  anywhere             anywhere
    0     0 ACCEPT     all  --  docker0 docker0  anywhere             anywhere
    0     0 ACCEPT     all  --  any    br-643d6580203c  anywhere             anywhere             ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  any    br-643d6580203c  anywhere             anywhere
    0     0 ACCEPT     all  --  br-643d6580203c !br-643d6580203c  anywhere             anywhere
    0     0 ACCEPT     all  --  br-643d6580203c br-643d6580203c  anywhere             anywhere
    0     0 ACCEPT     all  --  eno2   eno1    10.0.0.0/24          anywhere             ctstate NEW
   23  2682 ACCEPT     all  --  any    any     anywhere             anywhere             ctstate RELATED,ESTABLISHED
    6  1384 ACCEPT     tcp  --  any    any     anywhere             dione                tcp dpt:ssh

Chain OUTPUT (policy ACCEPT 505 packets, 66607 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     tcp  --  !br-643d6580203c br-643d6580203c  anywhere             172.18.0.2           tcp dpt:8443

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  anywhere             anywhere
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  br-643d6580203c !br-643d6580203c  anywhere             anywhere
  176 32752 RETURN     all  --  any    any     anywhere             anywhere

Chain DOCKER-ISOLATION-STAGE-2 (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  any    docker0  anywhere             anywhere
    0     0 DROP       all  --  any    br-643d6580203c  anywhere             anywhere
    0     0 RETURN     all  --  any    any     anywhere             anywhere

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
  176 32752 RETURN     all  --  any    any     anywhere             anywhere

For example, the static routes work. I can still access my workstation at 10.0.1.2 via port 22, but that same machine can't get out. Looking at the traffic out of the server, it looks like a ping isn't even making it out, much less back.

I tried simply adding my rules back in on top of the running Docker instance, but that didn't work. The documentation for Docker suggests putting things in the DOCKER-USER chain, though that doesn't exist in the nat table. The docker documentation also suggests that I can just turn off Docker's mangling of the table, though I don't know how I'd route the network manually to the containers.

Honestly, I just don't know enough about what Docker's rules do. Has anybody made this work?



Related Questions


Updated December 19, 2018 18:00 PM

Updated April 12, 2017 03:00 AM

Updated October 14, 2018 07:00 AM

Updated February 24, 2017 00:00 AM