engine/networking overhaul

[dvdksn]

Proposed changes

Complete overhaul of networking documentation: drivers, ipv6, iptables/firewalls

• fixes missing command in the documentation of enabling ipv6 in docker #16944

[netlify]

✅ Deploy Preview for docsdocker ready!

Name	Link
🔨 Latest commit	2be681d
🔍 Latest deploy log	https://app.netlify.com/sites/docsdocker/deploys/647dd2d210e9f70009755fcc
😎 Deploy Preview	https://deploy-preview-17176--docsdocker.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site settings.

[akerouanton]

We also need a section in the firewalling docs about iptables-nftables. Currently, dockerd only supports iptables-legacy. On some OS, users have to explicitly switch to that (by creating the proper symlink, or by using update-alternatives) or dockerd might break in unexpected ways (including at startup).

[akerouanton]

@dvdksn The command shown in network/drivers/none.md is weird and its output is wrong. I'd rather replace the three steps with a single one:

$ docker run --rm --network none alpine:latest ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

EDIT: maybe we can actually add this command too and add a note that say no IPv6 loopback address is configured when using none 🤔

$ docker run --rm --network none --name no-net-alpine alpine:latest ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever

[dvdksn]

@akerouanton ptal!

[akerouanton]

I can't edit the original description, so let me dump links to related issues here:

• Commit "engine: add note about port mapping security" is related to Summary of the issues about custom iptables rules moby/moby#45524
• Commit "engine: update description about default bind address for port mapping" is related to Summary of the issues about custom iptables rules moby/moby#45524
• Commit "engine: describe DNAT limitation of DOCKER-USER iptables chain" is related to Summary of the issues about custom iptables rules moby/moby#45524
• Commit "engine: describe iptables conflict with ufw" is related to Summary of the issues about custom iptables rules moby/moby#45524
• Commit "engine: fix examples for the none nw driver" is related to Both description and example of --network none are misleading #10415
• Commit "engine: note limit on no. of containers per bridge" is related to Docker network failures with more than 1002 containers moby/moby#44973
• Commit "engine: remove dated section on ip forwarding" is related to Default bridge configuration allows outside world connectivity #9022

@thaJeztah ISTR we closed an issue not so long ago about IPv6 resolver ending up in containers' /etc/resolv.conf, which was leading to flaky DNS resolution under Alpine-based images. I can't find it anymore, do you have any idea which one it is? Following commits are related to that:

• 3a62f3f
• 368a7cf

[dvdksn]

@akerouanton @thaJeztah I don't think we should add anything more to this PR -- so unless you have any change requests I suggest we get this merged. I have couple of follow-ups to come.

[polarathene]

My grammar isn't great, so I could be mistaken if a comma properly connects the two statements 😅

Suggested change

	The `ipv6` and `fixed-cidr-v6` parameters are optional.
	They assign an IPv6 subnet to the default bridge network.
	The `ipv6` and `fixed-cidr-v6` parameters are optional,
	they assign an IPv6 subnet to the default bridge network.

[akerouanton]

Users need to set default-address-pools to let the daemon assign IPv6 addresses to their custom network automatically (as described below).

Suggested change

	default bridge network, and to user-defined networks configured with an IPv6 subnet.
	default bridge network.

[polarathene]

Users need to set default-address-pools to let the daemon assign IPv6 addresses

docker network create --ipv6 --subnet fd00:d0ca::/112?

You can assign an IPv6 subnet explicitly for user-defined networks without default-address-pools being adjusted.

The statement doesn't say anything about automatically configuring IPv6 subnet for a user-defined network, just that containers with a user-defined network that has already been configured with an IPv6 subnet would enable the container to be assigned an IPv6 address from it.

[akerouanton]

The statement doesn't say anything about automatically configuring IPv6 subnet for a user-defined network

It talks about automatically assigning IPv6 addresses to containers:

Upon restart, the daemon assigns IPv6 addresses to containers connected [...] to user-defined networks configured with an IPv6 subnet.

My point is: experimental and ip6tables are needed to have the same experience with IPv6 as with IPv4 (wrt. isolation & port mapping), whether users rely on static or dyanmic allocation. But ipv6 and fixed-cidr-v6 are only required to let the daemon assign IPv6 addresses to containers connected to the default bridge network. Above options won't turn on dynamic allocation for custom networks, hence my suggestion.

Given the state of IPv6 support currently, and the fact old networking mode (with the default bridge) still exist, I also prefer to just tell users to configure ipv6 and fixed-cidr-v6 such that the default bridge and custom networks just work the same way.

[polarathene]

It talks about automatically assigning IPv6 addresses to containers:

Which is correct. That happens when the network (default bridge, or user-defined nework) has IPv6 subnet already configured?

My point is: experimental and ip6tables are needed to have the same experience with IPv6 as with IPv4 (wrt. isolation & port mapping), whether users rely on static or dyanmic allocation.

Yes these two settings are quite important. It's unfortunate ip6tables remains opt-in while userland-proxy: true is the default due to surprises that causes on an IPv6 host even when docker networks are IPv4 only.

Above options won't turn on dynamic allocation for custom networks, hence my suggestion.

I tried to communicate this distinction earlier in review, the two settings from the earlier JSON snippet could belong in their own section that enables IPv6 for the default (legacy) bridge.

A section that follows after it could then cover the same for user-defined networks with an explicit subnet given with docker network create or in the compose.yaml. After that, you detail implicit subnet assignment which almost works like creating IPv4 networks, but still requires opt-in for IPv6 (--ipv6 / enable_ipv6: true).

I also prefer to just tell users to configure ipv6 and fixed-cidr-v6 such that the default bridge and custom networks just work the same way.

I understand :)

I just think it's worth the distinction that the two settings are optional, only related to enabling the support on the bridge. Some users aren't aware of that in the past and think it's required even when using compose.yaml.

[akerouanton]

Which is correct. That happens when the network (default bridge, or user-defined nework) has IPv6 subnet already configured?

Right, but that's unrelated to the daemon parameters we're advertising in this section.

I tried to communicate this distinction earlier in review, the two settings from the earlier JSON snippet could belong in their own section that enables IPv6 for the default (legacy) bridge.

Got you. @dvdksn WDYT about what @polarathene proposes:

• Adding a section dedicated to the default bridge and moving ipv6 and fixed-cidr-v6 there ;
• Adding a section dedicated to static allocation ;

We can discuss this proposal on Slack if you're unsure what should go in these new sections.

[dvdksn]

Great, I think this would help clear things up. I'll give it it a go!

[dvdksn]

ptal 2be681d

[akerouanton]

The statement doesn't say anything about automatically configuring IPv6 subnet for a user-defined network

It talks about automatically assigning IPv6 addresses to containers:

Upon restart, the daemon assigns IPv6 addresses to containers connected [...] to user-defined networks configured with an IPv6 subnet.

My point is: experimental and ip6tables are needed to have the same experience with IPv6 as with IPv4 (wrt. isolation & port mapping), whether users rely on static or dyanmic allocation. But ipv6 and fixed-cidr-v6 are only required to let the daemon assign IPv6 addresses to containers connected to the default bridge network. Above options won't turn on dynamic allocation for custom networks, hence my suggestion.

Given the state of IPv6 support currently, and the fact old networking mode (with the default bridge) still exist, I also prefer to just tell users to configure ipv6 and fixed-cidr-v6 such that the default bridge and custom networks just work the same way.

[akerouanton]

I think you can create a pool of subnets larger than 2^24, it just depends on the memory the system has available.

You can, but it grows exponentially. A difference of 24 bits between the pool size and the desired subnet size already consumes 250MiB. 28 bits will consume 4GiB and 32 bits, 64GiB. Hence my initial recommendation I asked @dvdksn to put here.

Also, I prefer to precisely describe the limitation instead of vaguely stating it, as the latter case implies users have to discover it by themselves.

[akerouanton]

@dvdksn Can you add an item on your todo list about that please ⬆️?

[akerouanton]

LGTM 🎉

[thaJeztah]

LGTM; are there commits that need to be squashed before merging? (the rename commits probably must be kept separate)

also kept one comment about the "experimental" part, but we can look at that in a follow-up (I think we need to slightly make that more clear)