Natively support IPv6

[shykes]

Docker should provide ipv6 connectivity to its containers natively.

[shykes]

See pull request #2974

[MalteJ]

Opened a pull request (#8947) that would solve this issue

[MalteJ]

#8947 is merged
We have IPv6 support :-)

[thomasschaeferm]

Thanks for the work. I (in reality stefan-it) can confirm that --fixed-cidr-v6 with an extra prefix works.
You write in Issue 10159
"You need to have an IPv6 subnet routed to your host machine. From that subnet you can use addresses for your containers."

Is this the only possibility to use IPv6?
Why not just "bridging" the LAN and the containers, so that slaac/dhcpv6 works with one /64 instead of an /64 for every single container?

My network is big enough (/48) but I have a very restricted access to the router, so the extra route configuration has a big extra effort to me.

[MalteJ]

Hi Thomas,
at first, you don't need a /64 for every single container. A container will only get one IPv6 address. From a subnet of a size that is up to you (I recommend /80 minimum). We don't use SLAAC so we don't depend on a /64.
Yes, at the moment the routed configuration is the only supported one.
We cannot share a subnet between multiple hosts. The reason for this is hidden in the Docker internals: The IP addresses are defined before the container is running. And docker inspect shows only the IP addresses the container should have and not those it actually has.
Well, if you want to hack it, it may even be possible to use a bridged network and use SLAAC for assigning addresses to the containers. The problem is you will not see them in docker inspect. But you can see them via ifconfig from the inside of the container.
I have tried this a few months ago. There is a chance it is still working. Probably you can even see them from the outside using some ip namespace foo.

To emphasize it once again:
A /64 subnet for your whole datacenter should be enough. You can split it up into /72 subnets for each rack and /80 subnets for every host (255 racks with 255 hosts each, with 2^48 containers each, minus some broadcast addresses).

[thomasschaeferm]

Thank you for the detailed explanation. If I route a /64 to one host I can start 4 containers with /66 each?

[MalteJ]

No, we do not assign subnets to containers. Just single addresses.

[thomasschaeferm]

Now l completely confused. The option is an prefix? You take only one address out of this prefix? Is that right?

[MalteJ]

yes, that's right. Every container gets one address out of the subnet you specify with --fixed-cidr-v6. The subnet you define via this flag is the subnet all containers of ONE hosts are placed into.

[thomasschaeferm]

I am still not very happy with the "routed" configuration. This is very hard. Here at the Uni the routers are in the hand of the lrz. At home (speedport, fritz.box, ....except openwrt) you have no chance to set extra routes.
In respect to your last big comment (about docker inspect and the reasons why slaac does'nt work)
One further question:

Is your solution compatible with an /128 prefix (alias host) from the same LAN with an additional ndp-proxy-entry?

[MalteJ]

NDP-Proxying may work. I haven’t tried it yet.

Am 21.01.2015 um 09:17 schrieb thomasschaeferm notifications@github.com:

I am still not very happy with the "routed" configuration. This is very hard. Here at the Uni the routers are in the hand of the lrz. At home (speedport, fritz.box, ....except openwrt) you have no chance to set extra routes.
In respect to your last big comment (about docker inspect and the reasons why slaac does'nt work)
One further question:

Is your solution compatible with an /128 prefix (alias host) from the same LAN with an additional ndp-proxy-entry?

—
Reply to this email directly or view it on GitHub #3609 (comment).

[MalteJ]

Be careful with duplicate MAC addresses when using Docker on multiple hosts on the same ethernet link.
If I remember correctly the containers' MAC addresses are generated from their IPv4 address!!!
That means the container with IP 172.42.0.2 on host A has the same mac as the container with IP 172.42.0.2 on host B!
You need to assign different IPv4 subnets to the hosts via --fixed-cidr.

[thomasschaeferm]

First attempts with NDP-Proxying did not work. I tried it with /128 and one manual ndp proxy entry.
But now the doc say clear:
The subnet for Docker containers should at least have a size of /80

In respect to this, using the same network and ndp-proxying could only work with an additional help-daemon like ndppd or npd6. Maybe some IPv6 & docker experts have an idea here, to make the the (good) iPv6-solution easier to use for people who don't have access to their routers

[MalteJ]

The /80 subnet is just recommended to have a fixed mapping between MAC and IPv6 address, so you don't run into troubles with cache invalidation when you have short living containers.
You should be able to run docker using smaller subnets, too.
I have not tried NDP proxying yet, so unfortunately I am no big help on this topic.

[thomasschaeferm]

I stopped trying to use ndp-proxy. Maybe I have a understanding problem or it is not possible.
As already mentioned, the routing solution needs access to the router.
After the successful test in LRZ environment it was time to find a solution for home environments.

You need

• an ISP providing more than one /64 - here the Deutsche Telekom provides /56 - ok
• a router with access to configuration of routes or support of DHCP prefix delegation - here avm Fritzbox 7390 - ok

How it works:

• enable dhcp-pd
  fritz.box --> Netzwerk, Netzwerkeinstellungen, IPv6-Adressen, DNS-Server und IPv6-Präfix (IA_PD)zuweisen - ok,
  sorry I don't translate German GUI, it maybe totally different in international versions
• starting dhclient at your docker-host:
  dhclient -v -6 -d -P eth0
  Internet Systems Consortium DHCP Client 4.3.0
  Copyright 2004-2014 Internet Systems Consortium.
  All rights reserved.
  For info, please visit https://www.isc.org/software/dhcp/

Listening on Socket/eth0
Sending on Socket/eth0
PRC: Soliciting for leases (INIT).
XMT: Forming Solicit, 0 ms elapsed.
XMT: X-- IA_PD 2b:1a:c9:9a
XMT: | X-- Request renew in +3600
XMT: | X-- Request rebind in +5400
XMT: Solicit on eth0, interval 1010ms.
RCV: Advertise message on eth0 from fe80::2665:11ff:fe8f:181f.
RCV: X-- Preference 0.
RCV: X-- IA_PD 2b:1a:c9:9a
RCV: | X-- starts 1424425060
RCV: | X-- t1 - renew +1800
RCV: | X-- t2 - rebind +2880
RCV: | X-- [Options]
RCV: | | X-- IAPREFIX 2003:63:2443:b1fc::/62
RCV: | | | X-- Preferred lifetime 3600.
RCV: | | | X-- Max lifetime 7200.
RCV: X-- Server ID: 00:03:00:01:24:65:11:8f:18:1f
RCV: Advertisement recorded.
PRC: Selecting best advertised lease.
PRC: Considering best lease.
PRC: X-- Initial candidate 00:03:00:01:24:65:11:8f:18:1f (s: 154, p: 0).
XMT: Forming Request, 0 ms elapsed.
XMT: X-- IA_PD 2b:1a:c9:9a
XMT: | X-- Requested renew +3600
XMT: | X-- Requested rebind +5400
XMT: | | X-- IAPREFIX 2003:63:2443:b1fc::/62
XMT: | | | X-- Preferred lifetime +7200
XMT: | | | X-- Max lifetime +7500
XMT: V IA_PD appended.
XMT: Request on eth0, interval 950ms.
RCV: Reply message on eth0 from fe80::2665:11ff:fe8f:181f.
RCV: X-- Preference 0.
RCV: X-- IA_PD 2b:1a:c9:9a
RCV: | X-- starts 1424425061
RCV: | X-- t1 - renew +1800
RCV: | X-- t2 - rebind +2880
RCV: | X-- [Options]
RCV: | | X-- IAPREFIX 2003:63:2443:b1fc::/62
RCV: | | | X-- Preferred lifetime 3600.
RCV: | | | X-- Max lifetime 7200.
RCV: X-- Server ID: 00:03:00:01:24:65:11:8f:18:1f
PRC: Bound to lease 00:03:00:01:24:65:11:8f:18:1f.
PRC: Renewal event scheduled in 1799 seconds, to run for 1080 seconds.
PRC: Depreference scheduled in 3599 seconds.
PRC: Expiration scheduled in 7199 seconds.

look at the IAPREFIX, add it to your docker options,

here in opensuse:

cat /etc/sysconfig/docker

Path : System/Management

Description : Extra cli switches for docker daemon

Type : string

Default : ""

ServiceRestart : docker

#DOCKER_OPTS=""
DOCKER_OPTS="-g '/home/docker' --ipv6 --fixed-cidr-v6='2003:63:2443:b1fc::/62'"

start docker-daemon with

systemctl start docker

run docker with

docker run -t -i opensuse /bin/bash

ping6 heise.de

that's it.

PS: The firewall of the Fritzbox is little bit strange. You can't open the whole subnet.
PPS: The firewall of the Fritzbox is much better than others, e.g. speedport, where you can't open the firewall at all.

[MalteJ]

@thomasschaeferm thanks for your work!
I have written some docs on how to use Docker IPv6 with NDP proxying: #10831
Maybe it helps?
If it still does not work, ping me and I will come by and have a look at your university environment 😉

[thomasschaeferm]

I will have a look on it.
At the moment I am in my residential environment. But instead of switching off all computers...I make tests. :-)

Thanks again for integrating IPv6.

[thomasschaeferm]

some hours and some reboots later:

I can confirm that your beautiful manual for "Using NDP proxying" works. (at least one time)

I still don't know why I have some problems, the last time I had to start docker twice before it did work.

hpmini:~ # systemctl status docker
docker.service - Docker
Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled)
Active: failed (Result: exit-code) since Sa 2015-02-21 11:57:19 CET; 41s ago
Process: 2029 ExecStart=/usr/bin/docker -d $DOCKER_OPTS (code=exited, status=1/FAILURE)
Main PID: 2029 (code=exited, status=1/FAILURE)

Feb 21 11:57:18 hpmini docker[2029]: time="2015-02-21T11:57:18+01:00" level="info" msg="+job serveapi(unix:///var/run/docker.sock)"
Feb 21 11:57:18 hpmini docker[2029]: time="2015-02-21T11:57:18+01:00" level="info" msg="Listening for HTTP on unix (/var/run/docker.sock)"
Feb 21 11:57:18 hpmini docker[2029]: time="2015-02-21T11:57:18+01:00" level="info" msg="+job init_networkdriver()"
Feb 21 11:57:19 hpmini docker[2029]: time="2015-02-21T11:57:19+01:00" level="info" msg="Adding route to IPv6 network "2003:63:2443:...cker0""
Feb 21 11:57:19 hpmini docker[2029]: bridge IPv6 does not match existing bridge configuration fe80::1
Feb 21 11:57:19 hpmini docker[2029]: time="2015-02-21T11:57:19+01:00" level="info" msg="-job init_networkdriver() = ERR (1)"
Feb 21 11:57:19 hpmini docker[2029]: time="2015-02-21T11:57:19+01:00" level="fatal" msg="bridge IPv6 does not match existing bridge ...fe80::1"
Hint: Some lines were ellipsized, use -l to show in full.
hpmini:~ # systemctl start docker
hpmini:~ # systemctl status docker
docker.service - Docker
Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled)
Active: active (running) since Sa 2015-02-21 11:58:02 CET; 1s ago
Main PID: 2143 (docker)
CGroup: /system.slice/docker.service
├─2143 /usr/bin/docker -d -g /home/docker --ipv6 --fixed-cidr-v6='2003:63:2443:b100::c008/125'
└─2197 /sbin/apparmor_parser -r -W docker

Feb 21 11:58:02 hpmini docker[2143]: time="2015-02-21T11:58:02+01:00" level="info" msg="+job serveapi(unix:///var/run/docker.sock)"
Feb 21 11:58:02 hpmini docker[2143]: time="2015-02-21T11:58:02+01:00" level="info" msg="Listening for HTTP on unix (/var/run/docker.sock)"
Feb 21 11:58:02 hpmini docker[2143]: time="2015-02-21T11:58:02+01:00" level="info" msg="+job init_networkdriver()"
Feb 21 11:58:02 hpmini docker[2143]: time="2015-02-21T11:58:02+01:00" level="info" msg="-job init_networkdriver() = OK (0)"
Feb 21 11:58:02 hpmini docker[2143]: time="2015-02-21T11:58:02+01:00" level="info" msg="WARNING: Your kernel does not support cgroup... limit."
Hint: Some lines were ellipsized, use -l to show in full.
hpmini:~ #

Thomas

[MalteJ]

when changing IPv6 settings you always have to check if those settings are actually configured on the bridge. If there is an existing bridge Docker will not enforce some changes.
It's best to stop docker, delete the bridge, start docker again and let it create a new bridge:

service docker stop
ifconfig docker0 down
brctl delbr docker0
service docker0 start

This way the IPv6 address and the routes are added to the bridge / route tables.

[thomasschaeferm]

The Problem is the first start fails( there was no bridge at this time). But now is week end. In principle it works.