Skip to content

Commit

Permalink
Update devref to catchup with libnetwork 0.5.0
Browse files Browse the repository at this point in the history 
Updating devref to address following changes:
1. Removal of 'service' from docker
2. Vendoring in IPAM driver support from libnetwork to docker. This
allows user to pass subnet of their choice using docker cli. In case
if user does not provide subnet related configuration and neither
remote ipam driver while creating network, built-in ipam driver will
provide default subnet configuration.

refer to the links for details:
moby/moby#16645
https://github.com/docker/libnetwork/blob/master/docs/ipam.md

Change-Id: I30f5cb54d4feadafc954f6e4b09bcbd6243e6c00
Closes-Bug: #1505489
  • Loading branch information
@vikaschoudhary16
vikaschoudhary16 committed Oct 22, 2015
1 parent 16250e9 commit d510d33
Showing 1 changed file with 133 additions and 103 deletions.
236 changes: 133 additions & 103 deletions doc/source/devref/libnetwork_remote_driver_design.rst
View file
@@ -1,12 +1,12 @@
===============================
Libnetwork Remote Driver Design
================================
=======================================
Libnetwork Remote Network Driver Design
=======================================


What is Kuryr
--------------

Kuryr implements a `libnetwork remote driver`_ and maps its calls to OpenStack
Kuryr implements a `libnetwork remote network driver`_ and maps its calls to OpenStack
`Neutron`_. It works as a translator between libnetwork's
`Container Network Model`_ (CNM) and `Neutron's networking model`_.

Expand All @@ -26,7 +26,7 @@ the host, e.g., Linux bridge, Open vSwitch datapath and so on.
Kuryr Workflow - Host Networking
---------------------------------
Kuryr resides in each host that runs Docker containers and serves `APIs`_
required for the libnetwork remote driver.
required for the libnetwork remote network driver.
It is planned to use the `Adding tags to resources`_ new Neutron feature by Kuryr,
to map between Neutron resource Id's and Docker Id's (UUID's)

Expand All @@ -37,7 +37,7 @@ to map between Neutron resource Id's and Docker Id's (UUID's)

2. libnetwork registers Kuryr as a remote driver

3. A user makes requests against libnetwork with the driver specifier for Kuryr
3. A user makes requests against libnetwork with the network driver specifier for Kuryr

- i.e., ``--driver=kuryr`` or ``-d kuryr`` for the Docker CLI

Expand All @@ -56,121 +56,98 @@ to map between Neutron resource Id's and Docker Id's (UUID's)
- the key/value datastore is abstracted by `libkv`_


Libnetwork User Workflow (with Kuryr as remove driver) - Host Networking
-------------------------------------------------------------------------
Libnetwork User Workflow (with Kuryr as remote network driver) - Host Networking
---------------------------------------------------------------------------------
1. A user creates a network ``foo``
::

$ sudo docker network create --driver=kuryr foo
51c75a2515d47edecc3f720bb541e287224416fb66715eb7802011d6ffd499f1
286eddb51ebca09339cb17aaec05e48ffe60659ced6f3fc41b020b0eb506d364

This makes a HTTP POST call on ``/NetworkDriver.CreateNetwork`` with the
following JSON data.
::

{
"NetworkID": "51c75a2515d47edecc3f720bb541e287224416fb66715eb7802011d6ffd499f1",
"Options": {
...
}
"NetworkID": "286eddb51ebca09339cb17aaec05e48ffe60659ced6f3fc41b020b0eb506d364",
"IPv4Data": [{
"Pool": "172.18.0.0/16",
"Gateway": "172.18.0.1/16",
"AddressSpace": ""
}],
"IPv6Data": [],
"Options": { "com.docker.network.generic": {}}
}

The Kuryr remote driver will then generate a Neutron API request to create an underlying Neutron network.
When the Neutron network has been created, the Kuryr remote driver will generate an empty success response
The Kuryr remote network driver will then generate a Neutron API request to create an underlying Neutron network.
When the Neutron network has been created, the Kuryr remote network driver will generate an empty success response
to the docker daemon.
Kuryr tags the Neutron network with the NetworkID from docker.

2. A user creates a service ``bar`` against network ``foo``
2. A user launches a container against network ``foo``
::

$ sudo docker service publish bar.foo
98953db3f8e6628caf4a7cad3c866cb090654e3dee3e37206ad8c0a81355f1b7
$ sudo docker run --net=foo -itd --name=container1 busybox
78c0458ba00f836f609113dd369b5769527f55bb62b5680d03aa1329eb416703

This makes a HTTP POST call on ``/NetworkDriver.CreateEndpoint`` with the
following JSON data.
following JSON format.
::

{
"NetworkID": "51c75a2515d47edecc3f720bb541e287224416fb66715eb7802011d6ffd499f1",
"EndpointID": "98953db3f8e6628caf4a7cad3c866cb090654e3dee3e37206ad8c0a81355f1b7",
"Interfaces": [
...
],
"Options": {
...
}
}
{
"NetworkID": "286eddb51ebca09339cb17aaec05e48ffe60659ced6f3fc41b020b0eb506d364",
"Interface": {
"AddressIPv6": "",
"MacAddress": "",
"Address": "172.18.0.2/16"
},
"Options": {
"com.docker.network.endpoint.exposedports": [],
"com.docker.network.portmap": []
},
"EndpointID": "edb23d36d77336d780fe25cdb5cf0411e5edd91b0777982b4b28ad125e28a4dd"
}

The Kuryr remote driver then generate a Neutron API request to create a Neutron
subnet and a port with the matching fields for all interfaces in the request.
The Kuryr remote network driver then generate a Neutron API request to create a Neutron
subnet and a port with the matching fields for interface in the request.
Kuryr needs to create the subnet dynamically as it has no information on
the interfaces list IP's.
the interface IP.

the following steps are taken:
Following steps are taken:

1) On the endpoint creation Kuryr examine if there's a subnet which CIDR corresponds to
Address or AddressIPv6 requested.
2) If there's a subnet, Kuryr tries to reuse it without creating a new subnet.
otherwise it create a new one with the given CIDR
3) If a CIDR is not passed, Kuryr creates a default IPv4 or IPv6 subnets from a
specific subnet pool.
more information can be found in Kuryr `IPAM blueprint`_
4) Kuryr creates a port assigning the IP address to it and associating the port with
the subnet based on it's already allocated in 2.
3) Kuryr creates a port assigning the IP address to it and associating the port with
the subnet based on what it has already allocated in 2.
4) Kuryr tags the Neutron subnet and port with EndpointID.

On the subnet creation described in (2) and (3) above, Kuryr tries to grab
the allocation pool greedily by not specifying ``allocation_pool``. Without
``allocation_pool``, Neutron allocates all IP addresses in the range of the
subnet CIDR as described in `Neutron's API reference`_.

When the Neutron port has been created, the Kuryr remote driver will generate a response to the
docker daemon indicating the port's IPv4, IPv6, and Mac addresses as follows.
When the Neutron port has been created, the Kuryr remote driver will generate an empty response to the
docker daemon indicating the SUCCESS. {}
(https://github.com/docker/libnetwork/blob/master/docs/remote.md#create-endpoint)
::

{
"Interfaces": [{
"ID": <port-id>,
"Address": <port-fixed-IP-address>,
"AddressIPv6": <port-fixed-IPv6-address>,
"MacAddress": <port-mac-addr>
}, ...]
"Interface": {"MacAddress": "08:22:e0:a8:7d:db"}
}

Kuryr tags the Neutron subnet and port with Docker Interface id.

3. A user shows information of the service
::

$ sudo docker service info test.bar
Service Id: db0524fa27184de3dfe274908e77e05155e12e20269c782984468b251fe507d7
Name: bar
Network: foo

4. A user attaches a container to the service
On receiving success response, libnetwork makes a HTTP POST call on ``/NetworkDriver.Join`` with
the following JOSN data.
::

$ CID=$(sudo docker run -itd busybox)
$ sudo docker service attach $CID bar.foo

or if a network interface needs to be attached to the container before its
launch,
::

$ sudo docker run --publish-service=bar.foo -itd busybox
12bbda391ed0728787b2c2131f091f6d5744806b538b9314c15e789e5a1ba047

This makes a HTTP POST call on ``/NetworkDriver.Join`` with the following
JOSN data.
::

{
"NetworkID": "51c75a2515d47edecc3f720bb541e287224416fb66715eb7802011d6ffd499f1",
"EndpointID": "98953db3f8e6628caf4a7cad3c866cb090654e3dee3e37206ad8c0a81355f1b7",
"SandboxKey": "/var/run/docker/netns/12bbda391ed0",
"Options": {
...
}
}
{
"NetworkID": "286eddb51ebca09339cb17aaec05e48ffe60659ced6f3fc41b020b0eb506d364",
"SandboxKey": "/var/run/docker/netns/052b9aa6e9cd",
"Options": null,
"EndpointID": "edb23d36d77336d780fe25cdb5cf0411e5edd91b0777982b4b28ad125e28a4dd"
}

Kuryr connects the container to the corresponding neutron network by doing the following steps:

Expand All @@ -180,42 +157,99 @@ Libnetwork User Workflow (with Kuryr as remove driver) - Host Networking
3) Perform a neutron-port-type-dependent VIF-binding to the corresponding Neutron port
using the VIF binding layer and depending on the specific port type.

After the VIF-binding is completed, the Neutron remote driver generate a response to the Docker
After the VIF-binding is completed, the Kuryr remote network driver generates a response to the Docker
daemon as specified in the libnetwork documentation for a join request.
(https://github.com/docker/libnetwork/blob/master/docs/remote.md#join)

5. A user detaches the container from the service
3. A user requests information about the network
::

$ sudo docker service detach $CID bar.foo
$ sudo docker network inspect foo
{
"name": "foo",
"id": "286eddb51ebca09339cb17aaec05e48ffe60659ced6f3fc41b020b0eb506d364",
"scope": "local",
"driver": "kuryr",
"ipam": {
"driver": "default",
"config": [
{}
]
},
"containers": {
"78c0458ba00f836f609113dd369b5769527f55bb62b5680d03aa1329eb416703": {
"endpoint": "edb23d36d77336d780fe25cdb5cf0411e5edd91b0777982b4b28ad125e28a4dd",
"mac_address": "02:42:c0:a8:7b:cb",
"ipv4_address": "172.18.0.2/24",
"ipv6_address": ""
}
}
}


4. A user connects one more container to the network
::

$ sudo docker network connect foo container2
d7fcc280916a8b771d2375688b700b036519d92ba2989622627e641bdde6e646

$ sudo docker network inspect foo
{
"name": "foo",
"id": "286eddb51ebca09339cb17aaec05e48ffe60659ced6f3fc41b020b0eb506d364",
"scope": "local",
"driver": "kuryr",
"ipam": {
"driver": "default",
"config": [
{}
]
},
"containers": {
"78c0458ba00f836f609113dd369b5769527f55bb62b5680d03aa1329eb416703": {
"endpoint": "edb23d36d77336d780fe25cdb5cf0411e5edd91b0777982b4b28ad125e28a4dd",
"mac_address": "02:42:c0:a8:7b:cb",
"ipv4_address": "172.18.0.2/24",
"ipv6_address": ""
},
"d7fcc280916a8b771d2375688b700b036519d92ba2989622627e641bdde6e646": {
"endpoint": "a55976bafaad19f2d455c4516fd3450d3c52d9996a98beb4696dc435a63417fc",
"mac_address": "02:42:c0:a8:7b:cc",
"ipv4_address": "172.18.0.3/24",
"ipv6_address": ""
}
}
}


5. A user disconnects a container from the network
::

$ CID=d7fcc280916a8b771d2375688b700b036519d92ba2989622627e641bdde6e646
$ sudo docker network disconnet foo $CID

This makes a HTTP POST call on ``/NetworkDriver.Leave`` with the following
JSON data.
::

{
"NetworkID": "51c75a2515d47edecc3f720bb541e287224416fb66715eb7802011d6ffd499f1",
"EndpointID": "98953db3f8e6628caf4a7cad3c866cb090654e3dee3e37206ad8c0a81355f1b7"
"NetworkID": "286eddb51ebca09339cb17aaec05e48ffe60659ced6f3fc41b020b0eb506d364",
"EndpointID": "a55976bafaad19f2d455c4516fd3450d3c52d9996a98beb4696dc435a63417fc"
}

Kuryr remote driver will remove the VIF binding between the container and the Neutron port,
Kuryr remote network driver will remove the VIF binding between the container and the Neutron port,
and generate an empty response to the Docker daemon.

6. A user unpublishes the service
::

$ sudo docker unpublish bar.foo

This makes a HTTP POST call on ``/NetworkDriver.DeleteEndpoint`` with the
Then libnetwork makes a HTTP POST call on ``/NetworkDriver.DeleteEndpoint`` with the
following JSON data.
::

{
"NetworkID": "51c75a2515d47edecc3f720bb541e287224416fb66715eb7802011d6ffd499f1",
"EndpointID": "98953db3f8e6628caf4a7cad3c866cb090654e3dee3e37206ad8c0a81355f1b7"
"NetworkID": "286eddb51ebca09339cb17aaec05e48ffe60659ced6f3fc41b020b0eb506d364",
"EndpointID": "a55976bafaad19f2d455c4516fd3450d3c52d9996a98beb4696dc435a63417fc"
}

Kuryr remote driver generate a Neutron API request to delete the associated Neutron port,
Kuryr remote network driver generates a Neutron API request to delete the associated Neutron port,
in case the relevant port subnet is empty, Kuryr also deletes the subnet object using Neutron API
and generate an empty response to the Docker daemon: {}

Expand All @@ -229,18 +263,13 @@ Libnetwork User Workflow (with Kuryr as remove driver) - Host Networking
::

{
"NetworkID": "51c75a2515d47edecc3f720bb541e287224416fb66715eb7802011d6ffd499f1"
"NetworkID": "286eddb51ebca09339cb17aaec05e48ffe60659ced6f3fc41b020b0eb506d364"
}

Kuryr remote driver generate a Neutron API request to delete the corresponding Neutron network.
When the Neutron network has been deleted, the Kuryr remote driver generate an empty response
Kuryr remote network driver generates a Neutron API request to delete the corresponding Neutron network.
When the Neutron network has been deleted, the Kuryr remote network driver generate an empty response
to the docker daemon: {}

The workflows described in 2., 4., 5. and 6. can be done in the following
single command.::

$ sudo docker run --publish-service=cont.implicit.foo -itd busybox


Mapping between the CNM and the Neutron's Networking Model
------------------------------------------------------------
Expand All @@ -254,7 +283,7 @@ libnetwork Neutron
===================== ======================
Network Network
Sandbox Subnet, Port and netns
Endpoint Subnet, Port
Endpoint Port
===================== ======================

libnetwork's Sandbox and Endpoint can be mapped into Neutron's Subnet and Port,
Expand All @@ -263,7 +292,8 @@ visible and editable resource entity attachable to containers from users'
perspective. Sandbox manages information exposed by Endpoint behind the scene
automatically.

.. _libnetwork remote driver: https://github.com/docker/libnetwork/blob/master/docs/remote.md
.. _libnetwork remote network driver: https://github.com/docker/libnetwork/blob/master/docs/remote.md
.. _libnetwork IPAM driver: https://github.com/docker/libnetwork/blob/master/docs/ipam.md
.. _Neutron: https://wiki.openstack.org/wiki/Neutron
.. _Container Network Model: https://github.com/docker/libnetwork/blob/master/docs/design.md#the-container-network-model
.. _Neutron's networking model: https://wiki.openstack.org/wiki/Neutron/APIv2-specification
Expand Down

0 comments on commit d510d33

Please sign in to comment.