Add dual-stack support on flannel #3906

manuelbuil · 2021-08-24T18:16:30Z

Signed-off-by: Manuel Buil mbuil@suse.com

Proposed Changes

Add the flannel part of the dual-stack support for k3s. First version

Types of Changes

New Feature

Verification

Deploy k3s with:

write-kubeconfig-mode: 644
token: "secret"
cluster-cidr: 10.42.0.0/16,2001:cafe:42:0::/56
service-cidr: 10.43.0.0/16,2001:cafe:42:1::/112
disable-network-policy: true

And having your main interface with an ipv6 address. Of course your infra must allow ipv6 traffic!

Linked Issues

#1405

User-Facing Change

Further Comments

codecov-commenter · 2021-08-24T18:45:48Z

Codecov Report

Merging #3906 (9fcd79b) into master (5e0fae9) will decrease coverage by 0.04%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #3906      +/-   ##
==========================================
- Coverage   11.52%   11.47%   -0.05%     
==========================================
  Files         132      132              
  Lines        8928     8965      +37     
==========================================
  Hits         1029     1029              
- Misses       7673     7710      +37     
  Partials      226      226

Flag	Coverage Δ
inttests	`0.69% <0.00%> (-0.01%)`	⬇️
unittests	`11.12% <0.00%> (-0.05%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pkg/agent/flannel/flannel.go	`0.00% <0.00%> (ø)`
pkg/agent/flannel/setup.go	`0.00% <0.00%> (ø)`
pkg/cli/server/server.go	`0.00% <0.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5e0fae9...9fcd79b. Read the comment docs.

pkg/agent/flannel/flannel.go

pkg/agent/flannel/setup.go

scripts/version.sh

vadorovsky

First remark - could you please make clear in the PR name that it's only about flannel? I.e. flannel: Add dual-stack support

The current title gives an impression that it's supposed to solve dual-stack in the whole k3s at once, which is not true (there is work in kube-router to be done).

vadorovsky

~~Just two nits~~ lol I'm an idiot, I was commenting code from vendor/ 🤦

vadorovsky

LGTM, thanks!

pkg/agent/flannel/flannel.go

brandond · 2021-09-07T18:32:27Z

pkg/agent/flannel/setup.go

+		confJSON = strings.ReplaceAll(confJSON, "%DUALSTACK%", "true")
+		for _, cidr := range nodeConfig.AgentConfig.ClusterCIDRs {
+			if utilsnet.IsIPv6(cidr.IP) {
+				// Assuming there is going to be only one ipv6 range


Recent versions of Kubernetes allow you to specify multiple non-overlapping, non-contiguous ipv4/ipv6 ranges. Is it too much work to allow that? We should at least handle it properly.

This is a good point Brad! I wanted to look at this but forgot to do it. I don't think flannel supports multiple ranges but I need to verify this. If it does not, I guess we should fail with an error describing this, right?

After some investigations and questions to sig-network guys, the conclusion is that multiple non-overlapping, non-contiguous ipv4/ipv6 ranges is not supported yet. It's under development kubernetes/enhancements#2593

Interesting, I'll have to dig up the changelog entry I was thinking of. It's possible that one of the cloud providers has added support for this already but the core controller-manager nodeipam does not yet.

I added the link in the comment, so that we don't forget this

Here's what I was thinking of - it appears that this is GKE only and hasn't hit core Kubernetes yet:
https://cloud.google.com/kubernetes-engine/docs/how-to/multi-pod-cidr

pkg/agent/flannel/setup.go

manuelbuil · 2021-09-10T17:05:01Z

I'll add the tests in a different commit but same PR

brandond · 2021-09-14T17:24:37Z

Now that we have tests in, can we pull out the startup check for --flannel-backend=none when using dual-stack? I'm excited to land this!

manuelbuil · 2021-09-14T18:31:51Z

Now that we have tests in, can we pull out the startup check for --flannel-backend=none when using dual-stack? I'm excited to land this!

Wait, we have unit tests. I'm working on integration tests. Hopefully done tomorrow

Signed-off-by: Manuel Buil <mbuil@suse.com>

brandond · 2021-09-15T10:31:56Z

pkg/cli/server/server.go

@@ -467,7 +467,7 @@ func validateNetworkConfiguration(serverConfig server.Config) error {
 		return errors.Wrap(err, "failed to validate cluster-dns")
 	}

-	if (serverConfig.ControlConfig.FlannelBackend != "none" || serverConfig.ControlConfig.DisableNPC == false) && (dualCluster || dualService) {
+	if (serverConfig.ControlConfig.DisableNPC == false) && (dualCluster || dualService) {


Remove the flannel reference from the error message on the next line too.

good catch!

…kend Signed-off-by: Manuel Buil <mbuil@suse.com>

manuelbuil · 2021-09-15T15:06:49Z

@dereknola could you check the tests please?

dereknola

Tests look good

manuelbuil requested a review from a team as a code owner August 24, 2021 18:16

dweomer approved these changes Aug 25, 2021

View reviewed changes

xanmanning mentioned this pull request Sep 4, 2021

Enchancement: Document IPv6 options with the playbook PyratLabs/ansible-role-k3s#147

Closed

manuelbuil force-pushed the dual-stack branch 4 times, most recently from 3cdd054 to b58701f Compare September 7, 2021 15:18

manuelbuil requested a review from dweomer September 7, 2021 16:59

manuelbuil force-pushed the dual-stack branch 2 times, most recently from f0ae37b to 1cdc1b7 Compare September 7, 2021 17:04

manuelbuil changed the title ~~[WIP] Add dual-stack support~~ Add dual-stack support Sep 7, 2021

vadorovsky suggested changes Sep 7, 2021

View reviewed changes

manuelbuil changed the title ~~Add dual-stack support~~ Add dual-stack support on flannel Sep 7, 2021

vadorovsky suggested changes Sep 7, 2021

View reviewed changes

vadorovsky approved these changes Sep 7, 2021

View reviewed changes

brandond requested changes Sep 7, 2021

View reviewed changes

manuelbuil force-pushed the dual-stack branch from 1cdc1b7 to e38a22c Compare September 8, 2021 08:08

brandond reviewed Sep 8, 2021

View reviewed changes

pkg/agent/flannel/setup.go Show resolved Hide resolved

manuelbuil requested a review from brandond September 8, 2021 09:24

manuelbuil force-pushed the dual-stack branch from e38a22c to 0f1b056 Compare September 10, 2021 16:09

brandond approved these changes Sep 10, 2021

View reviewed changes

manuelbuil force-pushed the dual-stack branch from 0f1b056 to a181746 Compare September 10, 2021 16:57

manuelbuil changed the title ~~Add dual-stack support on flannel~~ [WIP] Add dual-stack support on flannel Sep 10, 2021

dweomer approved these changes Sep 10, 2021

View reviewed changes

manuelbuil mentioned this pull request Sep 14, 2021

[dual-stack] Flannel unit tests #3974

Closed

manuelbuil force-pushed the dual-stack branch 2 times, most recently from 72c9f92 to 95ca6cd Compare September 14, 2021 16:29

manuelbuil force-pushed the dual-stack branch 3 times, most recently from 35f6618 to 262b397 Compare September 15, 2021 07:59

manuelbuil changed the title ~~[WIP] Add dual-stack support on flannel~~ Add dual-stack support on flannel Sep 15, 2021

manuelbuil force-pushed the dual-stack branch 2 times, most recently from 957aba6 to 702addf Compare September 15, 2021 09:04

Add dual-stack support

681058b

Signed-off-by: Manuel Buil <mbuil@suse.com>

manuelbuil force-pushed the dual-stack branch from 702addf to 43fe774 Compare September 15, 2021 09:45

brandond requested changes Sep 15, 2021

View reviewed changes

Add tests to the dual-stack PR and enable dual-stack with flannel bac…

9fcd79b

…kend Signed-off-by: Manuel Buil <mbuil@suse.com>

manuelbuil force-pushed the dual-stack branch from 43fe774 to 9fcd79b Compare September 15, 2021 12:12

manuelbuil requested a review from brandond September 15, 2021 12:15

manuelbuil mentioned this pull request Sep 15, 2021

Support for dual-stack Kubernetes cluster #1405

Closed

manuelbuil requested a review from dereknola September 15, 2021 15:07

brandond approved these changes Sep 15, 2021

View reviewed changes

dereknola approved these changes Sep 15, 2021

View reviewed changes

manuelbuil merged commit 60cd86b into k3s-io:master Sep 15, 2021

This was referenced Oct 21, 2021

[Release-1.21] Update to the newest flannel #4267

Merged

[Release-1.20] Update to the newest flannel #4268

Merged

manuelbuil deleted the dual-stack branch October 28, 2021 08:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add dual-stack support on flannel #3906

Add dual-stack support on flannel #3906

manuelbuil commented Aug 24, 2021

codecov-commenter commented Aug 24, 2021 •

edited

vadorovsky left a comment •

edited

vadorovsky left a comment •

edited

vadorovsky left a comment

brandond Sep 7, 2021

manuelbuil Sep 8, 2021

manuelbuil Sep 8, 2021

brandond Sep 8, 2021 •

edited

manuelbuil Sep 8, 2021

brandond Sep 8, 2021

manuelbuil commented Sep 10, 2021

brandond commented Sep 14, 2021 •

edited

manuelbuil commented Sep 14, 2021

brandond Sep 15, 2021 •

edited

manuelbuil Sep 15, 2021

manuelbuil commented Sep 15, 2021

dereknola left a comment

Add dual-stack support on flannel #3906

Add dual-stack support on flannel #3906

Conversation

manuelbuil commented Aug 24, 2021

Proposed Changes

Types of Changes

Verification

Linked Issues

User-Facing Change

Further Comments

codecov-commenter commented Aug 24, 2021 • edited

Codecov Report

vadorovsky left a comment • edited

Choose a reason for hiding this comment

vadorovsky left a comment • edited

Choose a reason for hiding this comment

vadorovsky left a comment

Choose a reason for hiding this comment

brandond Sep 7, 2021

Choose a reason for hiding this comment

manuelbuil Sep 8, 2021

Choose a reason for hiding this comment

manuelbuil Sep 8, 2021

Choose a reason for hiding this comment

brandond Sep 8, 2021 • edited

Choose a reason for hiding this comment

manuelbuil Sep 8, 2021

Choose a reason for hiding this comment

brandond Sep 8, 2021

Choose a reason for hiding this comment

manuelbuil commented Sep 10, 2021

brandond commented Sep 14, 2021 • edited

manuelbuil commented Sep 14, 2021

brandond Sep 15, 2021 • edited

Choose a reason for hiding this comment

manuelbuil Sep 15, 2021

Choose a reason for hiding this comment

manuelbuil commented Sep 15, 2021

dereknola left a comment

Choose a reason for hiding this comment

codecov-commenter commented Aug 24, 2021 •

edited

vadorovsky left a comment •

edited

vadorovsky left a comment •

edited

brandond Sep 8, 2021 •

edited

brandond commented Sep 14, 2021 •

edited

brandond Sep 15, 2021 •

edited