How to leverage VFP NAT rules for outbound connectivity for overlay networks created by Docker Swarm in Windows server 2022

[dilip3710]

Description

Description
We are setting up the Docker Swarm Orchestration in the Windows server 2022 (Version 21H2 and OS Build:20348.2340) with MS SQL Server (2022) and Docker Engine version (25.0.3).
We are using .NetCore 8.0, RabbitMQ (For Message Queuing) and docker-compose files.
Each application is deployed as a Microservice.

Issue Description:
We initialized the Swarm (Single Manager node) with VM IP address and it run in Windows server 2022.
We deployed the services using command - docker stack deploy docker-compose.yml <app_name>.We can see the docker service got created with replicated status (1/1) and all the containers are launched successfully (in swarm overlay network)
However we cannot see any outbound connectivity/external traffic from docker swarm containers. It is unable to communicate even to localhost,1433 SQL port or any other ports.
We have tested using Test-NetConnection -ComputerName -Port 1433 command by running the swarm container.
Even swarm containers are showing UP status , It is not working as expected and seeing failures on SQL connectivity and others .(from docker logs <container_name).
We also found the swarm containers are not using WinNAT/NAT network for External connectivity and each container having it’s own vEthernet Adapter and each container having it’s own vEthernet container Adapter Note: Hyper-V is not enabled in Windows server 2022
Note: Hyper-V is not enabled in Windows server 2022

From MS link, Windows Network Containers we can see that Swarm containers use VFP (Virtual Filtering Platform) NAT Rules for Outbound connectivity. We are not successful on setting that VFP NAT rules yet.

Please help us to resolve this issue

Steps we tried:
Added required firewall connectivity for swarm setup with TCP and UDP ports opened.
Created overlay network for swarm services and use that network for deploying docker swarm.
Added required Firewall for SQL Server- port 1433 and also RabbitMQ.
Non Swarm Container are running fine in same windows 2022 server and it is using NAT network for outbound connectivity

Note: We also have same setup running successfully in Windows server 2016 with SQL server 2016 and Docker Version 20.10.9. In that 2016 server , we can see the swarm containers are using ‘WinNAT’ network/vEthernet NIC adapter for the Outbound connectivity but same step is missing in the 2022 server.

Steps we tried:

Restarted Docker services
Added required firewall connectivity for swarm setup with TCP and UDP ports opened.
Created overlay network for swarm services and use that network for deploying docker swarm.
Added required Firewall for SQL Server- port 1433 and also RabbitMQ.
Non Swarm Windows Container are running fine in same windows 2022 server and it is using NAT network for outbound connectivity
Tried Writing New-NetNat and netsh interface/routing commands to open the external connectivity but it didn't work out.

Reproduce

Install latest docker engine (25.0.3 version) in Windows server 2022
Docker swarm init --advertise-addr <VM_IP>
Docker stack deploy docker-compose.yml <app_name>
Docker service ls
Docker Container ls
docker exec -it Powershell

Test-NetConnection -ComputerName <VM_IP> -Port 1433 OR Ping www.google.com
Above command is failing with Timeout error

Expected behavior

Docker Container ls
docker exec -it Powershell

Test-NetConnection -ComputerName <VM_IP> -Port 1433 OR Ping www.google.com
Above TCP Ping command should succeed

docker version

Client:
 Version:           25.0.3
 API version:       1.44
 Go version:        go1.21.6
 Git commit:        4debf41
 Built:             Fri Feb 23 02:40:51 2024
 OS/Arch:           windows/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          25.0.3
  API version:      1.44 (minimum version 1.24)
  Go version:       go1.21.6
  Git commit:       f417435
  Built:            Fri Feb 23 02:39:35 2024
  OS/Arch:          windows/amd64
  Experimental:     false

docker info

Client:
 Version:    25.0.3
 Context:    default
 Debug Mode: false
 Plugins:
  compose: Docker Compose (Docker Inc.)
    Version:  v2.2.3
    Path:     C:\Program Files\Docker\cli-plugins\docker-compose.exe

Server:
 Containers: 16
  Running: 12
  Paused: 0
  Stopped: 4
 Images: 14
 Server Version: 25.0.3
 Storage Driver: windowsfilter
  Windows:
 Logging Driver: json-file
 Plugins:
  Volume: local
  Network: ics internal l2bridge l2tunnel nat null overlay private transparent
  Log: awslogs etwlogs fluentd gcplogs gelf json-file local splunk syslog
 Swarm: active
  NodeID: xxxxxxxxxxxxxxxxxxxxxxxx
  Is Manager: true
  ClusterID: xxxxxxxxxxxxxxxxxxxx
  Managers: 1
  Nodes: 1
  Default Address Pool: x.x.x.x/8
  SubnetSize: 24
  Data Path Port: 4789
  Orchestration:
   Task History Retention Limit: 5
  Raft:
   Snapshot Interval: 10000
   Number of Old Snapshots to Retain: 0
   Heartbeat Tick: 1
   Election Tick: 10
  Dispatcher:
   Heartbeat Period: About a minute
  CA Configuration:
   Expiry Duration: 3 months
   Force Rotate: 0
  Autolock Managers: false
  Root Rotation In Progress: false
  Node Address: x.x.x.x
  Manager Addresses:
   x.x.x.x:2377
 Default Isolation: process
 Kernel Version: 10.0 20348 (20348.1.amd64fre.fe_release.210507-1500)
 Operating System: Microsoft Windows Server Version 21H2 (OS Build 20348.2340)
 OSType: windows
 Architecture: x86_64
 CPUs: 2
 Total Memory: 16GiB
 Name: xxxxxxxxxxxxxxxxxxx
 ID: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
 Docker Root Dir: C:\ProgramData\docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

Additional Info

No response