-
Notifications
You must be signed in to change notification settings - Fork 18.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Docker Swarm on Windows 2019 ingress routing not working on some systems #39065
Comments
We're seeing this too. Have tried this with 2019 on vmware and on hyper-v, but no luck. Exact same scenario with standalone containers on nat being accessible from host, however, any service started in Swarm has published ports on ingress without any accessibility from host. Hopefully this will get some attention as we can't find any help anywhere on this topic. |
@drnybble @cmahoski there have been long discussion about similar issues on docker/for-win#1476 but it should works fine on 2019 from both localhost and remote. How ever there was some issues on Win Srv 2019 after it was released #38498 so please install latest Windows Updates and latest Docker version and try again then. |
Just tried again with the 2019-05 cumulative update. Under a VMWare hypervisor:
The problem I am describing is that Windows 2019 running under the KVM hypervisor is not working at all (cannot access remotely either). Looking for next steps to debug/diagnose -- logging etc. |
Exactly same issue here, I'm on WS2019 build 17763, and running docker EE version 18.09.6 |
Having the same issue on WS2019 build 17763.557 and running docker EE version 18.09.6. |
/cc @ddebroy ^^ |
The above issue was occurring in an on-premise environment behind a corporate firewall where I had to install Docker manually using the instructions here. This essentially just has you unpack the zip file into program files, rather than installing by package. I have since created a VM on Azure and tested both installing manually and installing via the package with the following commands:
Installing manually had the same issue as above, however using the package commands I can connect to the container on the host using hostname:port, but not using localhost:port, which is fine using hostname for me anyway. Can anyone tell me why/what the difference between the package installation and manual is? Is there a way to manually configure/install whatever is missing to bridge the gap? This would be a lifesaver! |
So I managed to configure PowerShell to use webproxy server to do a package install and Docker worked after doing this, however I am still having the following issues:
|
I am getting 3 Warnings in Application Events when running swarm init:
|
Tried again with Docker 18.09.7 and my original problem persists. This problem is that I cannot connect to my exposed port on the ingress network from localhost OR remotely. I verified with WireShark that my system receives the incoming connection to port 8000 but a RST is immediately sent indicating that my system is not listening on 8000.
Any updates on this issue? Diagnostics to capture? Also of note: my machine has two NICs. The external vswitch is created by Docker against the private NIC not the public one. Does that matter? |
I got it to work! I disabled the other NICs on this system so that only the NIC with the public IP was enabled. Otherwise it seems to create the external vswitch against an arbitrary NIC, I even saw it create it against the Npcap loopback adapter used by WireShark. So the next question: on a multi-NIC system how to control how the external vswitch is created; and is this documented somewhere? Related: docker/for-win#1399 |
How to customize the ingress network: https://docs.docker.com/network/overlay/#customize-the-default-ingress-network First I did 'docker network inspect ingress' to see the subnet/gateway settings. Then remove it and recreate. Here is an example using the com.docker.network.windowsshim.interface option to specify the interface:
Also the network configuration is used on all nodes so better hope you have the same named NIC everywhere You have to restart the Docker service for this to take effect. |
Same here. I can connect from other hosts no problem, cannot connect from local machine with localhost, 127.0.0.1, host name, any of the network IPs given in docker network inspect, etc. Docker version 19.03.5 docker network create --driver=overlay xxx |
I have similar problem my host machine is running: I tested my compose with the same images( version 17763.1039) on older version of windows (17763.107) today and everything works the way I would expect. I can access my images going to http://hostname:port. |
Windows Server 2019 running Docker/Swarm, ingress network was working fine until this was installed: 2020-05 Cumulative Update for Windows Server 2019 (1809) for x64-based Systems (KB4551853) This broke something with the ingress network such that no traffic could enter through any exposed/published ports. Uninstalling this update made it all work again. |
I believe I might have just come across the same thing on a new swarm setup - only a single port works when multiple are published. I'll try and get a simple repo together when I have time. |
@djarvis Would you happen to know if there's a canonical issue somewhere in the issue tracker about this problem? |
@timparkinson i might have similar issue with yours. #40606 though i think unrelated to the 2020-05 Cumulative Update as I've had the problem for a while :) |
not sure if my issue is related or not #41094 |
Is this issue resolved? My team is seeing a similar issue where we're unable to access a service from the machine it's running on. This is a breaking issue in most cases for using Swarm. |
@FrankAtHexagon unfortunately Microsoft looks to be constantly breaking swatm compatibility Last know good state is Win 2019 with https://support.microsoft.com/en-us/topic/october-20-2020-kb4580390-os-build-17763-1554-preview-ac4799c9-838f-8665-a968-0f19b6cb1049 |
is it resolved? |
The localhost issue is still present in Windows Server 2022.... Running a 3-node swarm cluster but the services cannot communicate with e.g RabbitMQ on their own host, basically making it a 2-node cluster... There's next to no mentions of this issue anywhere except here, did anyone ever find a resolution to this!? |
@bjork-dev I don't know about Windows Server 2022. But Microsoft has really dropped the ball on anything Docker related. any sort of enterprise support now goes straight through Mirantis, so probably with some paid support one can get some better support. |
Description
I create a simple stack to run IIS. It is not reachable through ingress routing on my VM, either via localhost or from a remote machine.
Steps to reproduce the issue:
Deploy the following stack:
Then:
Describe the results you received:
From another machine try to access port 8000 -> fails with unable to connect
Describe the results you expected:
Able to connect to IIS on port 8000.
Additional information you deem important (e.g. issue happens only occasionally):
This same testcase works on two other environments I have tried:
This VM runs under KVM. Whether that is the reason I am not sure.
Also, if I just run the IIS container not in the Swarm so it uses the NAT network it works:
Thus, it does not appear to be firewall related (and firewall is disabled on this box).
Looking for next steps or diagnostics to understand what is going wrong.
Also -- is it a documented limitation on Windows that ingress routing is not accessible via localhost on a Swarm node? Means I cannot run a Docker registry in the Swarm and access it via localhost on Swarm nodes -- it works on Linux.
Output of
docker version
:Output of
docker info
:Additional environment details (AWS, VirtualBox, physical, etc.):
The text was updated successfully, but these errors were encountered: