-
Notifications
You must be signed in to change notification settings - Fork 116
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DNS fails to resolve for private registry when connected to VPN #924
Comments
My guess is that this is a duplicate of #540 |
Thanks for the report. Could you try again and upload a diagnostics report with beta 30 (released ~2 hrs ago). I can't promise it's fixed but the DNS code has been improved:
|
@ShannonHickey this does appear to be a dupe of #540 -- I went through open issues looking to see if it was already reported, but somehow missed it. @djs55 w00! beta 30 does, indeed fix this problem. I did a factory reset, tried to start up my docker container, it failed (because I wasn't connected to VPN), then connected and tried to start, and it successfully pulled. This is great, thank you! I'll continue testing/using it throughout today and will report in this thread if I find any issues, but we can consider it preliminarily solved. |
After doing some additional testing, I'm finding that DNS resolution in the Moby VM and in a running docker container are not in sync and not working as expected: in VM
in running container
in macOSall resolve as expected. Also, when just trying to pull a container from the public index:
Since there are 2 sets of DNS servers (one for the (W)LAN that I'm on and one for the VPN, I checked how DNS resolves directly against those servers. The public domains above (google and ubuntu) both resolve against the LAN DNS servers, but not against the VPN DNS servers. The mycompany.com DNS all resolves only against the VPN DNS servers (as expected). It is weird that that the apt domain fails to resolve in the container where it resolves fine in the VM, but the other mycompany.com entries resolve fine in both. let me know if there's any other information I can forward. Diagnostics for Beta30:
|
@spikegrobstein we added lots of improvements on the |
@rogaha no you can't do that, there is no way to update Docker for Mac using |
ok cool. Thanks for the heads up @justincormack. Better to wait for the |
I'll await the next beta, then. |
Is the change in |
I'm running 1.13.0-rc4-beta34.1 (14853), and I still have the failing DNS lookups as lookups are using 192.168.65.1. |
I'm also still running into this issue in beta34.1 internal DNS resolves in the container, while external DNS (ie: google.com and archive.ubuntu.com) fail. diagnose output:
|
It used to work fine for me before the holidays, but now I see similar problem in:
I have a local DNS server running on my host.
I can pull from public registries but it won't resolve addresses in a non-public DNS.
But querying for the registry domain locally gives:
|
I just started having this issue after upgrading from Docker for Mac 1.12.6 to 1.13.0.
Diagnostic Output
|
Same here Worked fine on Mac 1.12.latest. Upgraded to 1.13.0 and now cannot resolve any corporate VPN addresses including internal docker registry. The public DNS names work ok.
The only workaround I found so far is to explicitly give corporate DNS server address:
I cannot use this workaround for docker pull or docker-compose. Docker for Mac: version: 1.13.0 (0c6d765c5) |
Found a solution. In Docker for Mac, go to "Preferences -- Uninstall/Reset --- Factory Reset". Do that, moby gets whacked and recreated. Problem solved for me. |
Sweet! Worked for me as well. |
Actually, spoke too soon. |
I'm still having the issue in |
I tried the Factory Reset, but it did not work for me. Still getting an I ended up doing a reinstall of version 1.12.1 (build: 12133) |
I went back to 1.12.6 to get it working again. |
Where did you get this version from? I can't find it anywhere anymore. I mean the whole Docker for Mac package. |
Have you guys tried moving to the stable 1.13? Thats what i'm on.. working well now. |
Upgrading to beta39 resolved my VPN issues! At least right now everything seems to be working again. I'll update if it stays this way. |
@spikegrobstein if you are still having this issue, could you please send us another diagnostic so we can look in to it? If not, please let us know it's fixed so we can close this! Thanks. |
I just noticed this happening on 1.13/.1 last night. Palo Alto Global Connect VPN.
Get Outlook for iOS<https://aka.ms/o0ukef>
On Mon, Feb 13, 2017 at 7:31 AM -0600, "Dave Tucker" <notifications@github.com<mailto:notifications@github.com>> wrote:
@spikegrobstein<https://github.com/spikegrobstein> if you are still having this issue, could you please send us another diagnostic so we can look in to it? If not, please let us know it's fixed so we can close this! Thanks.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<#924 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AANA7mMf4kw4IW2icv37s_ymLc8B2Kvbks5rcFssgaJpZM4KuCGf>.
|
Yes, still having issues:
Diagnostic ID: 1.13.1-beta42 is unable to resolve any dns in-container while connected to the VPN, while the moby VM is able to resolve only internal DNS (eg: not google.com or archive.ubuntu.com, but does resolve docker.mycompany.com). |
running on container can resolve public and internal domain names but connection to any docker registry fails. diagnostic upload failed as well. |
This was working on 1.13.0, but I recently updated to 1.13.1 and I'm no longer able to resolve our private docker registry. |
Was working great on 1.13.0 and when I updated to 1.13.1 I was unable to connect to my internal private registry via vpn. Like a previous commenter, I am also using the Palo Alto Globalprotect VPN Client. DNS for the registry resolves fine on the OSX side, but fails to resolve inside the container with the following error: |
Docker.. ya'll need to solve this one quickly. Folks not being able to access registries over their VPNs is a huge Blocker for adoption.. like code red, level |
Fortunately it's not super critical for me because no production machines are running Docker for Mac, however Development machines are affected if they were updated. A downgrade path from 1.13.1 -> 1.13.0 would be useful... |
@spikegrobstein thank you for your latest diagnostics, it has appeared on our servers and so I have escalated this to an internal ticket. @soupmatt & @sergeipogrebnyak your diagnostics are also present, not clear if you have the same issue as @spikegrobstein or not but I have referenced your diagnostics in the internal ticket. Everyone else, please use 🐳 menu ➡️ "Diagnose & Feedback" to upload a diagnostic and create a fresh issue describing what does and does not work for you, without that we will be unable to address your specific failure mode. |
The latest
If anyone gets a chance to try this, let me know how it goes. If it goes well I'd like to make it the default behaviour in a future release. Thanks! |
I run in same issue. Is it possible, that docker servers banned me? In past I have some troubles with PSN(Playstation Network) because their provider blacklisted my dynamic IP-address UPD: just restart my wifi router, now its ok UPD 2: nope, it worked about 5 minutes |
@djs55 It works! Before the change:
After the change:
|
@djs55 It broke again :( I just updated to Any idea on when are we going to have this working reliably? |
@mgilbir sorry to hear it stopped working. The method to switch to the native host resolver changed a little in recent edge versions. In
The new runes to turn it on or off are:
Could you let me know if it works or not with the most recent edge version? If it still doesn't work, could you upload a fresh diagnostics? Sorry for the inconvenience. |
Using:
I can get nothing to work. The problem seems to be related to Alpine 3.5 name resolution, but if I understand correctly that is what Docker is using itself. If you look below, it seems that even if it resolves to an IPv4 address, it fails anyway. Looking at the DNS order, it favors the IPv6 DNS address and that returns the correct data if I use dig on the host.
I really need my development environment running but don't know how to go back to a working version or how to patch this to work! |
@BrendonW could you upload a diagnostic report after the problem manifests? The report will contain more detailed diagnostics including a DNS packet trace. There are some unreleased DNS fixes -- if you'd like to try testing them, follow the instructions in this comment: #1569 (comment) Failing that, try this to revert to a previous setting:
Let me know if that helps (or not). Please upload another diagnostic if not and I'll take a look. |
This issue has been inactive for more than 14 days while marked as MORE_INFO_EXPIRY_TIMEOUT |
Closed issues are locked after 30 days of inactivity. If you have found a problem that seems similar to this, please open a new issue. Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. |
Expected behavior
docker pull
should successfully pull down images from private registry when connected to VPNActual behavior
Errors with:
Information
Diagnose & Feedback:
Additional notes
We're using an SSL vpn, which is configured through the Network System Preference. When connected, it doesn't update the macOS resolv.conf, so things like the
host
command on the macOS side do not resolve the DNS, either, however runningcurl
orping
against the hostname does work from the macOS side.Using the
scutil --dns
command, I get a section that looks like the following (slightly censored):If I connect to the Docker VM (
screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty
), and add the above DNS servers to/etc/resolv.conf
or hard-code the IP of our registry into/etc/hosts
, operations succeed as expected.I've been digging in and haven't figured out exactly how the VM is resolving DNS; it's pointing to a
192.168.65.1
DNS server, but I'm not sure where that lives or how it's configured. Ideally if that would use the DNS server from VPN, I believe this would all work.Steps to reproduce the behavior
The text was updated successfully, but these errors were encountered: