insecure registry with http-only

[vrothberg]

          Hi! I came across this issue while trying to debug an issue with an insecure registry behind a proxy. I noticed that, even with the workaround in place, `podman pull` will still try sending an HTTPS request to the server:

From my registries.conf:

[[registry]]
prefix = "localhost"
location = "localhost"
insecure = true

We can see that trying to pull will still try an HTTPS request first, which will fail:

# podman --log-level=debug pull localhost:5000/myimage:latest 2>&1 | grep https
time="2023-08-10T23:22:47-04:00" level=debug msg="GET https://localhost:5000/v2/"
time="2023-08-10T23:22:47-04:00" level=debug msg="Ping https://localhost:5000/v2/ err Get \"https://localhost:5000/v2/\": http: server gave HTTP response to HTTPS client (&url.Error{Op:\"Get\", URL:\"https://localhost:5000/v2/\", Err:(*errors.errorString)(0xc000352c40)})"

Is there a way to truly get Podman to only try pulling via HTTP? I ask this for a few reasons:

1. Some reverse proxies may not do a good job of handling an HTTPS request to an HTTP port. For example, in my testing with the Varnish cache, the connection ends up hanging for ~5 seconds.
2. This behavior wastes an entire round trip to the server, only to have the request fail and have to be retried as HTTP.
3. As a user, I would expect a way to only send an HTTP-based request.

I guess it isn't technically unexpected behavior, since the registries.conf docs say that both HTP and untrusted TLS connections will be allowed:

# # If true, unencrypted HTTP as well as TLS connections with untrusted
# # certificates are allowed.
# insecure = false

Given the above, I'm not sure if this belongs on this issue, or should be a separate issue or feature request (e.g., implement a way to force HTTP).

Thanks!

Originally posted by @acritelli in containers/podman#16486 (comment)

[vrothberg]

@mtrmac WDYT?

I think the proxy misbehaves.

[acritelli]

@mtrmac WDYT?

I think the proxy misbehaves.

I don't think the proxy necessarily misbehaves here. From its perspective, it never receives any HTTP traffic. Since it only implements HTTP, I don't think it's under any obligation to immediately terminate the connection upon receiving (what it sees as) malformed HTTP traffic.

I imagine we could see similar behavior by firewalls that will silently ignore non-HTTP traffic. Therefore, I think there is still a value in having a flag that forces HTTP.

I am far from a Go expert, but I've written a bit and would be willing to assist with this implementation if someone can point me in the right direction.

[mtrmac]

As you noted, the option works as designed. I could see an argument that that design is unfortunate, but it’s no longer practical to change the semantics of the option.

Adding a new option to registries.conf to never try HTTPS would be possible; every new option adds complexity for all other users, but, shrug on balance I guess I wouldn’t mind that being added. I think the blocked flag in registries.conf is a reasonably close template to follow, showing which subpackages need modifying, and how the option handling works.

---

OTOH, I also agree that fixing the proxy to recognize TLS frames when HTTP verbs are expected (like Go, in the quoted example, recognizes HTTP responses when TLS responses are expected) would probably be more generally useful than enhancing c/image — there are far fewer HTTP proxy implementations than HTTP servers and clients, so fixing the proxy would fix more of the universe.

So if you are looking for a place to contribute an improvement for this end-to-end situation, I think the proxy is a place with more impact.