Initial start on HTTP tunnel proxies

[sethmlarson]

Closes #201. This is my initial stab at implement HTTP tunnel proxy as a Dispatcher. Will need to implement the proxies argument for the Client to match Requests.

I'm wondering what the best way to test this implementation would be.

[sethmlarson]

This is to not replace the request property on a ProxyError which is different than the original request.

[florimondmanca]

I'd be OK with freezing that bit of information as a code comment. :-)

[tomchristie]

Crackin' stuff!
Not a detailed review, but some initial thoughts:

• start_tls doesn't appear to be getting called into anywhere yet - where will that end up being plugged in?
• It's not clear to me on first pass if it makes more sense to have the make-the-connection implementation take effect on the Dispatcher class, or if it'd actually be cleaner to have the "Make a CONNECT request" be implemented within an alternate HTTPConnection.
• If possible we ought to prefer composing implementations rather than inheriting them, since we're being a bit fuzzy about reaching into details of the subclass that we might prefer to be able to treat as private API. (Ie. in this cases that'd mean having a dispatcher class which contains an instance of ConnectionPool rather than inheriting from it.)

It'll probably be easier to fully review once start_tls, and the actually proxy configuration via-the-client stuff is all plumbed in.

[tomchristie]

What's the proposed API for this once it's linked up to the Client?

[sethmlarson]

After I implement the two separate proxy types (forward and tunnel) the interface for clients will be the same as in Requests:

proxies={"http": "http://proxy"} or proxies={"http://google.com": "http://proxy"}

or if you need more control:

proxies={"http": httpx.HTTPProxy("http://proxy", proxy_headers={"Proxy-Authorization": "...")}

The default HTTPProxy is a hybrid between these two where HTTP requests are sent via forwarding and HTTPS requests are sent via tunneling. Maybe I can merge the implementation of these two separate proxies into one an use arguments to control behavior like httpx.HTTPProxy(..., proxy_mode=httpx.ProxyMode.ALWAYS_TUNNEL) or the like.

[tomchristie]

Maybe I can merge the implementation of these two separate proxies into one

From an API point of view I think that having just HTTPProxy(...) and ConnectionPool(...) as our documented dispatcher classes is nicer than having two different class implementations for proxying yeah. (With a mode control available if neccessary?)

[sethmlarson]

You're right, I'll merge the two. What are your thoughts on getting the two separate implementations committed and then merging along with wiring up the Client as it's own PR?

Makes it easier to test the behavior of the specific proxy modes.

[tomchristie]

What are your thoughts on getting the two separate implementations committed and then merging along with wiring up the Client as it's own PR?

Probably would be okay with that since the dispatcher interface is documented already, yeah.

We should probably start exanding out the "Advanced" section of the docs tho.

[sethmlarson]

Closed in favor of #259