please support nonstandard http request

[Arrnitage]

send http request by httpx

import httpx

url = "http://127.0.0.1:4444/?test|id"

resp = httpx.get(url=url)

result:

$ nc -nlvp 4444
Ncat: Version 7.94 ( https://nmap.org/ncat )
Ncat: Listening on [::]:4444
Ncat: Listening on 0.0.0.0:4444
Ncat: Connection from 127.0.0.1:63685.
GET /?test%7Cid HTTP/1.1
Host: 127.0.0.1:4444
Accept: */*
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
User-Agent: python-httpx/0.24.1

send http request by curl

curl -k -vv 'http://127.0.0.1:4444/?test|id'

result:

$ nc -nlvp 4444
Ncat: Version 7.94 ( https://nmap.org/ncat )
Ncat: Listening on [::]:4444
Ncat: Listening on 0.0.0.0:4444
Ncat: Connection from 127.0.0.1:63812.
GET /?test|id HTTP/1.1
Host: 127.0.0.1:4444
User-Agent: curl/8.1.2
Accept: */*

[zanieb]

Hi, thanks for engaging with the project!

Please use text in markdown code fences instead of screenshots.

We require issues to start as discussions, so this will be transferred.

[Arrnitage]

Many vulnerabilities are achieved by sending non-standard http requests. I hope httpx can provide a method to enable sending non-standard http requests, which will be more convenient when writing Proof of Concept python scripts.

[tomchristie]

If you'd like to understand what changes we'd need to make in order to accomodate this...

• Take a look at httpcore - figure out if you're able to achieve what you want at that level. (You are)
• Take a look at the custom transport documentation - figure out how you'd achieve this using a custom transport.
• Take a look at our URL API as a stand-alone interface, as well as prior conversations about the target or raw_path.

[rafalkrupinski]

There are various server implementations out in the wild. Perhaps httpx should accept a raw path (assumed to be url encoded) as an an alternative to normal path?

[tomchristie]

So... in order to implement this I'd suggest we need a "target" request extension, that allows users to supply a byte-string that is used as the HTTP target instead of the URL path.

This would allow us to:

• Support OPTIONS * requests.
• Support proxy-style CONNECT requests.
• Support requests using non-standard path escaping.

I'd be happy to guide anyone through making a PR for this behaviour.

[karpetrosyan]

So... in order to implement this I'd suggest we need a "target" request extension, that allows users to supply a byte-string that is used as the HTTP target instead of the URL path.

Oh, that's interesting! Can I take this?

[tomchristie]

Yeah that'd be fantastic @karpetrosyan.

We've got some docs on the requests/response extensions now, so we've got a good place to describe the functionality... https://github.com/encode/httpx/blob/master/docs/advanced/extensions.md