Supporting list for POST data

[devl00p]

Hello,

I'm trying to port my project from requests to httpx and I bumped into a problem with POST requests.

The error was 'async for' requires an object with __aiter__ method but after debugging in turns out that build_request expects a dict only for the data parameter while it worked with requests with a list of lists like this:

[['search', ''], ['op', 'GO'], ['form_build_id', 'form-TBxTpbD2PcPFGwD'], ['form_id', 'os_search_block_form']]

Is there any reason why data expects type RequestData (dict) and not QueryParamTypes like the params parameter ?

As my project is a web vulnerability scanner it is important to be able to give duplicate keys in order to detect some kind of vulnerabilities and it may benefit for other usages too (I already saw some websites using duplicate fields in a form as a way to detect bots ; meaning that browsers will send the duplicate field but not the bots).

Can an issue be created to allow more types for the data parameter ?

Thank you

[tomchristie]

Could you be more specific about how to reproduce the behaviour you're seeing?
A working example that's as minimal as possible would be a good starting point for helping you work through any issues here.

[devl00p]

Hello, here is an example to reproduce the issue:

import asyncio
import httpx

async def main():
        client = httpx.AsyncClient()
        request = client.build_request(
                "POST",
                "http://httpbin.org/post",
                data=[["action", "delete"], ["id", "1"], ["id", 2]]
        )
        await client.send(request)

asyncio.run(main())

Traceback:

Traceback (most recent call last):
  File "test.py", line 13, in <module>
    asyncio.run(main())
  File "/usr/lib64/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib64/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "test.py", line 11, in main
    await client.send(request)
  File "/home/me/.local/share/virtualenvs/myenv-p7I6n6KS/lib/python3.8/site-packages/httpx/_client.py", line 1396, in send
    response = await self._send_handling_auth(
  File "/home/me/.local/share/virtualenvs/myenv-p7I6n6KS/lib/python3.8/site-packages/httpx/_client.py", line 1434, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "/home/me/.local/share/virtualenvs/myenv-p7I6n6KS/lib/python3.8/site-packages/httpx/_client.py", line 1466, in _send_handling_redirects
    response = await self._send_single_request(request, timeout)
  File "/home/me/.local/share/virtualenvs/myenv-p7I6n6KS/lib/python3.8/site-packages/httpx/_client.py", line 1492, in _send_single_request
    (status_code, headers, stream, ext) = await transport.arequest(
  File "/home/me/.local/share/virtualenvs/myenv-p7I6n6KS/lib/python3.8/site-packages/httpx/_transports/default.py", line 169, in arequest
    return await self._pool.arequest(
  File "/home/me/.local/share/virtualenvs/myenv-p7I6n6KS/lib/python3.8/site-packages/httpcore/_async/connection_pool.py", line 218, in arequest
    response = await connection.arequest(
  File "/home/me/.local/share/virtualenvs/myenv-p7I6n6KS/lib/python3.8/site-packages/httpcore/_async/connection.py", line 106, in arequest
    return await self.connection.arequest(method, url, headers, stream, ext)
  File "/home/me/.local/share/virtualenvs/myenv-p7I6n6KS/lib/python3.8/site-packages/httpcore/_async/http11.py", line 66, in arequest
    await self._send_request_body(stream, timeout)
  File "/home/me/.local/share/virtualenvs/myenv-p7I6n6KS/lib/python3.8/site-packages/httpcore/_async/http11.py", line 109, in _send_request_body
    async for chunk in stream:
TypeError: 'async for' requires an object with __aiter__ method, got list

versions:

httpx==0.17.1
Python 3.8.8

[tomchristie]

Okay, so...

You can get the usage you're expecting as follows:

import asyncio
import httpx

async def main():
        client = httpx.AsyncClient()
        request = client.build_request(
                "POST",
                "http://httpbin.org/post",
                data={"action": "delete", "id": ["1", 2]}
        )
        await client.send(request)

asyncio.run(main())

But we might well want to support the data=<list of two-tuples> style.

[devl00p]

Nice, thank for your reply.