bytes in form is handled in a very strange way

[trim21]

import httpx


r = httpx.post(
    "https://httpbin.org/post",
    data={
        "content": bytes.fromhex(
            "4f5f13d16da9f10d013933673a1ef4e7b974dbe3"
        ),  # random non-printable bytes
    },
)
print(r.json()["form"])

and it didn't throw a exception, but just transform bytes with str(b)

related: #1500 #1539 #1608

[tomchristie]

That's not ideal behavior, is it?

That's occurring because we've got an all-purpose string coercion here.

httpx/httpx/_content.py

Lines 136 to 149 in 4b85e6c

	def encode_urlencoded_data(
	data: RequestData,
	) -> tuple[dict[str, str], ByteStream]:
	plain_data = []
	for key, value in data.items():
	if isinstance(value, (list, tuple)):
	plain_data.extend([(key, primitive_value_to_str(item)) for item in value])
	else:
	plain_data.append((key, primitive_value_to_str(value)))
	body = urlencode(plain_data, doseq=True).encode("utf-8")
	content_length = str(len(body))
	content_type = "application/x-www-form-urlencoded"
	headers = {"Content-Length": content_length, "Content-Type": content_type}
	return headers, ByteStream(body)

httpx/httpx/_utils.py

Lines 56 to 68 in 4b85e6c

	def primitive_value_to_str(value: PrimitiveData) -> str:
	"""
	Coerce a primitive data type into a string value.
	Note that we prefer JSON-style 'true'/'false' for boolean values here.
	"""
	if value is True:
	return "true"
	elif value is False:
	return "false"
	elif value is None:
	return ""
	return str(value)

Presumably a better behavior here is to raise a TypeError for any form value that's not a string, and ensure the user is required to handle any coercion themselves?

There's actually a slightly wider design niggle here that I'd also like to pick up, but let's deal with one thing at a time.

[trim21]

bump to 0.28.x, maybe?

[trim21]

I think your suggestion in #3168 (reply in thread) looks fine to me,

[trim21]

better to create a issue and discuss there. Too much replies in this thread.

[tomchristie]

@trim21 - Okay would you like to open an issue for this?

We don't currently have a great place to write about this in the documentation, perhaps we can describe our API expectations to our users as part of the issue so that we've got something to later add into the docs.

Key points are...

• Form encoding is the same style that browsers use for uploading HTML forms.
• It either uses URL encoding or multipart encoding.
• Only text is supported. This isn't a structured data type like JSON.
• For convenience we support handling the following primitives [...]. Any other data needs to be coerced to a str.

[trim21]

#3170