Respect enctype specified in forms

[jsm28]

MechanicalSoup currently ignores the enctype specified for a form when submitting it, instead using the default used by requests (which is application/x-www-form-urlencoded unless you specify "files" to upload). I'd like it to behave more like a browser by respecting the given enctype (apparently you should specify "files" to requests to upload form fields that aren't files as multipart/form-data - which is the case I'm concerned with, making sure to submit as multipart/form-data when that's what the form specifies as its enctype). This is for testing web applications which behave differently when you submit with the wrong enctype (concretely, the Python cgi module defaults to ignoring fields with empty values for application/x-www-form-urlencoded but not for multipart/form-data, so any application using that may behave differently for submissions with the wrong enctype).

[hemberger]

Thanks for submitting this issue! It certainly seems like we should be able to respect the form enctype, assuming that requests allows it as part of their API. If you'd like to submit a pull request, we'd be happy to work with you on it; otherwise, we'll look into this as soon as possible.

[jsm28]

My understanding as per e.g. https://franklingu.github.io/programming/2017/10/30/post-multipart-form-data-using-requests/ is that while requests doesn't have a feature for specifying the enctype directly, you can force it to use multipart/form-data by passing the non-file form fields as files with None as filename (I haven't tested doing this).

[MartinDlry]

I'm currently working on this issue:

In Requests code, we can see that it tests the files argument with if files , so if files is null or is an empty dictionary, it won't choose multipart/form-data.

As @jsm28 said we can force it by putting some data items in the files dictionary.

@moy suggested that we could instead pass a "fake dictionary" object as files argument. This "fake dictionary" would be an instance of a small class created for this purpose, that implements basic dictionary functions, and most importantly, returns true when tested with if.
This way we could keep non-files fields in the data dictionary, it seems more consistent to me but it'll take a few more lines of codes.

[moy]

Actually, the fact that we need such hack is also a sign that something is missing in requests. It probably makes sense to submit a feature request to requests (e.g. suggest a force_content_type argument or so to one of the methods building the request). That doesn't mean we can't work around the absence of such feature for now, but future callers of requests may appreciate having this option.