"cgi" standard library module used by bottle is deprecated in Python 3.11, to be removed in 3.13

[liyanage]

I'm running bottle with Python 3.11 for the first time and I see this deprecation warning:

/xxx/lib/bottle/bottle.py:72: DeprecationWarning: 'cgi' is deprecated and slated for removal in Python 3.13
  import base64, calendar, cgi, email.utils, functools, hmac, itertools,\

I guess bottle needs to stop using this module?

[defnull]

That's unfortunate, bottle uses the form-data/multipart parser from that library. The PEP says:

FieldStorage/MiniFieldStorage has no direct replacement, but can typically be replaced by using multipart (for POST and PUT requests) or urllib.parse.parse_qsl (for GET and HEAD requests)

Great. I'm the author of multipart, so I'm okay with that, but copy&pasting all that into bottle (to maintain the zero-dependency design goal) feels wrong.

[zb3]

Removing the multipart/form-data parser from Python standard library is unthinkable. I understand the cgi module was deprecated because CGI is itself deprecated, but the multipart encoding is not deprecated, it's still used today, hence I totally don't understand this..

[simsong]

PEP 594: Remove the cgi and cgitb modules, deprecated in Python 3.11.

cgi.FieldStorage can typically be replaced with urllib.parse.parse_qsl() for GET and HEAD requests, and the email.message module or multipart PyPI project for POST and PUT.

(per https://docs.python.org/3.13/whatsnew/3.13.html)

[fedepell]

Any news maybe on this? 3.13 is now getting closer with 3.12 being out in the meantime, albeit the release plan is around half of next year.

[defnull]

The only viable solution is to copy&paste code from multipart into bottle. I really really do not want to do that, but I have no other idea how to maintain the single-file no-deps aspect any other way.

[fedepell]

The only viable solution is to copy&paste code from multipart into bottle. I really really do not want to do that, but I have no other idea how to maintain the single-file no-deps aspect any other way.

Thanks for the fast reply! I see your point and absolutely agree is not "nice", but if we don't want to have dependencies, then I guess it's really the only way to go ("soon" or as Python 3.13 start getting traction).

Would you mind when you have time in case create a branch (possibly also for 0.12.x ?) with this so we can start testing? (I'm looking at this in the scope of future Fedora 41 planning to use it)

[aisk]

@defnull Hi can we using the email module in the standard library to avoid the copy / pasting?

[ilrico]

@defnull Since @aisk proposal does not work out (for large data as you underlined), what is your position on this issue ?

[simsong]

If the only requirement is to handle multipart and nothing else, it should not be hard to write a parser that does just that. multipart is not complex.

[ilrico]

I wonder if the no-deps line should not be open to compromise, as it is theorical: we see with this issue we are always dependant anyway, as least from std lib.

[simsong]

I'm also fine with copying the lines we from the cgi library into bottle.

[valq7711]

Here is the POC
https://github.com/valq7711/bottle/tree/multipart

Features:

• It parses body while writing request.body and produces a markup, which is stored in request.body.multipart_markup
• files are not tmp-files, but BytesIOProxy file-like objects, which are proxied to corresponding parts of request.body (no extra copying from request.body to tmp-files that cgi does), so it is ~30% faster
• nested multipart not supported
• only CRLF delimiters allowed (RFC 7578), single LFs not supported
• if there is an error while parsing, it is stored in request.body.multipart_markup.error and raised if only request.POST/forms/files touched, so if body has some exotic format, one can parse it using custom tools, without croaking of undesired errors (just do not touch request.POST/forms/files)

[simsong]

Looks good; are you going to commit a test as well?

[valq7711]

@simsong what do you mean? It passed the test https://github.com/valq7711/bottle/actions/runs/7160872053
Multipart test is in test_environ.py
The problem is that it is breaking changes as cgi supports nested multiparts and allows LF-delimiters (instead of CRLF)

[simsong]

Ah. I didn’t see that.

…

On Mon, Dec 11, 2023 at 1:25 AM valq7711 ***@***.***> wrote: @simsong <https://github.com/simsong> what do you mean? It passed the test https://github.com/valq7711/bottle/actions/runs/7160872053 Multipart test is in test_environ.py The problem is that it is breaking changes as cgi supports nested multiparts and allows LF-delimiters (instead of CRLF) — Reply to this email directly, view it on GitHub <#1403 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMFHLGUEIPLDPCVBTOOFXLYI2RNRAVCNFSM6AAAAAARZPDW7WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBZGQYDAMBSHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[ilrico]

hi folks, a gentle reminder that 3.13 in alpha 4 and cgi confirmed removed!

PEP 594 (Removing dead batteries from the standard library) scheduled removals of many deprecated modules: aifc, audioop, chunk, cgi, cgitb, crypt, imghdr, mailcap, msilib, nis, nntplib, ossaudiodev, pipes, sndhdr, spwd, sunau, telnetlib, uu, xdrlib, lib2to3.