3.12.rst

What's New In Python 3.12

Release

Date

This article explains the new features in Python 3.12, compared to 3.11.

For full details, see the changelog <changelog>.

Note

Prerelease users should be aware that this document is currently in draft form. It will be updated substantially as Python 3.12 moves towards release, so it's worth checking back even after reading earlier versions.

Summary -- Release highlights

Important deprecations, removals or restrictions:

• 623, Remove wstr from Unicode

New Features

Other Language Changes

• types.MappingProxyType instances are now hashable if the underlying mapping is hashable. (Contributed by Serhiy Storchaka in 87995.)
• Converting between int and str in bases other than 2 (binary), 4, 8 (octal), 16 (hexadecimal), or 32 such as base 10 (decimal) now raises a ValueError if the number of digits in string form is above a limit to avoid potential denial of service attacks due to the algorithmic complexity. This is a mitigation for CVE-2020-10735. This limit can be configured or disabled by environment variable, command line flag, or sys APIs. See the integer string conversion length limitation <int_max_str_digits> documentation. The default limit is 4300 digits in string form.

New Modules

• None yet.

Improved Modules

pathlib

• Add ~pathlib.Path.walk for walking the directory trees and generating all file or directory names within them, similar to os.walk. (Contributed by Stanislav Zmiev in 90385.)

dis

• Pseudo instruction opcodes (which are used by the compiler but do not appear in executable bytecode) are now exposed in the dis module. ~dis.HAVE_ARGUMENT is still relevant to real opcodes, but it is not useful for pseudo instrcutions. Use the new ~dis.hasarg collection instead. (Contributed by Irit Katriel in 94216.)

os

• Add os.PIDFD_NONBLOCK to open a file descriptor for a process with os.pidfd_open in non-blocking mode. (Contributed by Kumar Aditya in 93312.)

sqlite3

• Add a command-line interface <sqlite3-cli>. (Contributed by Erlend E. Aasland in 77617.)

Optimizations

• Removed wstr and wstr_length members from Unicode objects. It reduces object size by 8 or 16 bytes on 64bit platform. (623) (Contributed by Inada Naoki in 92536.)
• Added experimental support for using the BOLT binary optimizer in the build process, which improves performance by 1-5%. (Contributed by Kevin Modzelewski in 90536.)

CPython bytecode changes

• Removed the LOAD_METHOD instruction. It has been merged into LOAD_ATTR. LOAD_ATTR will now behave like the old LOAD_METHOD instruction if the low bit of its oparg is set. (Contributed by Ken Jin in 93429.)

Deprecated

• typing.Hashable and typing.Sized aliases for collections.abc.Hashable and collections.abc.Sized. (94309.)
• The sqlite3 default adapters and converters <sqlite3-default-converters> are now deprecated. Instead, use the sqlite3-adapter-converter-recipes and tailor them to your needs. (Contributed by Erlend E. Aasland in 90016.)

Pending Removal in Python 3.13

The following modules and APIs have been deprecated in earlier Python releases, and will be removed in Python 3.13.

Modules (see 594):

• aifc
• audioop
• cgi
• cgitb
• chunk
• crypt
• imghdr
• mailcap
• msilib
• nis
• nntplib
• ossaudiodev
• pipes
• sndhdr
• spwd
• sunau
• telnetlib
• uu
• xdrlib

APIs:

• configparser.LegacyInterpolation (90765)
• locale.getdefaultlocale (90817)
• turtle.RawTurtle.settiltangle (50096)
• unittest.findTestCases (50096)
• unittest.makeSuite (50096)
• unittest.getTestCaseNames (50096)
• webbrowser.MacOSX (86421)

Pending Removal in Python 3.14

• Deprecated the following importlib.abc classes, scheduled for removal in Python 3.14:

  • importlib.abc.ResourceReader
  • importlib.abc.Traversable
  • importlib.abc.TraversableResources

  Use importlib.resources.abc classes instead:

  • importlib.resources.abc.TraversableResources
  • importlib.resources.abc.Traversable
  • importlib.resources.abc.TraversableResources

  (Contributed by Jason R. Coombs and Hugo van Kemenade in 93963.)

• Creating :cimmutable types <Py_TPFLAGS_IMMUTABLETYPE> with mutable bases using the C API.

Pending Removal in Future Versions

The following APIs were deprecated in earlier Python versions and will be removed, although there is currently no date scheduled for their removal.

• typing.Text (92332)
• Currently Python accepts numeric literals immediately followed by keywords, for example 0in x, 1or x, 0if 1else 2. It allows confusing and ambiguous expressions like [0x1for x in y] (which can be interpreted as [0x1 for x in y] or [0x1f or x in y]). A syntax warning is raised if the numeric literal is immediately followed by one of keywords and, else, for, if, in, is and or. In a future release it will be changed to a syntax error. (87999)

Removed

• Removed many old deprecated unittest features:

  • A number of ~unittest.TestCase method aliases:

    Deprecated alias	Method Name	Deprecated in
    failUnless	.assertTrue	3.1
    failIf	.assertFalse	3.1
    failUnlessEqual	.assertEqual	3.1
    failIfEqual	.assertNotEqual	3.1
    failUnlessAlmostEqual	.assertAlmostEqual	3.1
    failIfAlmostEqual	.assertNotAlmostEqual	3.1
    failUnlessRaises	.assertRaises	3.1
    assert_	.assertTrue	3.2
    assertEquals	.assertEqual	3.2
    assertNotEquals	.assertNotEqual	3.2
    assertAlmostEquals	.assertAlmostEqual	3.2
    assertNotAlmostEquals	.assertNotAlmostEqual	3.2
    assertRegexpMatches	.assertRegex	3.2
    assertRaisesRegexp	.assertRaisesRegex	3.2
    assertNotRegexpMatches	.assertNotRegex	3.5

    You can use https://github.com/isidentical/teyit to automatically modernise your unit tests.

  • Undocumented and broken ~unittest.TestCase method assertDictContainsSubset (deprecated in Python 3.2).
  • Undocumented TestLoader.loadTestsFromModule <unittest.TestLoader.loadTestsFromModule> parameter use_load_tests (deprecated and ignored since Python 3.2).
  • An alias of the ~unittest.TextTestResult class: _TextTestResult (deprecated in Python 3.2).

  (Contributed by Serhiy Storchaka in 45162.)

• Several names deprecated in the configparser way back in 3.2 have been removed per 89336:
  • configparser.ParsingError no longer has a filename attribute or argument. Use the source attribute and argument instead.
  • configparser no longer has a SafeConfigParser class. Use the shorter ~configparser.ConfigParser name instead.
  • configparser.ConfigParser no longer has a readfp method. Use ~configparser.ConfigParser.read_file instead.

• The following undocumented sqlite3 features, deprecated in Python 3.10, are now removed:

  • sqlite3.enable_shared_cache()
  • sqlite3.OptimizedUnicode

  If a shared cache must be used, open the database in URI mode using the cache=shared query parameter.

  The sqlite3.OptimizedUnicode text factory has been an alias for str since Python 3.3. Code that previously set the text factory to OptimizedUnicode can either use str explicitly, or rely on the default value which is also str.

  (Contributed by Erlend E. Aasland in 92548.)

• The --experimental-isolated-subinterpreters configure flag (and corresponding EXPERIMENTAL_ISOLATED_SUBINTERPRETERS) have been removed.
• smtpd has been removed according to the schedule in 594, having been deprecated in Python 3.4.7 and 3.5.4. Use aiosmtpd PyPI module or any other asyncio-based server instead. (Contributed by Oleg Iarygin in 93243.)
• Remove io.OpenWrapper and _pyio.OpenWrapper, deprecated in Python 3.10: just use open instead. The open (io.open) function is a built-in function. Since Python 3.10, _pyio.open is also a static method. (Contributed by Victor Stinner in 94169.)
• Remove the ssl.RAND_pseudo_bytes function, deprecated in Python 3.6: use os.urandom or ssl.RAND_bytes instead. (Contributed by Victor Stinner in 94199.)
• gzip: Remove the filename attribute of gzip.GzipFile, deprecated since Python 2.6, use the ~gzip.GzipFile.name attribute instead. In write mode, the filename attribute added '.gz' file extension if it was not present. (Contributed by Victor Stinner in 94196.)
• Remove the ssl.match_hostname function. The ssl.match_hostname was deprecated in Python 3.7. OpenSSL performs hostname matching since Python 3.7, Python no longer uses the ssl.match_hostname function. (Contributed by Victor Stinner in 94199.)
• Remove the locale.format function, deprecated in Python 3.7: use locale.format_string instead. (Contributed by Victor Stinner in 94226.)
• hashlib: Remove the pure Python implementation of hashlib.pbkdf2_hmac(), deprecated in Python 3.10. Python 3.10 and newer requires OpenSSL 1.1.1 (644): this OpenSSL version provides a C implementation of ~hashlib.pbkdf2_hmac() which is faster. (Contributed by Victor Stinner in 94199.)
• xml.etree: Remove the ElementTree.Element.copy() method of the pure Python implementation, deprecated in Python 3.10, use the copy.copy function instead. The C implementation of xml.etree has no copy() method, only a __copy__() method. (Contributed by Victor Stinner in 94383.)
• zipimport: Remove find_loader() and find_module() methods, deprecated in Python 3.10: use the find_spec() method instead. See 451 for the rationale. (Contributed by Victor Stinner in 94379.)
• Remove the ssl.wrap_socket function, deprecated in Python 3.7: instead, create a ssl.SSLContext object and call its ssl.SSLContext.wrap_socket method. Any package that still uses ssl.wrap_socket is broken and insecure. The function neither sends a SNI TLS extension nor validates server hostname. Code is subject to CWE-295: Improper Certificate Validation. (Contributed by Victor Stinner in 94199.)

Porting to Python 3.12

This section lists previously described changes and other bugfixes that may require changes to your code.

Changes in the Python API

• More strict rules are now applied for numerical group references and group names in regular expressions. Only sequence of ASCII digits is now accepted as a numerical reference. The group name in bytes patterns and replacement strings can now only contain ASCII letters and digits and underscore. (Contributed by Serhiy Storchaka in 91760.)
• Removed randrange() functionality deprecated since Python 3.10. Formerly, randrange(10.0) losslessly converted to randrange(10). Now, it raises a TypeError. Also, the exception raised for non-integral values such as randrange(10.5) or randrange('10') has been changed from ValueError to TypeError. This also prevents bugs where randrange(1e25) would silently select from a larger range than randrange(10**25). (Originally suggested by Serhiy Storchaka gh-86388.)
• argparse.ArgumentParser changed encoding and error handler for reading arguments from file (e.g. fromfile_prefix_chars option) from default text encoding (e.g. locale.getpreferredencoding(False) <locale.getpreferredencoding>) to filesystem encoding and error handler. Argument files should be encoded in UTF-8 instead of ANSI Codepage on Windows.
• Removed the asyncore-based smtpd module deprecated in Python 3.4.7 and 3.5.4. A recommended replacement is the asyncio-based aiosmtpd PyPI module.
• shlex.split: Passing None for s argument now raises an exception, rather than reading sys.stdin. The feature was deprecated in Python 3.9. (Contributed by Victor Stinner in 94352.)

Build Changes

• Python no longer uses setup.py to build shared C extension modules. Build parameters like headers and libraries are detected in configure script. Extensions are built by Makefile. Most extensions use pkg-config and fall back to manual detection. (Contributed by Christian Heimes in 93939.)
• va_start() with two parameters, like va_start(args, format), is now required to build Python. va_start() is no longer called with a single parameter. (Contributed by Kumar Aditya in 93207.)

C API Changes

New Features

• Added the new limited C API function :cPyType_FromMetaclass, which generalizes the existing :cPyType_FromModuleAndSpec using an additional metaclass argument. (Contributed by Wenzel Jakob in 93012.)

• API for creating objects that can be called using the vectorcall protocol <vectorcall> was added to the Limited API <stable>:

  • Py_TPFLAGS_HAVE_VECTORCALL
  • :cPyVectorcall_NARGS
  • :cPyVectorcall_Call
  • :cvectorcallfunc

  The Py_TPFLAGS_HAVE_VECTORCALL flag is now removed from a class when the class's :py~object.__call__ method is reassigned. This makes vectorcall safe to use with mutable types (i.e. heap types without the immutable <Py_TPFLAGS_IMMUTABLETYPE> flag). Mutable types that do not override :c~PyTypeObject.tp_call now inherit the Py_TPFLAGS_HAVE_VECTORCALL flag. (Contributed by Petr Viktorin in 93274.)

  The Py_TPFLAGS_MANAGED_DICT and Py_TPFLAGS_MANAGED_WEAKREF flags have been added. This allows extensions classes to support object __dict__ and weakrefs with less bookkeeping, using less memory and with faster access.

Porting to Python 3.12

• Legacy Unicode APIs based on Py_UNICODE* representation has been removed. Please migrate to APIs based on UTF-8 or wchar_t*.
• Argument parsing functions like :cPyArg_ParseTuple doesn't support Py_UNICODE* based format (e.g. u, Z) anymore. Please migrate to other formats for Unicode like s, z, es, and U.
• tp_weaklist for all static builtin types is always NULL. This is an internal-only field on PyTypeObject but we're pointing out the change in case someone happens to be accessing the field directly anyway. To avoid breakage, consider using the existing public C-API instead, or, if necessary, the (internal-only) _PyObject_GET_WEAKREFS_LISTPTR() macro.

• This internal-only :cPyTypeObject.tp_subclasses may now not be a valid object pointer. Its type was changed to :cvoid * to reflect this. We mention this in case someone happens to be accessing the internal-only field directly.

  To get a list of subclasses, call the Python method :py~class.__subclasses__ (using :cPyObject_CallMethod, for example).

• An unrecognized format character in :cPyUnicode_FromFormat and :cPyUnicode_FromFormatV now sets a SystemError. In previous versions it caused all the rest of the format string to be copied as-is to the result string, and any extra arguments discarded. (Contributed by Serhiy Storchaka in 95781.)
• Fixed wrong sign placement in :cPyUnicode_FromFormat and :cPyUnicode_FromFormatV. (Contributed by Philip Georgi in 95504.)
• Extension classes wanting to add a __dict__ or weak reference slot should use Py_TPFLAGS_MANAGED_DICT and Py_TPFLAGS_MANAGED_WEAKREF instead of tp_dictoffset and tp_weaklistoffset, respectively. The use of tp_dictoffset and tp_weaklistoffset is still supported, but does not fully support multiple inheritance (:gh: 95589), and performance may be worse. Classes declaring Py_TPFLAGS_MANAGED_DICT should call :c_PyObject_VisitManagedDict and :c_PyObject_ClearManagedDict to traverse and clear their instance's dictionaries. To clear weakrefs, call :cPyObject_ClearWeakRefs, as before.

Deprecated

• Deprecate global configuration variable:

  • :cPy_DebugFlag: use :cPyConfig.parser_debug
  • :cPy_VerboseFlag: use :cPyConfig.verbose
  • :cPy_QuietFlag: use :cPyConfig.quiet
  • :cPy_InteractiveFlag: use :cPyConfig.interactive
  • :cPy_InspectFlag: use :cPyConfig.inspect
  • :cPy_OptimizeFlag: use :cPyConfig.optimization_level
  • :cPy_NoSiteFlag: use :cPyConfig.site_import
  • :cPy_BytesWarningFlag: use :cPyConfig.bytes_warning
  • :cPy_FrozenFlag: use :cPyConfig.pathconfig_warnings
  • :cPy_IgnoreEnvironmentFlag: use :cPyConfig.use_environment
  • :cPy_DontWriteBytecodeFlag: use :cPyConfig.write_bytecode
  • :cPy_NoUserSiteDirectory: use :cPyConfig.user_site_directory
  • :cPy_UnbufferedStdioFlag: use :cPyConfig.buffered_stdio
  • :cPy_HashRandomizationFlag: use :cPyConfig.use_hash_seed and :cPyConfig.hash_seed
  • :cPy_IsolatedFlag: use :cPyConfig.isolated
  • :cPy_LegacyWindowsFSEncodingFlag: use :cPyConfig.legacy_windows_fs_encoding
  • :cPy_LegacyWindowsStdioFlag: use :cPyConfig.legacy_windows_stdio
  • :cPy_FileSystemDefaultEncoding: use :cPyConfig.filesystem_encoding
  • :cPy_FileSystemDefaultEncodeErrors: use :cPyConfig.filesystem_errors
  • :cPy_UTF8Mode: use :cPyPreConfig.utf8_mode (see :cPy_PreInitialize)

  The :cPy_InitializeFromConfig API should be used with :cPyConfig instead. (Contributed by Victor Stinner in 77782.)

• Creating :cimmutable types <Py_TPFLAGS_IMMUTABLETYPE> with mutable bases is deprecated and will be disabled in Python 3.14.

Removed

• Remove the token.h header file. There was never any public tokenizer C API. The token.h header file was only designed to be used by Python internals. (Contributed by Victor Stinner in 92651.)

• Leagcy Unicode APIs has been removed. See 623 for detail.

  • :cPyUnicode_WCHAR_KIND
  • :cPyUnicode_AS_UNICODE
  • :cPyUnicode_AsUnicode
  • :cPyUnicode_AsUnicodeAndSize
  • :cPyUnicode_AS_DATA
  • :cPyUnicode_FromUnicode
  • :cPyUnicode_GET_SIZE
  • :cPyUnicode_GetSize
  • :cPyUnicode_GET_DATA_SIZE

• Remove the PyUnicode_InternImmortal() function and the SSTATE_INTERNED_IMMORTAL macro. (Contributed by Victor Stinner in 85858.)