building: collect built-in extensions into lib-dynload sub-directory

On macOS and linux, some of the python's built-ins have extension
modules that originally reside in python3.X/lib-dynload directory.
This directory is in sys.path, therefore the collected extensions
have no parent directory and end up directly in the _MEIPASS.

This commit explicitly diverts such extensions into lib-dynload
sub-directory in the _MEIPASS.

In addition to decluttering the _MEIPASS on linux and macOS, this
also prevents ctypes.CDLL() from picking up the extensions'
shared libraries and causing inconsistent behavior between
frozen and unfrozen application, which in some corner cases
leads to issues with shadowing, such as in pyinstaller#5583.

PyInstaller/building/api.py

@@ -840,7 +840,11 @@ def _set_dependencies(self, analysis, path):
                     # relative path needs to be reconstructed from the
                     # name components.
                     if tpl[2] == 'EXTENSION':
-                        ext_components = tpl[0].split('.')[:-1]
+                        # Split on os.path.sep first, to handle additional
+                        # path prefix (e.g., lib-dynload)
+                        ext_components = tpl[0].split(os.path.sep)
+                        ext_components = ext_components[:-1] \
+                            + ext_components[-1].split('.')[:-1]
                         if ext_components:
                             rel_path = os.path.join(*ext_components)
                         else:

PyInstaller/building/build_main.py

@@ -480,6 +480,18 @@ def assemble(self):
             self.binding_redirects[:] = list(set(self.binding_redirects))
             logger.info("Found binding redirects: \n%s", self.binding_redirects)
 
+        # Filter binaries to adjust path of extensions that come from
+        # python's lib-dynload directory. Prefix them with lib-dynload
+        # so that we'll collect them into subdirectory instead of
+        # directly into _MEIPASS
+        for idx, tpl in enumerate(self.binaries):
+            name, path, typecode = tpl
+            if typecode == 'EXTENSION' \
+               and not os.path.dirname(os.path.normpath(name)) \
+               and os.path.basename(os.path.dirname(path)) == 'lib-dynload':
+                name = os.path.join('lib-dynload', name)
+                self.binaries[idx] = (name, path, typecode)
+
         # Place Python source in data files for the noarchive case.
         if self.noarchive:
             # Create a new TOC of ``(dest path for .pyc, source for .py, type)``.

news/5583.bugfix.rst

@@ -0,0 +1 @@
+(OSX) Fix issues with ``pycryptodomex`` on macOS.

news/5604.core.rst

@@ -0,0 +1,5 @@
+Collect python extension modules that correspond to built-ins into
+``lib-dynload`` sub-directory instead of directly into bundle's root
+directory. This prevents them from shadowing shared libraries with the
+same basename that are located in a package and loaded via ``ctypes`` or
+``cffi``, and also declutters the bundle's root directory.