Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

building: collect built-in extensions into lib-dynload sub-directory #5604

Merged
merged 5 commits into from Apr 16, 2021

Commits on Apr 8, 2021

  1. tests: add test for ctypes.CDLL incorrectly picking up builtin extens…

    …ion libraries
    
    On linux and macOS, some of the built-ins are provided as extensions
    (e.g., _sha1.cpython-39-x86_64-linux-gnu.so) that originally reside
    in python3.X/lib-dynload directory. This directory is not in the
    ctypes' library search path, therefore running
    ctypes.CDLL('_sha1.cpython-39-x86_64-linux-gnu.so')
    in python interpreter will come up empty.
    
    In a frozen application, however, these extensions end up collected
    directly in the _MEIPASS, which is searched by ctypes (because search
    behavior is explicitly overriden by the hook). Therefore, trying to
    load the extension's library file via ctypes.CDLL() will succeed,
    resulting in inconsistent behavior between unfrozen and frozen
    program.
    
    On macOS, this causes issues with `pycryptodomex` (pyinstaller#5583), which,
    due to its library search implementation, ends up with handle of
    `_sha1` extension module instead of its private extension/library
     with the same name prefix.
    rokm committed Apr 8, 2021
    Copy the full SHA
    a3d3089 View commit details
    Browse the repository at this point in the history
  2. building: collect built-in extensions into lib-dynload sub-directory

    On macOS and linux, some of the python's built-ins have extension
    modules that originally reside in python3.X/lib-dynload directory.
    This directory is in sys.path, therefore the collected extensions
    have no parent directory and end up directly in the _MEIPASS.
    
    This commit explicitly diverts such extensions into lib-dynload
    sub-directory in the _MEIPASS.
    
    In addition to decluttering the _MEIPASS on linux and macOS, this
    also prevents ctypes.CDLL() from picking up the extensions'
    shared libraries and causing inconsistent behavior between
    frozen and unfrozen application, which in some corner cases
    leads to issues with shadowing, such as in pyinstaller#5583.
    rokm committed Apr 8, 2021
    Copy the full SHA
    d1bc7b3 View commit details
    Browse the repository at this point in the history
  3. Copy the full SHA
    35e13c7 View commit details
    Browse the repository at this point in the history
  4. bootloader: ensure pypath and pypath_w buffers are of same size

    Having pypath with greater size than pypath_w invites potential
    trouble when path to executable is long enough that length of
    pypath string exceeds PATH_MAX. And it makes little sense for the
    two buffers to be of differently sized, anyway.
    rokm committed Apr 8, 2021
    Copy the full SHA
    c2f4c4e View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2021

  1. rebuild bootloaders

    Legorooj committed Apr 15, 2021
    Copy the full SHA
    a2362b1 View commit details
    Browse the repository at this point in the history