-
-
Notifications
You must be signed in to change notification settings - Fork 227
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
pkgutil.get_data() with in-memory resources #457
Comments
Here's the implementation of def get_data(package, resource):
spec = importlib.util.find_spec(package)
if spec is None:
return None
loader = spec.loader
if loader is None or not hasattr(loader, 'get_data'):
return None
# XXX needs test
mod = (sys.modules.get(package) or
importlib._bootstrap._load(spec))
if mod is None or not hasattr(mod, '__file__'):
return None
# Modify the resource name to be compatible with the loader.get_data
# signature - an os.path format "filename" starting with the dirname of
# the package's __file__
parts = resource.split('/')
parts.insert(0, os.path.dirname(mod.__file__))
resource_name = os.path.join(*parts)
return loader.get_data(resource_name) That I'm really tempted to give up on |
FWIW jsonschema is now using importlib.resources (https://github.com/Julian/jsonschema/pull/873/files) - v4.2.0. |
Hmm, the new code doesn't seem to work for me on a pyoxidized Python 3.9:
|
You're right @dae, I can reproduce when the location is |
jsonschema 4.2 introduced a change that broke our current workaround for in-memory support. python-jsonschema/jsonschema#873 indygreg/PyOxidizer#457
The new code breaks due to #237. It's doing exactly that:
|
Third-party libraries like jsonschema, that call pkgutil.get_data() to read their bundled resources, currently fail when the sources are in memory, as they lack a
__file__
attribute. PyOxidizer's ResourceReader interface does not require__file__
, and can successfully retrieve the resources. By dynamically replacing get_data() with a call to open_resource(), it appears to be possible to get jsonschema to work when in memory (mostly - it still has a dependency on an extension module):The sys.oxidized gate is necessary to ensure things run correctly outside PyOxidizer, as the default Python ResourceReader can not handle subdirs in the resource path.
I have not tested this with any other libraries yet, but presume this approach is not limited to jsonschema. It's a hack, and I'm not suggesting this be included in PyOxidizer itself, but I thought it might be worth mentioning in case anyone finds it useful.
Partially related to #436
The text was updated successfully, but these errors were encountered: