Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flag for packages with native components (C, Rust etc.) #293

Open
IsakTheHacker opened this issue Oct 8, 2023 · 4 comments
Open

Flag for packages with native components (C, Rust etc.) #293

IsakTheHacker opened this issue Oct 8, 2023 · 4 comments
Labels
awaiting-feedback Issues that are waiting for the author's response enhancement help-wanted

Comments

@IsakTheHacker
Copy link

It would be excellent if you could add a flag for only showing packages that contain native code (C, C++, Rust etc.). Or just marking the native packages as bold or in another color

@xiacunshun
Copy link
Collaborator

Can you explain why you need this feature?

@IsakTheHacker
Copy link
Author

When running Python on a non standard platform such as Android using Chaquopy it is sometimes needed to use a PyPI package that transitively depends on a package with native code. Since Android is not an officially supported platform by PyPI nor Python itself all native packages that are used in an Android app are required to be built (and patched in some cases) manually using a special tool provided by Chaquopy. I would need this feature to find out which packages I would need to cross compile in order to build my Android app successfully. Would this be possible to implement?

@xiacunshun
Copy link
Collaborator

Thank you very much for your explanation @IsakTheHacker . I think I have a general understanding of your required scenario. However, this is a very specific case. If we were to scan every file of Python packages, it would inevitably slow down the execution speed of the tool, especially when the number is large. Moreover, our goal is to handle the dependency relationships between Python packages, so I feel that thisfeature somewhat exceeds the scope of the tool itself. If you want to achieve your goal, I think you can accomplish it by extending scripts based on the tool's results or using importlib.metadata to get all the distributions. For example:

from importlib.metadata import distributions
dists = distributions()
for d in dists:
    print(d.files)

@xiacunshun xiacunshun added the awaiting-feedback Issues that are waiting for the author's response label Apr 4, 2024
@kemzeb
Copy link
Collaborator

kemzeb commented May 10, 2024

I believe it would only be a performance problem when we end up using it as an option (e.g. --native or something like that) as only then we would need to recurse through each package's source directory looking for particular source files. This would also mean that we need to keep somehow determine if a file is C++, Java, Rust, etc. I'm unsure how we can do this reliably without having to store all the different file extensions that exist for these langauges.

One other thought I had was using the programming language classifiers that PyPi mentions here. This would make it really easy to determine if a package contains native code as we already have this metadata and we can just search for it. I'm not sure if this is reliable, however, as there could be packages that have, for example, C++ code but don't add the Programming Language :: C++ classifier.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting-feedback Issues that are waiting for the author's response enhancement help-wanted
Projects
None yet
Development

No branches or pull requests

4 participants