Enhancement: Split services into separate installable modules

[jwrackware]

I work for a company that uses this Python SDK as part of a product. The SDK works very well for us. Recently, the size of the SDK has become... not an issue, per se, but maybe "noticeable" is the right word? We only use a small part of the SDK which got me thinking about if support for each service could be selectively installed. So, for example, there could be an oci-core package that supports the Compute, Virtual Network, and Block Volume services. And maybe an oci-identity package for the Identity service. Etc. If I only need access to the Identity service, I could install only oci-identity and not have any source files related to the core services (or anything else).

I hope it won't be considered out-of-line to mention a competing cloud's SDK. The Azure Python SDK (https://github.com/Azure/azure-sdk-for-python) has implemented this kind of granular service installation (though I haven't actually used it). I could, for example, install just the azure-mgmt-compute package which would allow me to work with the compute service without installing anything else that I don't need. They appear to still use a single repository for all of the SDK components and just provide multiple packages to the Python Packaging Index. I haven't looked hard at their repository to see if there is anything special going on to support that but my guess is that they are simply using a package namespace to bring each separate package into a cohesive unit transparently to the user. I'm not at all a Python packaging expert.

I certainly don't claim that this is an easy request or that it should be started without some major design discussions. Just that I think it could be useful to your customers/users so may be worth putting on the roadmap. This also may have already been discussed. If so, I was unable to find that discussion in issues on this repository, the Cloud Customer Connect forums or the docs on readthedocs.io. I did also notice the Survey link in the docs and in this repository and that would have been my first choice for where to mention this request, but it appears that survey has been closed.

[github-anurag]

@jwrackware
You can enable selective imports for Python SDK by following the steps mentioned in:-

https://docs.oracle.com/en-us/iaas/tools/python/latest/sdk_behaviors/enable_selective_service_imports.html

We are looking into making the imports selective by default without introducing a breaking change in the SDK for existing customers.

[jwrackware]

@github-anurag, Thanks. I have seen that before, but it's a good reminder. In this issue I am specifically interested in selective install rather than selective import. We have to package and distribute our software and the OCI SDK takes a non-trivial amount of space.

[github-anurag]

Thanks for bringing this up @jwrackware . We will open a ticket internally to track this request.

[github-anurag]

FYI
We enabled the lazy imports when using Python 3.7+ with v2.88.1. We would appreciate it if you can check if using this version and up improves the startup time/memory utilized by OCI Python SDK
PS: You will need to disable selective imports mentioned earlier to use this feature.

[jwrackware]

Thanks @github-anurag that is great. However, I am more interested in selective install than selective/lazy import. The downloaded package size is the part that I would like to reduce. We have to include your python package in our packages and so if we can make the size of your package smaller, that would be ideal.

[jwrackware]

Another anecdote: I was looking at adding support for burstable instances but our current version of oci-python-sdk doesn't have the baseline_ocpu_utilization property on the LaunchInstanceShapeConfigDetails model. Upgrading to the latest version of oci-python-sdk grows our copy from 45M to 231M. Just going to the minimum version that includes baseline_ocpu_utilization is 175M. If we needed all of that it would be fine, but we need such a small portion that it seems pretty wasteful.

I tried updating to the latest version and then removing all the services that we don't use from the package. That got it down to 27M and everything we need still works. Certainly, I don't presume that our use case matches everyone else's. But we are using less than 12% of the latest version of the SDK. It would just be nice to have less wasted space.