Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Python 3.9 support to Hive #21893

Merged
merged 1 commit into from Mar 1, 2022

Conversation

potiuk
Copy link
Member

@potiuk potiuk commented Mar 1, 2022

Hive support for Python 3.9 has been removed in #15515 but cloudera
released new ssl 0.3.1 library version to support it and we should
be able to get Hive provider working for Python 3.9 too.

Fixes: #21891


^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

@potiuk
Copy link
Member Author

potiuk commented Mar 1, 2022

I would love this one merged before I release a new wave of providers for Feb

@potiuk potiuk force-pushed the enable-hive-provider-for-python-3.9 branch from f2f7f73 to 31d3da8 Compare March 1, 2022 12:41
@uranusjr
Copy link
Member

uranusjr commented Mar 1, 2022

I see dropbox/PyHive#380 is still open. What changed to make this viable?

@potiuk
Copy link
Member Author

potiuk commented Mar 1, 2022

I see dropbox/PyHive#380 is still open. What changed to make this viable?

Yeah, I thought the same initially and commented the same - but I looked closer - see #21891 discussion. Rather than PyHive switching to another implementation (pure-sasl based) cloudera released the sasl library with 3.9 support - this was the 5 years old library I mentioned in the original PR https://pypi.org/project/sasl/#history

I am going to close dropbox/PyHive#380 as soon as we merge this one.

@potiuk potiuk added the full tests needed We need to run full set of tests for this PR to merge label Mar 1, 2022
@potiuk potiuk closed this Mar 1, 2022
@potiuk potiuk reopened this Mar 1, 2022
@potiuk
Copy link
Member Author

potiuk commented Mar 1, 2022

Closed/Reopened to ACTUALLY run Python 3.9 tests (with full tests needed label :)

@potiuk potiuk force-pushed the enable-hive-provider-for-python-3.9 branch from 31d3da8 to 8240de3 Compare March 1, 2022 13:03
@potiuk
Copy link
Member Author

potiuk commented Mar 1, 2022

And 🤦 I also removed the skipIfs to actually RUN the tests on Python 3.9 :D

@uranusjr
Copy link
Member

uranusjr commented Mar 1, 2022

Ah, so the 0.3 series of sasl supports 3.9, solving our issue.

Perhaps it’d be worthwhile to add some additional specifiers to describe this? e.g.

# Some comment explaining 3.0 added 3.9 support...
sasl; python_version<"3.9"
sasl>=3.0; python_version>="3.9"

@potiuk
Copy link
Member Author

potiuk commented Mar 1, 2022

Ah, so the 0.3 series of sasl supports 3.9, solving our issue.

Perhaps it’d be worthwhile to add some additional specifiers to describe this? e.g.

# Some comment explaining 3.0 added 3.9 support...
sasl; python_version<"3.9"
sasl>=3.0; python_version>="3.9"

Good idea. I prefer not to complicate our setup if not needed sasl is a very specific library and pragmatically speaking anyone installing hive provider on Python 3.9 wll get latest (0.3.1) version installed. But yeah that might be a good idea as there might be some cases where someone has an older version of sasl library installed.

Adding it.

@potiuk potiuk force-pushed the enable-hive-provider-for-python-3.9 branch from 8240de3 to 52795fd Compare March 1, 2022 13:16
@potiuk
Copy link
Member Author

potiuk commented Mar 1, 2022

Added it now @uranusjr ! Thanks for pointing it out ! I actually made it >=3.0.1 as apparently 3.0.1 0.3.1 was released a day after 3.0.0 0.3.0 (after 5 years) and this might only mean one thing even if 3.0.0 0.3.0 is not yanked :)

@potiuk potiuk force-pushed the enable-hive-provider-for-python-3.9 branch from 52795fd to 59537ac Compare March 1, 2022 13:41
@potiuk
Copy link
Member Author

potiuk commented Mar 1, 2022

🤦 🤦 🤦 0.3.1 not 3.0.1 @potiuk. Try to copy&paste next time :D

@potiuk potiuk force-pushed the enable-hive-provider-for-python-3.9 branch 2 times, most recently from 06b5e10 to 991fc27 Compare March 1, 2022 15:28
Hive support for Python 3.9 has been removed in apache#15515 but cloudera
released new ssl 0.3.1 library version to support it and we should
be able to get Hive provider working for Python 3.9 too.

Fixes: apache#21891
@potiuk potiuk force-pushed the enable-hive-provider-for-python-3.9 branch from 991fc27 to 4e2c954 Compare March 1, 2022 20:40
@potiuk potiuk closed this Mar 1, 2022
@potiuk potiuk reopened this Mar 1, 2022
@potiuk
Copy link
Member Author

potiuk commented Mar 1, 2022

Just rundom "maybe ? " test

@potiuk potiuk merged commit 563ecfa into apache:main Mar 1, 2022
@potiuk potiuk deleted the enable-hive-provider-for-python-3.9 branch March 1, 2022 22:16
@ephraimbuddy ephraimbuddy added the changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) label Apr 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) full tests needed We need to run full set of tests for this PR to merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

hive provider support for python 3.9
4 participants