Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

hive provider support for python 3.9 #21891

Closed
1 of 2 tasks
Aakcht opened this issue Mar 1, 2022 · 5 comments · Fixed by #21893
Closed
1 of 2 tasks

hive provider support for python 3.9 #21891

Aakcht opened this issue Mar 1, 2022 · 5 comments · Fixed by #21893
Assignees
@potiuk
Labels
area:providers kind:bug This is a clearly a bug

Comments

@Aakcht
Copy link
Contributor

Aakcht commented Mar 1, 2022

Apache Airflow Provider(s)

apache-hive

Versions of Apache Airflow Providers

No response

Apache Airflow version

2.2.4 (latest released)

Operating System

Debian “buster”

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

What happened

Hive provider can not be used in python 3.9 airflow image without also manually installing PyHive, sasl, thrift-sasl python libraries.

What you expected to happen

Hive provider can be used in python 3.9 airflow image after installing only hive provider.

How to reproduce

No response

Anything else

Looks like for python 3.9 hive provider support was removed in #15515 (comment) , because sasl library did not support python 3.9. However now python 3.9 is supported in sasl: cloudera/python-sasl#21 (comment) .

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct
@Aakcht Aakcht added area:providers kind:bug This is a clearly a bug labels Mar 1, 2022
@potiuk
Copy link
Member

potiuk commented Mar 1, 2022

ah cool. Thanks for the info @Aakcht - I am about to prepare new providers. So let me try this one out

@potiuk potiuk self-assigned this Mar 1, 2022
@potiuk
Copy link
Member

potiuk commented Mar 1, 2022

I think this is not as easy, unfortunately @Aakcht .

In order to add Python 3.9, this one should be implemented: dropbox/PyHive#380
PyHive uses thrift-sasl library which is different than the python-sasl library.

@potiuk
Copy link
Member

potiuk commented Mar 1, 2022

Ah I am wrong!

Originally it was about switching to thrift-sasl, but indeed now sasl 0.3.1 is released and this versio nis already downloaded in other python versions, so we shoudl be perfectly fine.

@potiuk potiuk mentioned this issue Mar 1, 2022
Merged
potiuk added a commit to potiuk/airflow that referenced this issue Mar 1, 2022
@potiuk
Add Python 3.9 support to Hive
59537ac 
Hive support for Python 3.9 has been removed in apache#15515 but cloudera
released new ssl 0.3.1 library version to support it and we should
be able to get Hive provider working for Python 3.9 too.

Fixes: apache#21891
potiuk added a commit to potiuk/airflow that referenced this issue Mar 1, 2022
@potiuk
Add Python 3.9 support to Hive
4e2c954 
Hive support for Python 3.9 has been removed in apache#15515 but cloudera
released new ssl 0.3.1 library version to support it and we should
be able to get Hive provider working for Python 3.9 too.

Fixes: apache#21891
potiuk added a commit that referenced this issue Mar 1, 2022
@potiuk
Add Python 3.9 support to Hive (#21893)
563ecfa 
Hive support for Python 3.9 has been removed in #15515 but cloudera
released new ssl 0.3.1 library version to support it and we should
be able to get Hive provider working for Python 3.9 too.

Fixes: #21891
@sergei3000
Copy link

Hi does all this mean Hive provider will work for Python 3.10 as well?

@Aakcht
Copy link
Contributor Author

Aakcht commented Jan 25, 2023

I didn't personally test it, but I don't see any issues related to hive provider and python 3.10 so I suspect it should work fine (since the latest airflow versions support python 3.10).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@potiuk potiuk

Labels
area:providers kind:bug This is a clearly a bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add Python 3.9 support to Hive potiuk/airflow
3 participants
@potiuk @Aakcht @sergei3000