SNOW-395801: MemoryError when connecting on OSX using PyOpenSSL #799
Comments
github-actions
bot
changed the title
|
This issue is currently blocking users from installing dbt-snowflake on standard Apple M1 architectures. While it's not a blocker to using dbt + Snowflake at all—there are valid workarounds via Rosetta—it makes installation much more difficult than it needs to be. My understanding is that the operative issue here is the monkey-patch of It looks like there are two places where the Snowflake python connector turns on the monkey patch: The actual
My questions:
|
This is actually done by
Yes, unfortunately
We require |
|
@sfc-gh-mkeller Really appreciate the quick response. You're right, it looks like the most recent version of From pyca/pyopenssl#873 (comment), it sounds like
Do you have any plans, or see a way forward, to supporting native installations of this package on M1 Macbooks / ARM-based architectures? |
|
|
For what it's worth, I unsuccessfully tried a rosetta approach to install dbt. I also tried other recommended methods such as pip and virtualenv. So, while it's not blocking everyone, it's causing me a headache. |
|
We also have a bunch of users on M1 macs who we want to get set up on dbt; and this issue is causing us a fair amount of pain in our dbt installation |
|
As a temporary fix: This is a working solution using rosetta. I'm using it right now on a M1 MBP. |
sfc-gh-hkapre
added
enhancement
|
@jtcohen6 - Can you share an update on when this might get fixed? So far we can't get any of these workarounds to work with our new analysts' M1. |
|
Instead of the linked workaround above (installing Homebrew via Rosetta), I customized a docker image based on this image https://hub.docker.com/r/xemuliam/dbt. Docker Desktop now natively supports M1 architectures, and this image can build arm64 containers. |
|
In case it helps anyone, after many unsuccessful attempts, what worked for me was bumping python version to 3.10. |
|
I installed Python 3.9 through and then installed the current I read that Is anyone still able to reproduce this error? |
|
@sfc-gh-mkeller I can still reproduce this error with the following env: Env was created via miniforge with these commands: conda create -n test-env python=3.9.12
conda activate test-env
conda install -c conda-forge snowflake-connector-python==2.7.6Excerpts from And here is my minimal test function (with creds stored in env vars) import snowflake.connector
import os
con = snowflake.connector.connect(
user=os.getenv('SNOWFLAKE_USER'),
password=os.getenv('SNOWFLAKE_PASSWORD'),
account=os.getenv('SNOWFLAKE_ACCOUNT'),
role=os.getenv('SNOWFLAKE_ROLE'),
warehouse=os.getenv('SNOWFLAKE_WH'),
)
c = con.cursor().execute("select current_user()")
print(c.fetchone())EDIT: The only thing that's worked for me is to install everything under Rosetta emulation osx-64. However, Rosetta is not ideal for some other ML packages I want in the same env. |
|
@JasonTam I wonder if this is because of the conda compiled Python binary. However; I don't know anything about Anaconda. |
|
@sfc-gh-mkeller You're right, it's the I had to do this since there were also a whole bunch of things that required Otherwise, more simply, installing |
|
Since nobody seems to be having issues recently I'm going to close this issue, please don't hesitate to reach out if you are having issues! |
I still have issues. I got the error when trying to run "pip install cffi" following the suggestion mentioned above - My environment Any ideas? |
While trying to establish a connection, I receive a
MemoryErrorwhich appears to be connected to some low level permission error on OSX which is triggered by using PyOpenSSL, at least on M1 mac books
I believe this is caused by pyca/pyopenssl#873
Python 3.9.6macOS-11.4-arm64-arm-64bitConda list
What did you do?
If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
What did you expect to see?
What did you see instead?
Logs
The text was updated successfully, but these errors were encountered: