SNOW-634017: MemoryError: Cannot allocate write+execute memory for ffi.callback() #1205
Comments
github-actions
bot
changed the title
|
I think |
|
This comment seems to suggest that the issue might be caused by not using |
Can you give you some pointer on how to rebuild/check the Python executable with |
|
@sfc-gh-mkeller while Homebrew Python 3.9 might resolve this, Snowflake's own Snowpark doesn't support Python 3.9 (and doesn't look like it's going to any time soon), and Homebrew Python 3.8 doesn't resolve the issue. |
|
This seems to be a big blocker for M1 Mac users. Any chance to prioritize to fix this? |
|
I believe this is a direct consequence of the problem (with a suggested solution) described in #857
|
@sfc-gh-mkeller What is the proposed solution here then? This is a huge blocker for all M1 Mac users. EDIT: I am using conda-forge to install my environments. I can confirm that uninstalling the cffi package and reinstalling using pip seems to solve this issue. Does anyone know why that would fix this issue, and how it can be resolved so that package can be installed through conda? Using pip is not a solution for me. |
|
Folks, we have documented a workaround for this issue for Python3.8. Do try it out and let us know. |
Thanks for the update, @sfc-gh-achandrasekaran! Note that the first command in the guide, as it stands, leads to an error: It should probably be like this: |
|
Yep, that is it. We will update the docs. Thank you |
|
FYI, I've got it working now. Thanks for providing a workaround. |
|
Unfortunately, this workaround doesn't help when attempting to pair snowpark with tensorflow on an M1. tensorflow (installed via miniforge) requires |
|
I have get around this problem by updating my python version to 3.10 ( I've created a new conda environment with 3.10) and install snowflake-connector-python on it |
@anthonnyO Can we install snowflake-connector-python in 3.10? I tried to use pip but got the following error.
|
|
@ironerumi Anthony was talking about |
|
The root cause of this issue has been found and fixed by https://github.com/conda-forge/cffi-feedstock/pull/47/files |
|
Can we close this issue? |
|
@iamontheinet we want to make sure customers are no longer running into this before closing it. |
|
Thank you @sfc-gh-mkeller! I was able to fix this by installing channels:
- 'conda-forge'
- 'defaults'
dependencies:
- 'python'
- 'cffi >= 1.15.1=*_3'
- 'snowflake-connector-python'
# etc.Note that Conda Forge has builds for Python 3.8 - 3.10, but apparently not yet for 3.11, for this particular build revision. |
|
@jordantshaw It does on the the miniforge too thanks for the help |
|
conda-forge/cffi-feedstock#47 was just merged and Thank you! |
|
Hello, I'm unable to fix this error on M1 Mac. conda create -y -q --no-default-packages --name conda_env python=3.8 virtualenv
conda activate conda_env
python -m venv --system-site-packages venv
source venv/bin/activate
python -m pip install -U -q dbt-snowflake
python -m pip install --force-reinstall "cffi>=1.15.1"Even if if I do the installation of conda create -y -q --no-default-packages --name conda_env python=3.8 virtualenv cffi=1.15.1=*_3Thanks @jpzhangvincent for your issue. |
|
@4sushi I had no issues on my M1 with the YAML I posted above. Try conda create -p test-env python 'cffi>=1.15.1=*_3' snowflake-connector-pythonCreating a virtualenv inside a Conda env also seems redundant and unnecessary. Edit: Does CFFI 1.15.1 v3 support Python 3.8? I am using 3.10 in my project, but I know e.g. Snowpark still requires 3.8. |
Using python 3.10.4 this solution worked for me |
|
This has since been fixed. |
|
Hie can anybody please help me with this I am trying to run and getting the error |
|
I'm using a Macbook M1 with a conda environment for Python 3.10.4. PD: @aliamirr try this |
|
I am using macbook m2 and I am currently using python 3.8 |
|
This issue and two similar ones (#986 and #799) are closed (and some of those threads describe the issue as "fixed"), but it seems that they still occur, and at least some users still need to add an extra step of installing a particular cffi version to work around it. There is a workaround. FWIW I still hit this issue with the latest connector version. Below I create a new conda environment on my M1 Mac and try using the latest snowflake connector for python 3.8 (I end up with the latest version, 3.7.0). conda create -n snowflake-connector-python-m1 python=3.8
conda activate snowflake-connector-python-m1
pip install snowflake-connector-python
pip install ipython
ipythonimport snowflake.connector
snowflake.connector.connect()I get a stack trace ending with Stack trace---------------------------------------------------------------------------
MemoryError Traceback (most recent call last)
Cell In[2], line 1
----> 1 snowflake.connector.connect()
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/__init__.py:54, in Connect(**kwargs)
52 @wraps(SnowflakeConnection.__init__)
53 def Connect(**kwargs) -> SnowflakeConnection:
---> 54 return SnowflakeConnection(**kwargs)
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/connection.py:429, in SnowflakeConnection.__init__(self, connection_name, connections_file_path, **kwargs)
427 kwargs = _get_default_connection_params()
428 self.__set_error_attributes()
--> 429 self.connect(**kwargs)
430 self._telemetry = TelemetryClient(self._rest)
431 self.expired = False
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/connection.py:719, in SnowflakeConnection.connect(self, **kwargs)
717 raise Exception(str(exceptions_dict))
718 else:
--> 719 self.__open_connection()
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/connection.py:1045, in SnowflakeConnection.__open_connection(self)
1037 else:
1038 # okta URL, e.g., https://<account>.okta.com/
1039 self.auth_class = AuthByOkta(
1040 application=self.application,
1041 timeout=self._login_timeout,
1042 backoff_generator=self._backoff_generator,
1043 )
-> 1045 self.authenticate_with_retry(self.auth_class)
1047 self._password = None # ensure password won't persist
1048 self.auth_class.reset_secrets()
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/connection.py:1317, in SnowflakeConnection.authenticate_with_retry(self, auth_instance)
1314 def authenticate_with_retry(self, auth_instance) -> None:
1315 # make some changes if needed before real __authenticate
1316 try:
-> 1317 self._authenticate(auth_instance)
1318 except ReauthenticationRequest as ex:
1319 # cached id_token expiration error, we have cleaned id_token and try to authenticate again
1320 logger.debug("ID token expired. Reauthenticating...: %s", ex)
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/connection.py:1345, in SnowflakeConnection._authenticate(self, auth_instance)
1343 auth_instance._retry_ctx.set_start_time()
1344 try:
-> 1345 auth.authenticate(
1346 auth_instance=auth_instance,
1347 account=self.account,
1348 user=self.user,
1349 database=self.database,
1350 schema=self.schema,
1351 warehouse=self.warehouse,
1352 role=self.role,
1353 passcode=self._passcode,
1354 passcode_in_password=self._passcode_in_password,
1355 mfa_callback=self._mfa_callback,
1356 password_callback=self._password_callback,
1357 session_parameters=self._session_parameters,
1358 )
1359 except OperationalError as e:
1360 logger.debug(
1361 "Operational Error raised at authentication"
1362 f"for authenticator: {type(auth_instance).__name__}"
1363 )
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/auth/_auth.py:250, in Auth.authenticate(self, auth_instance, account, user, database, schema, warehouse, role, passcode, passcode_in_password, mfa_callback, password_callback, session_parameters, timeout)
244 logger.debug(
245 "body['data']: %s",
246 {k: v for (k, v) in body["data"].items() if k != "PASSWORD"},
247 )
249 try:
--> 250 ret = self._rest._post_request(
251 url,
252 headers,
253 json.dumps(body),
254 socket_timeout=auth_instance._socket_timeout,
255 )
256 except ForbiddenError as err:
257 # HTTP 403
258 raise err.__class__(
259 msg=(
260 "Failed to connect to DB. "
(...)
267 sqlstate=SQLSTATE_CONNECTION_WAS_NOT_ESTABLISHED,
268 )
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/network.py:734, in SnowflakeRestful._post_request(self, url, headers, body, token, timeout, socket_timeout, _no_results, no_retry, _include_retry_params)
731 ret = probe_connection(full_url)
732 pprint(ret)
--> 734 ret = self.fetch(
735 "post",
736 full_url,
737 headers,
738 data=body,
739 timeout=timeout,
740 token=token,
741 no_retry=no_retry,
742 _include_retry_params=_include_retry_params,
743 socket_timeout=socket_timeout,
744 )
745 logger.debug(
746 "ret[code] = {code}, after post request".format(
747 code=(ret.get("code", "N/A"))
748 )
749 )
751 if ret.get("code") == MASTER_TOKEN_EXPIRED_GS_CODE:
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/network.py:843, in SnowflakeRestful.fetch(self, method, full_url, headers, data, timeout, **kwargs)
841 retry_ctx.set_start_time()
842 while True:
--> 843 ret = self._request_exec_wrapper(
844 session, method, full_url, headers, data, retry_ctx, **kwargs
845 )
846 if ret is not None:
847 return ret
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/network.py:979, in SnowflakeRestful._request_exec_wrapper(self, session, method, full_url, headers, data, retry_ctx, no_retry, token, **kwargs)
977 except Exception as e:
978 if not no_retry:
--> 979 raise e
980 logger.debug("Ignored error", exc_info=True)
981 return {}
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/network.py:884, in SnowflakeRestful._request_exec_wrapper(self, session, method, full_url, headers, data, retry_ctx, no_retry, token, **kwargs)
882 full_url = SnowflakeRestful.add_request_guid(full_url)
883 try:
--> 884 return_object = self._request_exec(
885 session=session,
886 method=method,
887 full_url=full_url,
888 headers=headers,
889 data=data,
890 token=token,
891 **kwargs,
892 )
893 if return_object is not None:
894 return return_object
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/network.py:1193, in SnowflakeRestful._request_exec(self, session, method, full_url, headers, data, token, catch_okta_unauthorized_error, is_raw_text, is_raw_binary, binary_data_handler, socket_timeout, is_okta_authentication)
1183 except Exception as err:
1184 TelemetryService.get_instance().log_http_request_error(
1185 "HttpException%s" % str(err),
1186 full_url,
(...)
1191 stack_trace=traceback.format_exc(),
1192 )
-> 1193 raise err
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/network.py:1078, in SnowflakeRestful._request_exec(self, session, method, full_url, headers, data, token, catch_okta_unauthorized_error, is_raw_text, is_raw_binary, binary_data_handler, socket_timeout, is_okta_authentication)
1074 download_start_time = get_time_millis()
1075 # socket timeout is constant. You should be able to receive
1076 # the response within the time. If not, ConnectReadTimeout or
1077 # ReadTimeout is raised.
-> 1078 raw_ret = session.request(
1079 method=method,
1080 url=full_url,
1081 headers=headers,
1082 data=input_data,
1083 timeout=socket_timeout,
1084 verify=True,
1085 stream=is_raw_binary,
1086 auth=SnowflakeAuth(token),
1087 )
1088 download_end_time = get_time_millis()
1090 try:
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/vendored/requests/sessions.py:589, in Session.request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
584 send_kwargs = {
585 "timeout": timeout,
586 "allow_redirects": allow_redirects,
587 }
588 send_kwargs.update(settings)
--> 589 resp = self.send(prep, **send_kwargs)
591 return resp
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/vendored/requests/sessions.py:703, in Session.send(self, request, **kwargs)
700 start = preferred_clock()
702 # Send the request
--> 703 r = adapter.send(request, **kwargs)
705 # Total elapsed time of the request (approximately)
706 elapsed = preferred_clock() - start
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/vendored/requests/adapters.py:486, in HTTPAdapter.send(self, request, stream, timeout, verify, cert, proxies)
483 timeout = TimeoutSauce(connect=timeout, read=timeout)
485 try:
--> 486 resp = conn.urlopen(
487 method=request.method,
488 url=url,
489 body=request.body,
490 headers=request.headers,
491 redirect=False,
492 assert_same_host=False,
493 preload_content=False,
494 decode_content=False,
495 retries=self.max_retries,
496 timeout=timeout,
497 chunked=chunked,
498 )
500 except (ProtocolError, OSError) as err:
501 raise ConnectionError(err, request=request)
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/vendored/urllib3/connectionpool.py:715, in HTTPConnectionPool.urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
712 self._prepare_proxy(conn)
714 # Make the request on the httplib connection object.
--> 715 httplib_response = self._make_request(
716 conn,
717 method,
718 url,
719 timeout=timeout_obj,
720 body=body,
721 headers=headers,
722 chunked=chunked,
723 )
725 # If we're going to release the connection in ``finally:``, then
726 # the response doesn't need to know about the connection. Otherwise
727 # it will also try to release it and we'll have a double-release
728 # mess.
729 response_conn = conn if not release_conn else None
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/vendored/urllib3/connectionpool.py:404, in HTTPConnectionPool._make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
402 # Trigger any extra validation we need to do.
403 try:
--> 404 self._validate_conn(conn)
405 except (SocketTimeout, BaseSSLError) as e:
406 # Py2 raises this as a BaseSSLError, Py3 raises it as socket timeout.
407 self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/vendored/urllib3/connectionpool.py:1058, in HTTPSConnectionPool._validate_conn(self, conn)
1056 # Force connect early to allow us to validate the connection.
1057 if not getattr(conn, "sock", None): # AppEngine might not have `.sock`
-> 1058 conn.connect()
1060 if not conn.is_verified:
1061 warnings.warn(
1062 (
1063 "Unverified HTTPS request is being made to host '%s'. "
(...)
1068 InsecureRequestWarning,
1069 )
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/vendored/urllib3/connection.py:400, in HTTPSConnection.connect(self)
398 if self.ssl_context is None:
399 default_ssl_context = True
--> 400 self.ssl_context = create_urllib3_context(
401 ssl_version=resolve_ssl_version(self.ssl_version),
402 cert_reqs=resolve_cert_reqs(self.cert_reqs),
403 )
405 context = self.ssl_context
406 context.verify_mode = resolve_cert_reqs(self.cert_reqs)
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/vendored/urllib3/util/ssl_.py:339, in create_urllib3_context(ssl_version, cert_reqs, options, ciphers)
333 # The order of the below lines setting verify_mode and check_hostname
334 # matter due to safe-guards SSLContext has to prevent an SSLContext with
335 # check_hostname=True, verify_mode=NONE/OPTIONAL. This is made even more
336 # complex because we don't know whether PROTOCOL_TLS_CLIENT will be used
337 # or not so we don't know the initial state of the freshly created SSLContext.
338 if cert_reqs == ssl.CERT_REQUIRED:
--> 339 context.verify_mode = cert_reqs
340 disable_check_hostname()
341 else:
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/snowflake/connector/vendored/urllib3/contrib/pyopenssl.py:442, in PyOpenSSLContext.verify_mode(self, value)
440 @verify_mode.setter
441 def verify_mode(self, value):
--> 442 self._ctx.set_verify(_stdlib_to_openssl_verify[value], _verify_callback)
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/OpenSSL/SSL.py:1264, in Context.set_verify(self, mode, callback)
1261 if not callable(callback):
1262 raise TypeError("callback must be callable")
-> 1264 self._verify_helper = _VerifyHelper(callback)
1265 self._verify_callback = self._verify_helper.callback
1266 _lib.SSL_CTX_set_verify(self._context, mode, self._verify_callback)
File ~/miniconda3/envs/snowflake-connector-python-m1/lib/python3.8/site-packages/OpenSSL/SSL.py:494, in _VerifyHelper.__init__(self, callback)
491 else:
492 return 0
--> 494 self.callback = _ffi.callback(
495 "int (*)(int, X509_STORE_CTX *)", wrapper
496 )
MemoryError: Cannot allocate write+execute memory for ffi.callback(). You might be running on a system that prevents this. For more information, see https://cffi.readthedocs.io/en/latest/using.html#callbacksSystem:
|
Please answer these questions before submitting your issue. Thanks!
Python 3.9.13 | packaged by conda-forge
What operating system and processor architecture are you using?
macOS-10.16-arm64-arm-64bit
What are the component versions in the environment (
pip freeze)?cffi==1.15.1
cryptography==37.0.4
snowflake-sqlalchemy==1.3.4
snowflake-connector-python==2.7.9
requests==2.28.1
SQLAlchemy==1.4.39
What did you do?
I installed the package with 'pip install snowflake-connector-python' and then simply tried to run a query with the created snowflow connector engine. But I got the error
Please help! I had spent a lot of time debugging and tried different workaround. For example, using the suggestion from this post
I suspect it's related to cffi since when I run
pip install cffiI still got the errorThanks a lot!
The text was updated successfully, but these errors were encountered: