Assertion failure in _sempahore.py

[dmbaggett]

• gevent 23.9.1 (built from source)
• greenlet 3.0.0.rc3 (wheel built from source)
• cPython 3.12 (built from source)
• Ubuntu 22.04.3 LTS on aarch64 (AWS Graviton3)

Description:

Very rarely (only seen it once), when the lauch-darkly client aquires a read lock an assertion gets raised in _semphore.py.

  File "/home/ubuntu/inky-main/ink-drop/addinserver/server_utils.py", line 66, in wrapped_method
    return method(*args, **kwargs)
  File "/home/ubuntu/inky-main/ink-drop/addinserver/inky_analysis_server.py", line 4263, in check_sender_profile
    if feature_flags.check("disable-all-sender-profiling", data.get('teamid'), False):
  File "/home/ubuntu/inky-main/ink-drop/addinserver/feature_flags.py", line 108, in check
    log.exception('caught exception checking feature flag; will use default')
  File "/home/ubuntu/inky-main/ink-drop/addinserver/feature_flags.py", line 99, in check
    return ldclient.get().variation(feature_name, {'key': teamid, 'custom': CUSTOM_ATTRIBUTES }, default)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/install/linux/3.12/lib/python3.12/site-packages/ldclient/client.py", line 289, in variation
    return self._evaluate_internal(key, context, default, self._event_factory_default).value
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/install/linux/3.12/lib/python3.12/site-packages/ldclient/client.py", line 317, in _evaluate_internal
    if not self.is_initialized():
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/install/linux/3.12/lib/python3.12/site-packages/ldclient/client.py", line 262, in is_initialized
    return self.is_offline() or self._config.use_ldd or self._update_processor.initialized()
                                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/install/linux/3.12/lib/python3.12/site-packages/ldclient/impl/datasource/streaming.py", line 123, in initialized
    return self._running and self._ready.is_set() is True and self._store.initialized is True
                                                              ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/install/linux/3.12/lib/python3.12/site-packages/ldclient/client.py", line 62, in initialized
    return self.store.initialized
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/install/linux/3.12/lib/python3.12/site-packages/ldclient/feature_store.py", line 161, in initialized
    self._lock.rlock()
  File "/home/ubuntu/install/linux/3.12/lib/python3.12/site-packages/ldclient/impl/rwlock.py", line 15, in rlock
    self._read_ready.acquire()
  File "/home/ubuntu/install/linux/3.12/lib/python3.12/site-packages/gevent/thread.py", line 112, in acquire
    acquired = BoundedSemaphore.acquire(self, blocking, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "src/gevent/_semaphore.py", line 180, in gevent._gevent_c_semaphore.Semaphore.acquire
  File "src/gevent/_semaphore.py", line 272, in gevent._gevent_c_semaphore.Semaphore.acquire
AssertionError: (0, True, None, True)

This has happened after processing many thousands of messages so whatever it is, it's very low-frequency.

Note that I saw something similar under Python 3.11 when testing issue #1985 though in that case it manifested as a timeout and was more frequent.