Worker instance keep restarting after upgrade to 4.0.0-dev

[fzhan]

Bug description

After upgrade to 4.0.0, the workers keeps restarting

How to reproduce the bug

Install helm with this value.yaml

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# Default values for superset.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

# A README is automatically generated from this file to document it, using helm-docs (see https://github.com/norwoodj/helm-docs)
# To update it, install helm-docs and run helm-docs from the root of this chart

# -- Provide a name to override the name of the chart
nameOverride: ~
# -- Provide a name to override the full names of resources
fullnameOverride: ~

# -- User ID directive. This user must have enough permissions to run the bootstrap script
# Running containers as root is not recommended in production. Change this to another UID - e.g. 1000 to be more secure
runAsUser: 0

# -- Specify service account name to be used
serviceAccountName: ~
serviceAccount:
  # -- Create custom service account for Superset. If create: true and serviceAccountName is not provided, `superset.fullname` will be used.
  create: false
  annotations: {}

# -- Install additional packages and do any other bootstrap configuration in this script
# For production clusters it's recommended to build own image with this step done in CI
# @default -- see `values.yaml`
bootstrapScript: |
  #!/bin/bash

  pip install --upgrade pip &&\
  pip install sqlalchemy-bigquery redis \
    pymssql pyodbc pymysql mysql-connector-python holidays==0.32 prophet==1.1.4 authlib
  
  apt-get update
  apt-get install -y wget zip libodbc2
  wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
  apt-get install -y --no-install-recommends ./google-chrome-stable_current_amd64.deb
  CHROMEVER=$(google-chrome --product-version | grep -o "[^\.]*\.[^\.]*\.[^\.]*")
  DRIVERVER=$(curl -s "https://chromedriver.storage.googleapis.com/LATEST_RELEASE_$CHROMEVER")
  wget "https://storage.googleapis.com/chrome-for-testing-public/124.0.6367.60/linux64/chromedriver-linux64.zip"
  unzip chromedriver-linux64.zip
  ls -la
  chmod +x chromedriver-linux64/chromedriver
  mv chromedriver-linux64/chromedriver /usr/local/bin
  apt-get autoremove -yqq --purge
  apt-get clean
  rm -f google-chrome-stable_current_amd64.deb chromedriver_linux64.zip

  if [ ! -f ~/bootstrap ]; then echo "Running Superset with uid {{ .Values.runAsUser }}" > ~/bootstrap; fi

# -- The name of the secret which we will use to generate a superset_config.py file
# Note: this secret must have the key superset_config.py in it and can include other files as well
configFromSecret: '{{ template "superset.fullname" . }}-config'

# -- The name of the secret which we will use to populate env vars in deployed pods
# This can be useful for secret keys, etc.
envFromSecret: '{{ template "superset.fullname" . }}-env'
# -- This can be a list of templated strings
envFromSecrets: []

# -- Extra environment variables that will be passed into pods
extraEnv: 
  # Different gunicorn settings, refer to the gunicorn documentation
  # https://docs.gunicorn.org/en/stable/settings.html#
  # These variables are used as Flags at the gunicorn startup
  # https://github.com/apache/superset/blob/master/docker/run-server.sh#L22
  # Extend timeout to allow long running queries.
  GUNICORN_TIMEOUT: 300
  # Increase the gunicorn worker amount, can improve performance drastically
  # See: https://docs.gunicorn.org/en/stable/design.html#how-many-workers
  SERVER_WORKER_AMOUNT: 4
  WORKER_MAX_REQUESTS: 0
  WORKER_MAX_REQUESTS_JITTER: 0
  SERVER_THREADS_AMOUNT: 20
  GUNICORN_KEEPALIVE: 2
  SERVER_LIMIT_REQUEST_LINE: 0
  SERVER_LIMIT_REQUEST_FIELD_SIZE: 0
  SUPERSET_WEBSERVER_PROTOCOL: "http"
  WEBDRIVER_BASEURL_USER_FRIENDLY: "https://company-domain.***.com/"
  SMTP_HOST: "mail.***.com"
  SMTP_PORT: 2525
  SMTP_STARTTLS: 'True'
  SMTP_SSL_SERVER_AUTH: 'True'
  SMTP_SSL: 'False'
  SMTP_USER: "smtp@***.com"
  SMTP_MAIL_FROM: "noreply@***.com"
  EMAIL_REPORTS_SUBJECT_PREFIX: "[*** BI] "

  # OAUTH_HOME_DOMAIN: ..
  # # If a whitelist is not set, any address that can use your OAuth2 endpoint will be able to login.
  # #   this includes any random Gmail address if your OAuth2 Web App is set to External.
  # OAUTH_WHITELIST_REGEX: ...

# -- Extra environment variables in RAW format that will be passed into pods
extraEnvRaw: 
  # Load DB password from other secret (e.g. for zalando operator)
  - name: REDIS_PASS
    valueFrom: 
      secretKeyRef:
        name: redis
        key: redis-password

# -- Extra environment variables to pass as secrets
extraSecretEnv: 
  SMTP_PASSWORD: "***"
  MAPBOX_API_KEY: '***'
  SUPERSET_SECRET_KEY: '***'
  # MAPBOX_API_KEY: ...
  # # Google API Keys: https://console.cloud.google.com/apis/credentials
  # GOOGLE_KEY: ...
  # GOOGLE_SECRET: ...
  #   # Generate your own secret key for encryption. Use openssl rand -base64 42 to generate a good key
  #  SUPERSET_SECRET_KEY: 'CHANGE_ME_TO_A_COMPLEX_RANDOM_SECRET'

# -- Extra files to mount on `/app/pythonpath`
extraConfigs: 
  custom_sso_security_manager.py: | 
      import logging
      from superset.security import SupersetSecurityManager

      logger = logging.getLogger("azure_ad")

      class CustomSsoSecurityManager(SupersetSecurityManager):
        from flask_appbuilder.security.manager import (AUTH_DB, AUTH_OAUTH)
        def oauth_user_info(self, provider, response=None):
          #In case if userDetail url is nor working case use response attribute to get the user details like name,mail etc
          logger.debug("Oauth2 provider: {0}.".format(provider))
          if provider == 'azure':
              # As example, this line request a GET to base_url + '/' + userDetails with Bearer  Authentication,
              # and expects that authorization server checks the token, and response with user details
              #me = self.appbuilder.sm.oauth_remotes[provider].get('userDetails').data
              #logging.debug("user_data: {0}".format(me))
              #check the values of user_name, mail and others values in me variable
              #return { 'name' : me['User_name'], 'email' : me['mail'], 'id' : me['user_name'], 'username' : me['user_name'], 'first_name':'', 'last_name':''}

              logger.debug("Azure response received : {0}".format(response))
              id_token = response["id_token"]
              logger.debug(str(id_token))
              #logging.debug(" -- oauth user detail -- : {0}".format(self.appbuilder.sm.oauth_remotes[provider].get('userDetails')))
              me = self._decode_and_validate_azure_jwt(id_token)
              logger.debug("Parse JWT token : {0}".format(me))
              return {
                  "name": me.get("name", ""),
                  "email": me["upn"],
                  "first_name": me.get("given_name", ""),
                  "last_name": me.get("family_name", ""),
                  "id": me["unique_name"],
                  "username": me["upn"],
                  "role_keys": me.get("roles", []),
              }
  # import_datasources.yaml: |
  #     databases:
  #     - allow_file_upload: true
  #       allow_ctas: true
  #       allow_cvas: true
  #       database_name: example-db
  #       extra: "{\r\n    \"metadata_params\": {},\r\n    \"engine_params\": {},\r\n    \"\
  #         metadata_cache_timeout\": {},\r\n    \"schemas_allowed_for_file_upload\": []\r\n\
  #         }"
  #       sqlalchemy_uri: example://example-db.local
  #       tables: []

# -- Extra files to mount on `/app/pythonpath` as secrets
extraSecrets: {}

extraVolumes: 
  - name: v
    configMap:
      name: v
      items:
      - key: k
        path: v
  # - name: customConfig
  #   configMap:
  #     name: '{{ template "superset.fullname" . }}-custom-config'
  # - name: additionalSecret
  #   secret:
  #     secretName: my-secret
  #     defaultMode: 0600

extraVolumeMounts: 
  - name: v
    mountPath: /tmp
    subPath: v
    readOnly: true
  # - name: customConfig
  #   mountPath: /mnt/config
  #   readOnly: true
  # - name: additionalSecret:
  #   mountPath: /mnt/secret

# -- A dictionary of overrides to append at the end of superset_config.py - the name does not matter
# WARNING: the order is not guaranteed
# Files can be passed as helm --set-file configOverrides.my-override=my-file.py
configOverrides: 
  feature_flags: |
    import ast

    FEATURE_FLAGS = {
        "ALERT_REPORTS": True
    }

    SMTP_HOST = os.getenv("SMTP_HOST","localhost")
    SMTP_STARTTLS = ast.literal_eval(os.getenv("SMTP_STARTTLS", "True"))
    SMTP_SSL = ast.literal_eval(os.getenv("SMTP_SSL", "False"))
    SMTP_USER = os.getenv("SMTP_USER","superset")
    SMTP_PORT = os.getenv("SMTP_PORT",25)
    SMTP_PASSWORD = os.getenv("SMTP_PASSWORD","superset")
    SMTP_MAIL_FROM = os.getenv("SMTP_MAIL_FROM","superset@superset.com")
    ALERT_REPORTS_NOTIFICATION_DRY_RUN = ast.literal_eval(os.getenv("ALERT_REPORTS_NOTIFICATION_DRY_RUN","False"))

    SLACK_API_TOKEN = os.getenv("SLACK_API_TOKEN",None)
  celery_conf: |
    from celery.schedules import crontab

    class CeleryConfig:
      broker_url = f"redis://:{env('REDIS_PASS')}@{env('REDIS_HOST')}:{env('REDIS_PORT')}/0"
      imports = (
          "superset.sql_lab",
          "superset.tasks.cache",
          "superset.tasks.scheduler",
      )
      result_backend = f"redis://:{env('REDIS_PASS')}@{env('REDIS_HOST')}:{env('REDIS_PORT')}/0"
      worker_prefetch_multiplier = 10
      task_acks_late = True
      task_annotations = {
          "sql_lab.get_sql_results": {
              "rate_limit": "100/s",
          },
      }
      beat_schedule = {
          "reports.scheduler": {
              "task": "reports.scheduler",
              "schedule": crontab(minute="*", hour="*"),
          },
          "reports.prune_log": {
              "task": "reports.prune_log",
              'schedule': crontab(minute=0, hour=0),
          },
          'cache-warmup-hourly': {
              "task": "cache-warmup",
              "schedule": crontab(minute="*/30", hour="*"),
              "kwargs": {
                  "strategy_name": "top_n_dashboards",
                  "top_n": 10,
                  "since": "7 days ago",
              },
          }
      }

    CELERY_CONFIG = CeleryConfig
    
  cache_config: |
    from superset.tasks.types import ExecutorType

    FILTER_CACHE_REDIS_URL = f"{REDIS_BASE_URL}/{env('REDIS_DB', 2)}{REDIS_URL_PARAMS}"

    FILTER_STATE_CACHE_CONFIG = {
          'CACHE_TYPE': 'RedisCache',
          'CACHE_DEFAULT_TIMEOUT': 24*60*60,
          'CACHE_KEY_PREFIX': 'superset_filter_state',
          'CACHE_REDIS_URL': FILTER_CACHE_REDIS_URL,
    }

    THUMBNAIL_CACHE_REDIS_URL = f"{REDIS_BASE_URL}/{env('REDIS_DB', 3)}{REDIS_URL_PARAMS}"

    THUMBNAIL_CACHE_CONFIG = {
          'CACHE_TYPE': 'RedisCache',
          'CACHE_DEFAULT_TIMEOUT': 24*60*60*7,
          'CACHE_KEY_PREFIX': 'thumbnail_',
          'CACHE_REDIS_URL': THUMBNAIL_CACHE_REDIS_URL,
    }

    THUMBNAIL_SELENIUM_USER = "admin"
    THUMBNAIL_EXECUTE_AS = [ExecutorType.SELENIUM]
    
  reports: |
    EMAIL_PAGE_RENDER_WAIT = 60
    WEBDRIVER_BASEURL = "http://{{ template "superset.fullname" . }}:{{ .Values.service.port }}/"
    WEBDRIVER_BASEURL_USER_FRIENDLY = os.getenv("WEBDRIVER_BASEURL_USER_FRIENDLY", "https://www.example.com/")
    WEBDRIVER_TYPE= "chrome"
    WEBDRIVER_OPTION_ARGS = [
        "--force-device-scale-factor=2.0",
        "--high-dpi-support=2.0",
        "--headless",
        "--disable-gpu",
        "--disable-dev-shm-usage",
        # This is required because our process runs as root (in order to install pip packages)
        "--no-sandbox",
        "--disable-setuid-sandbox",
        "--disable-extensions",
    ]

    SCREENSHOT_LOCATE_WAIT = 100
    SCREENSHOT_LOAD_WAIT = 1200

    from superset.utils.urls import headless_url
    from superset.utils.machine_auth import MachineAuthProvider

    def auth_driver(driver, user):
        # Setting cookies requires doing a request first, but /login is redirected to oauth provider, and stuck there.
        driver.get(headless_url("/doesnotexist"))

        cookies = MachineAuthProvider.get_auth_cookies(user)

        for cookie_name, cookie_val in cookies.items():
            driver.add_cookie(dict(name=cookie_name, value=cookie_val))

        return driver

    WEBDRIVER_AUTH_FUNC = auth_driver
  extend_timeout: |
     # Extend timeout to allow long running queries.
     SUPERSET_WEBSERVER_TIMEOUT = 300
  enable_oauth: |
    import sys
    sys.path.append('/app/configs/')
    print (sys.path)
    from flask_appbuilder.security.manager import AUTH_OAUTH
    from custom_sso_security_manager import CustomSsoSecurityManager

    CUSTOM_SECURITY_MANAGER = CustomSsoSecurityManager # For managing User data after fetching from SSO app

    AUTH_TYPE = AUTH_OAUTH
    OAUTH_PROVIDERS = [
      {   
        'name':'azure',
          'token_key':'access_token', # Name of the token in the response of access_token_url
          'icon':'fa-address-card',   # Icon for the provider
          'remote_app': {
              'client_id':'***',  # Client Id (Identify Superset application)
              'client_secret':'***', # Secret for this Client Id (Identify Superset application)
              'client_kwargs':{
                  'scope': 'User.read name preferred_username email profile upn groups'               # Scope for the Authorization
              },
              'access_token_method':'POST',    # HTTP Method to call access_token_url
              #'access_token_params':{        # Additional parameters for calls to access_token_url
              #    'client_id':'myClientId'
              #},
              #'jwks_uri':'https://myAuthorizationServe/adfs/discovery/keys', # may be required to generate token
              #'access_token_headers':{    # Additional headers for calls to access_token_url
              #    'Authorization': 'Basic Base64EncodedClientIdAndSecret'
              #},
              'api_base_url':'https://login.microsoftonline.com/***/oauth2',
              'access_token_url':'https://login.microsoftonline.com/***/oauth2/token',
              'authorize_url':'https://login.microsoftonline.com/***/oauth2/authorize'
          }
      }
    ]
    AUTH_ROLE_PUBLIC = 'Public'
    AUTH_USER_REGISTRATION = True

  additional_config: |
    APP_NAME = '*** BI'
    APP_ICON = "/static/assets/images/logo.png"
    APP_ICON_WIDTH = 200
    LOGO_TARGET_PATH = '/' 
    LOGO_TOOLTIP = '*** BI'

    SUPERSET_WEBSERVER_PROTOCOL = os.getenv("SUPERSET_WEBSERVER_PROTOCOL","https")
    EMAIL_REPORTS_SUBJECT_PREFIX = os.getenv("EMAIL_REPORTS_SUBJECT_PREFIX","*** BI] ")
    ENABLE_PROXY_FIX = True
    RATELIMIT_ENABLED = False 
    AUTH_RATE_LIMITED = False
    DEFAULT_FEATURE_FLAGS: dict[str, bool] = {
        "CLIENT_CACHE": False,  # deprecated
        "DISABLE_DATASET_SOURCE_EDIT": False,  # deprecated
        "DRUID_JOINS": False,
        "DYNAMIC_PLUGINS": False,
        "DISABLE_LEGACY_DATASOURCE_EDITOR": True,
        "ENABLE_EXPLORE_JSON_CSRF_PROTECTION": False,  # deprecated
        "ENABLE_TEMPLATE_PROCESSING": False,
        "ENABLE_TEMPLATE_REMOVE_FILTERS": True,  # deprecated
        "ENABLE_JAVASCRIPT_CONTROLS": False,
        "KV_STORE": True,
        "PRESTO_EXPAND_DATA": False,
        "THUMBNAILS": True,
        "THUMBNAILS_SQLA_LISTENERS": True,
        "DASHBOARD_CACHE": False,  # deprecated
        "REMOVE_SLICE_LEVEL_LABEL_COLORS": False,  # deprecated
        "SHARE_QUERIES_VIA_KV_STORE": True,
        "TAGGING_SYSTEM": True,
        "SQLLAB_BACKEND_PERSISTENCE": True,
        "LISTVIEWS_DEFAULT_CARD_VIEW": False,
        "ESCAPE_MARKDOWN_HTML": False,
        "DASHBOARD_NATIVE_FILTERS": True,  # deprecated
        "DASHBOARD_CROSS_FILTERS": True,
        "DASHBOARD_NATIVE_FILTERS_SET": False,  # deprecated
        "DASHBOARD_FILTERS_EXPERIMENTAL": False,  # deprecated
        "DASHBOARD_VIRTUALIZATION": False,
        "GLOBAL_ASYNC_QUERIES": False,
        "VERSIONED_EXPORT": True,  # deprecated
        "EMBEDDED_SUPERSET": False,
        "ALERT_REPORTS": True,
        "DASHBOARD_RBAC": True,
        "ENABLE_EXPLORE_DRAG_AND_DROP": True,  # deprecated
        "ENABLE_ADVANCED_DATA_TYPES": True,
        "ALERTS_ATTACH_REPORTS": True,
        "ALLOW_FULL_CSV_EXPORT": True,
        "GENERIC_CHART_AXES": True,  # deprecated
        "ALLOW_ADHOC_SUBQUERY": True,
        "USE_ANALAGOUS_COLORS": False,
        "RLS_IN_SQLLAB": False,
        "CACHE_IMPERSONATION": False,
        "CACHE_QUERY_BY_USER": True,
        "EMBEDDABLE_CHARTS": True,
        "DRILL_TO_DETAIL": True,
        "DRILL_BY": True,
        "DATAPANEL_CLOSED_BY_DEFAULT": False,
        "HORIZONTAL_FILTER_BAR": True,
        "ESTIMATE_QUERY_COST": False,
        "SSH_TUNNELING": False,
        "AVOID_COLORS_COLLISION": True,
        "MENU_HIDE_USER_INFO": False,
        "ENABLE_SUPERSET_META_DB": True,
        "PLAYWRIGHT_REPORTS_AND_THUMBNAILS": False,
    }
  # extend_timeout: |
  #    # Extend timeout to allow long running queries.
  #    SUPERSET_WEBSERVER_TIMEOUT = ...
  # enable_oauth: |
  #   from flask_appbuilder.security.manager import (AUTH_DB, AUTH_OAUTH)
  #   AUTH_TYPE = AUTH_OAUTH
  #   OAUTH_PROVIDERS = [
  #       {
  #           "name": "google",
  #           "whitelist": [ os.getenv("OAUTH_WHITELIST_REGEX", "") ],
  #           "icon": "fa-google",
  #           "token_key": "access_token",
  #           "remote_app": {
  #               "client_id": os.environ.get("GOOGLE_KEY"),
  #               "client_secret": os.environ.get("GOOGLE_SECRET"),
  #               "api_base_url": "https://www.googleapis.com/oauth2/v2/",
  #               "client_kwargs": {"scope": "email profile"},
  #               "request_token_url": None,
  #               "access_token_url": "https://accounts.google.com/o/oauth2/token",
  #               "authorize_url": "https://accounts.google.com/o/oauth2/auth",
  #               "authorize_params": {"hd": os.getenv("OAUTH_HOME_DOMAIN", "")}
  #           }
  #       }
  #   ]
  #   # Map Authlib roles to superset roles
  #   AUTH_ROLE_ADMIN = 'Admin'
  #   AUTH_ROLE_PUBLIC = 'Public'
  #   # Will allow user self registration, allowing to create Flask users from Authorized User
  #   AUTH_USER_REGISTRATION = True
  #   # The default user self registration role
  #   AUTH_USER_REGISTRATION_ROLE = "Admin"
  # secret: |
  #   # Generate your own secret key for encryption. Use `openssl rand -base64 42` to generate a good key
  #   SECRET_KEY = 'CHANGE_ME_TO_A_COMPLEX_RANDOM_SECRET'

# -- Same as above but the values are files
configOverridesFiles: {}
  # extend_timeout: extend_timeout.py
  # enable_oauth: enable_oauth.py

configMountPath: "/app/pythonpath"

extraConfigMountPath: "/app/configs"

image:
  repository: apachesuperset.docker.scarf.sh/apache/superset
  tag: 4.0.0-dev
  pullPolicy: Always

imagePullSecrets: []

initImage:
  repository: apache/superset
  tag: dockerize
  pullPolicy: Always

service:
  type: ClusterIP
  port: 8088
  annotations: {}
    # cloud.google.com/load-balancer-type: "Internal"
  loadBalancerIP: ~
  nodePort:
    # -- (int)
    http: nil

ingress:
  enabled: true
  ingressClassName: "public"
  annotations: 
    kubernetes.io/tls-acme: "true"
    ## Extend timeout to allow long running queries.
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "300"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "300"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "300"
    nginx.ingress.kubernetes.io/client_max_body_size : "50m"
    nginx.ingress.kubernetes.io/client-body-buffer-size: "50m" 
    nginx.ingress.kubernetes.io/proxy-body-size: "50m"
    # kubernetes.io/tls-acme: "true"
    ## Extend timeout to allow long running queries.
    # nginx.ingress.kubernetes.io/proxy-connect-timeout: "300"
    # nginx.ingress.kubernetes.io/proxy-read-timeout: "300"
    # nginx.ingress.kubernetes.io/proxy-send-timeout: "300"
  path: /
  pathType: ImplementationSpecific
  hosts:
    - ***.com
  tls: 
    - hosts:
        - ***.com
      secretName: ***.com-tls
  extraHostsRaw: []
  #  - secretName: chart-example-tls
  #    hosts:
  #      - chart-example.local

resources: {}
  # We usually recommend not to specify default resources and to leave this as a conscious
  # choice for the user. This also increases chances charts run on environments with little
  # resources, such as Minikube. If you do want to specify resources, uncomment the following
  # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
  # The limits below will apply to all Superset components. To set individual resource limitations refer to the pod specific values below.
  # The pod specific values will overwrite anything that is set here.
  # limits:
  #   cpu: 100m
  #   memory: 128Mi
  # requests:
  #   cpu: 100m
  #   memory: 128Mi

# -- Custom hostAliases for all superset pods
## https://kubernetes.io/docs/tasks/network/customize-hosts-file-for-pods/
hostAliases: []
# - hostnames:
#   - nodns.my.lan
#   ip: 18.27.36.45

# Superset node configuration
supersetNode:
  replicaCount: 1
  autoscaling:
    enabled: false
    minReplicas: 1
    maxReplicas: 100
    targetCPUUtilizationPercentage: 80
    # targetMemoryUtilizationPercentage: 80
  # -- Sets the [pod disruption budget](https://kubernetes.io/docs/tasks/run-application/configure-pdb/) for supersetNode pods
  podDisruptionBudget:
    # -- Whether the pod disruption budget should be created
    enabled: false
    # -- If set, maxUnavailable must not be set - see https://kubernetes.io/docs/tasks/run-application/configure-pdb/#specifying-a-poddisruptionbudget
    minAvailable: 1
    # -- If set, minAvailable must not be set - see https://kubernetes.io/docs/tasks/run-application/configure-pdb/#specifying-a-poddisruptionbudget
    maxUnavailable: 1

  # -- Startup command
  # @default -- See `values.yaml`
  command:
    - "/bin/sh"
    - "-c"
    - ". {{ .Values.configMountPath }}/superset_bootstrap.sh; /usr/bin/run-server.sh"
  connections:
    # -- Change in case of bringing your own redis and then also set redis.enabled:false
    redis_host: 'redis-***'
    redis_port: "6379"
    redis_user: ""
    redis_password: ***
    redis_cache_db: "1"
    redis_celery_db: "0"
    # Or SSL port is usually 6380
    # Update following for using Redis with SSL
    redis_ssl:
      enabled: false
      ssl_cert_reqs: CERT_NONE
    # You need to change below configuration incase bringing own PostgresSQL instance and also set postgresql.enabled:false
    db_host: 'postgresql-***'
    db_port: "5432"
    db_user: ***
    db_pass: "***"
    db_name: ***
  env: {}
  # -- If true, forces deployment to reload on each upgrade
  forceReload: true
  # -- Init containers
  # @default -- a container waiting for postgres
  initContainers:
    - name: wait-for-postgres
      image: "{{ .Values.initImage.repository }}:{{ .Values.initImage.tag }}"
      imagePullPolicy: "{{ .Values.initImage.pullPolicy }}"
      envFrom:
        - secretRef:
            name: "{{ tpl .Values.envFromSecret . }}"
      command:
        - /bin/sh
        - -c
        - dockerize -wait "tcp://$DB_HOST:$DB_PORT" -timeout 120s

  # -- Launch additional containers into supersetNode pod
  extraContainers: []
  # -- Annotations to be added to supersetNode deployment
  deploymentAnnotations: {}
  # -- Labels to be added to supersetNode deployment
  deploymentLabels: {}
  # -- Affinity to be added to supersetNode deployment
  affinity: {}
  # -- TopologySpreadConstrains to be added to supersetNode deployments
  topologySpreadConstraints: []
  # -- Annotations to be added to supersetNode pods
  podAnnotations: {}
  # -- Labels to be added to supersetNode pods
  podLabels: {}
  startupProbe:
    httpGet:
      path: /health
      port: http
    initialDelaySeconds: 15
    timeoutSeconds: 1
    failureThreshold: 60
    periodSeconds: 5
    successThreshold: 1
  livenessProbe:
    httpGet:
      path: /health
      port: http
    initialDelaySeconds: 15
    timeoutSeconds: 1
    failureThreshold: 3
    periodSeconds: 15
    successThreshold: 1
  readinessProbe:
    httpGet:
      path: /health
      port: http
    initialDelaySeconds: 15
    timeoutSeconds: 1
    failureThreshold: 3
    periodSeconds: 15
    successThreshold: 1
  # -- Resource settings for the supersetNode pods - these settings overwrite might existing values from the global resources object defined above.
  resources: {}
    # limits:
    #  cpu: 100m
    #  memory: 128Mi
    # requests:
    #  cpu: 100m
    #  memory: 128Mi
  podSecurityContext: {}
  containerSecurityContext: {}
  strategy: {}
    # type: RollingUpdate
    # rollingUpdate:
    #   maxSurge: 25%
    #   maxUnavailable: 25%

# Superset Celery worker configuration
supersetWorker:
  replicaCount: 5
  autoscaling:
    enabled: false
    minReplicas: 1
    maxReplicas: 100
    targetCPUUtilizationPercentage: 80
    # targetMemoryUtilizationPercentage: 80
  # -- Sets the [pod disruption budget](https://kubernetes.io/docs/tasks/run-application/configure-pdb/) for supersetWorker pods
  podDisruptionBudget:
    # -- Whether the pod disruption budget should be created
    enabled: false
    # -- If set, maxUnavailable must not be set - see https://kubernetes.io/docs/tasks/run-application/configure-pdb/#specifying-a-poddisruptionbudget
    minAvailable: 1
    # -- If set, minAvailable must not be set - see https://kubernetes.io/docs/tasks/run-application/configure-pdb/#specifying-a-poddisruptionbudget
    maxUnavailable: 1
  # -- Worker startup command
  # @default -- a `celery worker` command
  command:
    - "/bin/sh"
    - "-c"
    - ". {{ .Values.configMountPath }}/superset_bootstrap.sh; celery --app=superset.tasks.celery_app:app worker"
  # -- If true, forces deployment to reload on each upgrade
  forceReload: true
  # -- Init container
  # @default -- a container waiting for postgres and redis
  initContainers:
    - name: wait-for-postgres-redis
      image: "{{ .Values.initImage.repository }}:{{ .Values.initImage.tag }}"
      imagePullPolicy: "{{ .Values.initImage.pullPolicy }}"
      envFrom:
        - secretRef:
            name: "{{ tpl .Values.envFromSecret . }}"
      command:
        - /bin/sh
        - -c
        - dockerize -wait "tcp://$DB_HOST:$DB_PORT" -wait "tcp://$REDIS_HOST:$REDIS_PORT" -timeout 120s
  # -- Launch additional containers into supersetWorker pod
  extraContainers: []
  # -- Annotations to be added to supersetWorker deployment
  deploymentAnnotations: {}
  # -- Labels to be added to supersetWorker deployment
  deploymentLabels: {}
  # -- Affinity to be added to supersetWorker deployment
  affinity: {}
  # -- TopologySpreadConstrains to be added to supersetWorker deployments
  topologySpreadConstraints: []
  # -- Annotations to be added to supersetWorker pods
  podAnnotations: {}
  # -- Labels to be added to supersetWorker pods
  podLabels: {}
  # -- Resource settings for the supersetWorker pods - these settings overwrite might existing values from the global resources object defined above.
  resources: {}
    # limits:
    #  cpu: 100m
    #  memory: 128Mi
    # requests:
    #  cpu: 100m
    #  memory: 128Mi
  podSecurityContext: {}
  containerSecurityContext: {}
  strategy: {}
    # type: RollingUpdate
    # rollingUpdate:
    #   maxSurge: 25%
    #   maxUnavailable: 25%
  livenessProbe:
    exec:
      # -- Liveness probe command
      # @default -- a `celery inspect ping` command
      command:
        - sh
        - -c
        - celery -A superset.tasks.celery_app:app inspect ping -d celery@$HOSTNAME
    initialDelaySeconds: 120
    timeoutSeconds: 60
    failureThreshold: 3
    periodSeconds: 60
    successThreshold: 1
  # -- No startup/readiness probes by default since we don't really care about its startup time (it doesn't serve traffic)
  startupProbe: {}
  # -- No startup/readiness probes by default since we don't really care about its startup time (it doesn't serve traffic)
  readinessProbe: {}

# Superset beat configuration (to trigger scheduled jobs like reports)
supersetCeleryBeat:
  # -- This is only required if you intend to use alerts and reports
  enabled: true
  # -- Sets the [pod disruption budget](https://kubernetes.io/docs/tasks/run-application/configure-pdb/) for supersetCeleryBeat pods
  podDisruptionBudget:
    # -- Whether the pod disruption budget should be created
    enabled: false
    # -- If set, maxUnavailable must not be set - see https://kubernetes.io/docs/tasks/run-application/configure-pdb/#specifying-a-poddisruptionbudget
    minAvailable: 1
    # -- If set, minAvailable must not be set - see https://kubernetes.io/docs/tasks/run-application/configure-pdb/#specifying-a-poddisruptionbudget
    maxUnavailable: 1
  # -- Command
  # @default -- a `celery beat` command
  command:
    - "/bin/sh"
    - "-c"
    - ". {{ .Values.configMountPath }}/superset_bootstrap.sh; celery --app=superset.tasks.celery_app:app beat --pidfile /tmp/celerybeat.pid --schedule /tmp/celerybeat-schedule"
  # -- If true, forces deployment to reload on each upgrade
  forceReload: false
  # -- List of init containers
  # @default -- a container waiting for postgres
  initContainers:
    - name: wait-for-postgres-redis
      image: "{{ .Values.initImage.repository }}:{{ .Values.initImage.tag }}"
      imagePullPolicy: "{{ .Values.initImage.pullPolicy }}"
      envFrom:
        - secretRef:
            name: "{{ tpl .Values.envFromSecret . }}"
      command:
        - /bin/sh
        - -c
        - dockerize -wait "tcp://$DB_HOST:$DB_PORT" -wait "tcp://$REDIS_HOST:$REDIS_PORT" -timeout 120s
  # -- Launch additional containers into supersetCeleryBeat pods
  extraContainers: []
  # -- Annotations to be added to supersetCeleryBeat deployment
  deploymentAnnotations: {}
  # -- Affinity to be added to supersetCeleryBeat deployment
  affinity: {}
  # -- TopologySpreadConstrains to be added to supersetCeleryBeat deployments
  topologySpreadConstraints: []
  # -- Annotations to be added to supersetCeleryBeat pods
  podAnnotations: {}
  # -- Labels to be added to supersetCeleryBeat pods
  podLabels: {}
  # -- Resource settings for the CeleryBeat pods - these settings overwrite might existing values from the global resources object defined above.
  resources: {}
    # limits:
    #  cpu: 100m
    #  memory: 128Mi
    # requests:
    #  cpu: 100m
    #  memory: 128Mi
  podSecurityContext: {}
  containerSecurityContext: {}

supersetCeleryFlower:
  # -- Enables a Celery flower deployment (management UI to monitor celery jobs)
  # WARNING: on superset 1.x, this requires a Superset image that has `flower<1.0.0` installed (which is NOT the case of the default images)
  # flower>=1.0.0 requires Celery 5+ which Superset 1.5 does not support
  enabled: false
  replicaCount: 1
  # -- Sets the [pod disruption budget](https://kubernetes.io/docs/tasks/run-application/configure-pdb/) for supersetCeleryFlower pods
  podDisruptionBudget:
    # -- Whether the pod disruption budget should be created
    enabled: false
    # -- If set, maxUnavailable must not be set - see https://kubernetes.io/docs/tasks/run-application/configure-pdb/#specifying-a-poddisruptionbudget
    minAvailable: 1
    # -- If set, minAvailable must not be set - see https://kubernetes.io/docs/tasks/run-application/configure-pdb/#specifying-a-poddisruptionbudget
    maxUnavailable: 1
  # -- Command
  # @default -- a `celery flower` command
  command:
    - "/bin/sh"
    - "-c"
    - "pip install redis authlib flower && celery --app=superset.tasks.celery_app:app flower"
  service:
    type: ClusterIP
    annotations: {}
    loadBalancerIP: ~
    port: 5555
    nodePort:
      # -- (int)
      http: nil
  startupProbe:
    httpGet:
      path: /api/workers
      port: flower
    initialDelaySeconds: 5
    timeoutSeconds: 1
    failureThreshold: 60
    periodSeconds: 5
    successThreshold: 1
  livenessProbe:
    httpGet:
      path: /api/workers
      port: flower
    initialDelaySeconds: 5
    timeoutSeconds: 1
    failureThreshold: 3
    periodSeconds: 5
    successThreshold: 1
  readinessProbe:
    httpGet:
      path: /api/workers
      port: flower
    initialDelaySeconds: 5
    timeoutSeconds: 1
    failureThreshold: 3
    periodSeconds: 5
    successThreshold: 1
  # -- List of init containers
  # @default -- a container waiting for postgres and redis
  initContainers:
    - name: wait-for-postgres-redis
      image: "{{ .Values.initImage.repository }}:{{ .Values.initImage.tag }}"
      imagePullPolicy: "{{ .Values.initImage.pullPolicy }}"
      envFrom:
        - secretRef:
            name: "{{ tpl .Values.envFromSecret . }}"
      command:
        - /bin/sh
        - -c
        - dockerize -wait "tcp://$DB_HOST:$DB_PORT" -wait "tcp://$REDIS_HOST:$REDIS_PORT" -timeout 120s
  # -- Launch additional containers into supersetCeleryFlower pods
  extraContainers: []
  # -- Annotations to be added to supersetCeleryFlower deployment
  deploymentAnnotations: {}
  # -- Affinity to be added to supersetCeleryFlower deployment
  affinity: {}
  # -- TopologySpreadConstrains to be added to supersetCeleryFlower deployments
  topologySpreadConstraints: []
  # -- Annotations to be added to supersetCeleryFlower pods
  podAnnotations: {}
  # -- Labels to be added to supersetCeleryFlower pods
  podLabels: {}
  # -- Resource settings for the CeleryBeat pods - these settings overwrite might existing values from the global resources object defined above.
  resources: {}
    # limits:
    #  cpu: 100m
    #  memory: 128Mi
    # requests:
    #  cpu: 100m
    #  memory: 128Mi
  podSecurityContext: {}
  containerSecurityContext: {}

supersetWebsockets:
  # -- This is only required if you intend to use `GLOBAL_ASYNC_QUERIES` in `ws` mode
  # see https://github.com/apache/superset/blob/master/CONTRIBUTING.md#async-chart-queries
  enabled: false
  replicaCount: 1
  # -- Sets the [pod disruption budget](https://kubernetes.io/docs/tasks/run-application/configure-pdb/) for supersetWebsockets pods
  podDisruptionBudget:
    # -- Whether the pod disruption budget should be created
    enabled: false
    # -- If set, maxUnavailable must not be set - see https://kubernetes.io/docs/tasks/run-application/configure-pdb/#specifying-a-poddisruptionbudget
    minAvailable: 1
    # -- If set, minAvailable must not be set - see https://kubernetes.io/docs/tasks/run-application/configure-pdb/#specifying-a-poddisruptionbudget
    maxUnavailable: 1
  ingress:
    path: /ws
    pathType: Prefix
  image:
    # -- There is no official image (yet), this one is community-supported
    repository: oneacrefund/superset-websocket
    tag: latest
    pullPolicy: IfNotPresent
  # -- The config.json to pass to the server, see https://github.com/apache/superset/tree/master/superset-websocket
  # Note that the configuration can also read from environment variables (which will have priority), see https://github.com/apache/superset/blob/master/superset-websocket/src/config.ts for a list of supported variables
  # @default -- see `values.yaml`
  config:
    {
      "port": 8080,
      "logLevel": "debug",
      "logToFile": false,
      "logFilename": "app.log",
      "statsd": { "host": "127.0.0.1", "port": 8125, "globalTags": [] },
      "redis":
        {
          "port": 6379,
          "host": "127.0.0.1",
          "password": "",
          "db": 0,
          "ssl": false,
        },
      "redisStreamPrefix": "async-events-",
      "jwtSecret": "CHANGE-ME",
      "jwtCookieName": "async-token",
    }
  service:
    type: ClusterIP
    annotations: {}
    loadBalancerIP: ~
    port: 8080
    nodePort:
      # -- (int)
      http: nil
  command: []
  resources: {}
  # -- Launch additional containers into supersetWebsockets pods
  extraContainers: []
  deploymentAnnotations: {}
  # -- Affinity to be added to supersetWebsockets deployment
  affinity: {}
  # -- TopologySpreadConstrains to be added to supersetWebsockets deployments
  topologySpreadConstraints: []
  podAnnotations: {}
  podLabels: {}
  strategy: {}
  podSecurityContext: {}
  containerSecurityContext: {}
  startupProbe:
    httpGet:
      path: /health
      port: ws
    initialDelaySeconds: 5
    timeoutSeconds: 1
    failureThreshold: 60
    periodSeconds: 5
    successThreshold: 1
  livenessProbe:
    httpGet:
      path: /health
      port: ws
    initialDelaySeconds: 5
    timeoutSeconds: 1
    failureThreshold: 3
    periodSeconds: 5
    successThreshold: 1
  readinessProbe:
    httpGet:
      path: /health
      port: ws
    initialDelaySeconds: 5
    timeoutSeconds: 1
    failureThreshold: 3
    periodSeconds: 5
    successThreshold: 1

init:
  # Configure resources
  # Warning: fab command consumes a lot of ram and can
  # cause the process to be killed due to OOM if it exceeds limit
  # Make sure you are giving a strong password for the admin user creation( else make sure you are changing after setup)
  # Also change the admin email to your own custom email.
  resources: {}
    # limits:
    #   cpu:
    #   memory:
    # requests:
    #   cpu:
    #   memory:
  # -- Command
  # @default -- a `superset_init.sh` command
  command:
    - "/bin/sh"
    - "-c"
    - ". {{ .Values.configMountPath }}/superset_bootstrap.sh; . {{ .Values.configMountPath }}/superset_init.sh"
  enabled: false
  jobAnnotations:
    "helm.sh/hook": post-install,post-upgrade
    "helm.sh/hook-delete-policy": "before-hook-creation"
  loadExamples: false
  createAdmin: false
  adminUser:
    username: ***
    firstname: ***
    lastname: ***
    email: ***
    password: ***
  # -- List of initContainers
  # @default -- a container waiting for postgres
  initContainers:
    - name: wait-for-postgres
      image: "{{ .Values.initImage.repository }}:{{ .Values.initImage.tag }}"
      imagePullPolicy: "{{ .Values.initImage.pullPolicy }}"
      envFrom:
        - secretRef:
            name: "{{ tpl .Values.envFromSecret . }}"
      command:
        - /bin/sh
        - -c
        - dockerize -wait "tcp://$DB_HOST:$DB_PORT" -timeout 120s
  # -- A Superset init script
  # @default -- a script to create admin user and initialize roles
  initscript: |-
    #!/bin/sh
    pip install holidays==0.23
    set -eu
    echo "Upgrading DB schema..."
    superset db upgrade
    echo "Initializing roles..."
    superset init
    {{ if .Values.init.createAdmin }}
    echo "Creating admin user..."
    superset fab create-admin \
                    --username {{ .Values.init.adminUser.username }} \
                    --firstname {{ .Values.init.adminUser.firstname }} \
                    --lastname {{ .Values.init.adminUser.lastname }} \
                    --email {{ .Values.init.adminUser.email }} \
                    --password {{ .Values.init.adminUser.password }} \
                    || true
    {{- end }}
    {{ if .Values.init.loadExamples }}
    echo "Loading examples..."
    superset load_examples
    {{- end }}
    if [ -f "{{ .Values.extraConfigMountPath }}/import_datasources.yaml" ]; then
      echo "Importing database connections.... "
      superset import_datasources -p {{ .Values.extraConfigMountPath }}/import_datasources.yaml
    fi
  # -- Launch additional containers into init job pod
  extraContainers: []
  ## Annotations to be added to init job pods
  podAnnotations: {}
  podSecurityContext: {}
  containerSecurityContext: {}
  ## Tolerations to be added to init job pods
  tolerations: []
  ## Affinity to be added to init job pods
  affinity: {}
  # -- TopologySpreadConstrains to be added to init job
  topologySpreadConstraints: []

# -- Configuration values for the postgresql dependency.
# ref: https://github.com/bitnami/charts/tree/main/bitnami/postgresql
# @default -- see `values.yaml`
postgresql:
  ##
  ## Use the PostgreSQL chart dependency.
  ## Set to false if bringing your own PostgreSQL.
  enabled: false

  ## Authentication parameters
  auth:
    ## The name of an existing secret that contains the postgres password.
    existingSecret:
    ## PostgreSQL name for a custom user to create
    username: ***
    ## PostgreSQL password for the custom user to create. Ignored if `auth.existingSecret` with key `password` is provided
    password: ***
    ## PostgreSQL name for a custom database to create
    database: ***

  image:
    tag: "14.6.0-debian-11-r13"

  ## PostgreSQL Primary parameters
  primary:
    ##
    ## Persistent Volume Storage configuration.
    ## ref: https://kubernetes.io/docs/user-guide/persistent-volumes
    persistence:
      ##
      ## Enable PostgreSQL persistence using Persistent Volume Claims.
      enabled: true
      ##
      ## Persistent class
      # storageClass: classname
      ##
      ## Access modes:
      accessModes:
        - ReadWriteOnce
    ## PostgreSQL port
    service:
      ports:
        postgresql: "5432"

# -- Configuration values for the Redis dependency.
# ref: https://github.com/bitnami/charts/blob/master/bitnami/redis
# More documentation can be found here: https://artifacthub.io/packages/helm/bitnami/redis
# @default -- see `values.yaml`
redis:
  ##
  ## Use the redis chart dependency.
  ##
  ## If you are bringing your own redis, you can set the host in supersetNode.connections.redis_host
  ##
  ## Set to false if bringing your own redis.
  enabled: false
  ##
  ## Set architecture to standalone/replication
  architecture: standalone
  ##
  ## Auth configuration:
  ##
  auth:
    ## Enable password authentication
    enabled: false
    ## The name of an existing secret that contains the redis password.
    existingSecret: ""
    ## Name of the key containing the secret.
    existingSecretKey: ""
    ## Redis password
    password: ***
  ##
  ## Master configuration
  ##
  master:
    ##
    ## Image configuration
    # image:
    ##
    ## docker registry secret names (list)
    # pullSecrets: nil
    ##
    ## Configure persistance
    persistence:
      ##
      ## Use a PVC to persist data.
      enabled: false
      ##
      ## Persistent class
      # storageClass: classname
      ##
      ## Access mode:
      accessModes:
        - ReadWriteOnce

nodeSelector: {}

tolerations: []

affinity: {}

# -- TopologySpreadConstrains to be added to all deployments
topologySpreadConstraints: []

Screenshots/recordings

Logs from the worker pod

logging was configured successfully
2024-05-13 02:26:25,220:INFO:superset.utils.logging_configurator:logging was configured successfully
2024-05-13 02:26:25,222:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
2024-05-13 02:26:26,655:DEBUG:celery.utils.functional:
def chord(self, header, body, partial_args=0, interval=1, countdown=2, max_retries=3, eager=4, **kwargs):
    return 1
                                                                                                                                                      2024-05-13 02:26:26,655:DEBUG:celery.utils.functional:
def accumulate(self, *args, **kwargs):
    return 1

2024-05-13 02:26:26,656:DEBUG:celery.utils.functional:
def cache_chart_thumbnail(current_user, chart_id, force=0, window_size=1, thumb_size=2):
    return 1

2024-05-13 02:26:26,656:DEBUG:celery.utils.functional:
def execute(self, report_schedule_id):
    return 1
                                                                                                                                                      2024-05-13 02:26:26,656:DEBUG:celery.utils.functional:
def unlock_chord(self, group_id, callback, interval=0, max_retries=1, result=2, Result=3, GroupResult=4, result_from_tuple=5, **kwargs):
    return 1

2024-05-13 02:26:26,656:DEBUG:celery.utils.functional:
def backend_cleanup():
    return 1

2024-05-13 02:26:26,657:DEBUG:celery.utils.functional:
def cache_dashboard_thumbnail(current_user, dashboard_id, force=0, thumb_size=1):
    return 1

2024-05-13 02:26:26,657:DEBUG:celery.utils.functional:
def xmap(task, it):
    return 1
                                                                                                                                                      2024-05-13 02:26:26,657:DEBUG:celery.utils.functional:
def prune_log():
    return 1

2024-05-13 02:26:26,657:DEBUG:celery.utils.functional:
def xstarmap(task, it):
    return 1

2024-05-13 02:26:26,657:DEBUG:celery.utils.functional:
def cache_warmup(strategy_name, *args, **kwargs):
    return 1

2024-05-13 02:26:26,658:DEBUG:celery.utils.functional:                                                                                                def get_sql_results(query_id, rendered_query, return_results=0, store_results=1, username=2, start_time=3, expand_data=4, log_params=5):
    return 1

2024-05-13 02:26:26,658:DEBUG:celery.utils.functional:
def fetch_url(data, headers):
    return 1
                                                                                                                                                      2024-05-13 02:26:26,658:DEBUG:celery.utils.functional:
def chunks(task, it, n):
    return 1

2024-05-13 02:26:26,658:DEBUG:celery.utils.functional:
def group(self, tasks, result, group_id, partial_args, add_to_parent=0):
    return 1

2024-05-13 02:26:26,659:DEBUG:celery.utils.functional:
def scheduler():
    return 1
                                                                                                                                                      2024-05-13 02:26:26,659:DEBUG:celery.utils.functional:
def chain(*args, **kwargs):
    return 1

/usr/local/lib/python3.10/site-packages/celery/platforms.py:829: SecurityWarning: You're running the worker with superuser privileges: this is
absolutely not recommended!

Please specify a different user using the --uid option.

User information: uid=0 euid=0 gid=0 egid=0

  warnings.warn(SecurityWarning(ROOT_DISCOURAGED.format(
['/usr/local/bin', '/app/pythonpath', '/usr/local/lib/python310.zip', '/usr/local/lib/python3.10', '/usr/local/lib/python3.10/lib-dynload', '/usr/local/lib/python3.10/site-packages', '/app', '/app/configs/']
Loaded your LOCAL configuration at [/app/pythonpath/superset_config.py]

 -------------- celery@superset-worker-54bc9dffd4-skkk8 v5.3.6 (emerald-rush)
--- ***** -----
-- ******* ---- Linux-5.15.0-102-generic-x86_64-with-glibc2.36 2024-05-13 02:26:26
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app:         __main__:0x7f934c950a90
- ** ---------- .> transport:   redis://:**@redis-master:6379/0
- ** ---------- .> results:     redis://:**@redis-master:6379/0
- *** --- * --- .> concurrency: 24 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery

                                                                                                                                                      [2024-05-13 02:26:29,760: WARNING/MainProcess] /usr/local/lib/python3.10/site-packages/celery/worker/consumer/consumer.py:507: CPendingDeprecationWarning: The broker_connection_retry configuration setting will no longer determine
whether broker connection retries are made during startup in Celery 6.0 and above.
If you wish to retain the existing behavior for retrying connections on startup,
you should set broker_connection_retry_on_startup to True.
  warnings.warn(

[2024-05-13 02:26:29,768: WARNING/MainProcess] /usr/local/lib/python3.10/site-packages/celery/worker/consumer/consumer.py:507: CPendingDeprecationWarning: The broker_connection_retry configuration setting will no longer determine
whether broker connection retries are made during startup in Celery 6.0 and above.
If you wish to retain the existing behavior for retrying connections on startup,
you should set broker_connection_retry_on_startup to True.
  warnings.warn(

Superset version

4.0.0

Python version

3.10

Node version

16

Browser

Chrome

Additional context

No response

Checklist

• I have searched Superset docs and Slack and didn't find a solution to my problem.
• I have searched the GitHub issue tracker and didn't find a similar bug report.
• I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

[dosubot]

To address the issue of worker instances continuously restarting after upgrading to Superset version 4.0.0, consider the following steps:

1. Compatibility Check: Ensure all components, including Python 3.11 and Node 16, are compatible with Superset 4.0.0. While Python 3.11 is compatible, Node.js version 18.20.1 is recommended for the best development experience with Superset's frontend.

2. Review Custom Bootstrap Script: Inspect your custom bootstrap script for any conflicts or deprecated packages that might not be compatible with Superset 4.0.0.

3. Worker Configuration: Verify the command used to start worker instances in your helm chart is correct for Superset 4.0.0, focusing on the path to superset_bootstrap.sh and the Celery command.

4. Liveness and Readiness Probes: Ensure the configuration for these probes in the supersetWorker section is correctly set up for Superset 4.0.0, as incorrect configurations can lead to continuous restarts.

5. Environment Variables and Configurations: Check that all necessary environment variables and configurations are correctly set up, especially those derived from secrets or config maps.

6. Log Analysis: Review the logs of the restarting worker instances for errors or warnings that could indicate the cause of the restarts.

7. Dependencies and Package Versions: Make sure all dependencies specified in your bootstrap script or helm chart are compatible with Superset 4.0.0.

8. Resource Limits: Review and adjust the resource limits and requests for the worker instances if they are being terminated due to resource constraints.

9. Review Helm Chart Annotations and Checksums: Update checksums for configurations if changes are made, ensuring Kubernetes recognizes and applies these changes.

10. Security Concerns with runAsUser: Running containers as root (UID 0) is discouraged due to security risks. Consider running containers with a non-root user.

These steps should help identify and resolve the issue with worker instances restarting. If the problem persists, further investigation into Superset's change logs for version 4.0.0 may be necessary to identify any breaking changes or additional migration steps required.

Details

Sources

• supersetWorker Liveness Probes always fail causing restarts
• helm/superset/values.yaml
• helm/superset/templates/deployment-worker.yaml
• helm/superset/templates/deployment.yaml
• CHANGELOG/4.0.0.md
• pyproject.toml
• superset-frontend/.nvmrc

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[fzhan]

It's an issue with Liveness probe :

Liveness probe failed: ['/usr/local/bin', '/app/pythonpath', '/usr/local/lib/python310.zip', '/usr/local/lib/python3.10', '/usr/local/lib/python3.10/lib-dynload', '/usr/local/lib/python3.10/site-packages', '/app', '/app/configs/'] Loaded your LOCAL configuration at [/app/pythonpath/superset_config.py] logging was configured successfully 2024-05-13 02:49:48,445:INFO:superset.utils.logging_configurator:logging was configured successfully 2024-05-13 02:49:48,447:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'> Error: No nodes replied within time constraint

[fzhan]

After running 'celery -A superset.tasks.celery_app:app inspect ping -d celery@$HOSTNAME' manually in worker pod, response was Error:

root@superset-worker-54fcfd4b-bzpzd:/app# celery -A superset.tasks.celery_app:app inspect ping -d celery@$HOSTNAME
['/usr/local/bin', '/app/pythonpath', '/usr/local/lib/python310.zip', '/usr/local/lib/python3.10', '/usr/local/lib/python3.10/lib-dynload', '/usr/local/lib/python3.10/site-packages', '/app', '/app/configs/']
Loaded your LOCAL configuration at [/app/pythonpath/superset_config.py]
logging was configured successfully
2024-05-13 02:58:06,359:INFO:superset.utils.logging_configurator:logging was configured successfully
2024-05-13 02:58:06,361:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
Error: No nodes replied within time constraint

[fzhan]

After done some digging, it appears to be related to the Celery version issue with Redis. #28471

[fzhan]

After upgrade to celery 5.4, flower can see workers listed, but the ping for liveness within worker still fails with no nodes replied.

[fzhan]

LivenessProbe still fails with 'Error: No nodes replied within time constraint' as result from "celery -A superset.tasks.celery_app:app inspect ping -d celery@$HOSTNAME"