Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CeleryKubernetesExecutor jobs are not queued. #293

Closed
jensgrnb opened this issue Jul 9, 2021 · 2 comments
Closed

CeleryKubernetesExecutor jobs are not queued. #293

jensgrnb opened this issue Jul 9, 2021 · 2 comments
Labels
kind/question kind - user questions

Comments

@jensgrnb
Copy link

jensgrnb commented Jul 9, 2021

What is your question?

I'm trying to get the CeleryKubernetesExecutor up and running on our airflow cluster. I'm upgrading from a CeleryExecutor where the jobs are running fine, but when I simply upgrade the executor without adding anything, no jobs are queued at all.

task instance details gives me

image

(the dependencies error is present when I only use CeleryExecutor as well, but that doesn't stop the jobs from running fine.)
the queue id for this job is default and I was exepecting that this upgrade of executor would not affect anything at all, but instead everything grinds to a halt.

Anyone experiencing similar issues? I'm on latest helm chart 8.4.0 and latest airflow 2-1-1-python3.8.

I've looked around for error logs on both the scheduler side and the worker side, but there's just nothing.

@jensgrnb jensgrnb added the kind/question kind - user questions label Jul 9, 2021
@jensgrnb jensgrnb changed the title CeleryKubernetesExecutor jobs are not queued on default queue id. CeleryKubernetesExecutor jobs are not queued. Jul 9, 2021
@thesuperzapper
Copy link
Member

@jensgrnb this issue is caused by airflow 2.1.1, not the chart.

This apache/airflow#16700, has addressed the issue, and will be included in airflow 2.1.2.

@thesuperzapper thesuperzapper added this to Waiting for Response in Issue Triage and PR Tracking Jul 12, 2021
@thesuperzapper thesuperzapper moved this from Waiting for Response to To Investigate in Issue Triage and PR Tracking Jul 12, 2021
@jensgrnb
Copy link
Author

Okay thanks for the reference! I'll be waiting eagerly for 2.1.2 then) Dodging meetings etc.

Issue Triage and PR Tracking automation moved this from To Investigate to Done Aug 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/question kind - user questions
Development

No branches or pull requests

2 participants