Silent failure when running airflow db init
#18408
Unanswered
ziplokk1
asked this question in
Show and tell
Replies: 3 comments 10 replies
-
|
Not sure if I understand the issue correctly, but the error "invalid command 'db'" suggests to me you're running |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
I'm running airflow 2.0.2. |
Beta Was this translation helpful? Give feedback.
-
|
Maybe for some reason you installed Airlfow 1.10 over Airflow 2? For example if you try to install backport-providers, they are Airflow 1.10-only and when you install them from requirements, they will DOWNGRADE your airflow to 1.10.... |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
I'm running airflow 2.0.2 although your comment got me curious and all my provider versions are 1.0.x. I'm wondering if there's some compatibility issues. I'm going to update the providers and see if that remedies the issue. |
Beta Was this translation helpful? Give feedback.
-
|
I think I saw it before. I can't remember the details, but I am quite sure that one of your dags likely has some code that it should not suppose to run. I saw sometimes when people used to run their dags to test their DAGs as python code and added some code at the end of DAG that should never be run when DAGs are parsed by Airflow (this is what happens - your upgrade is actually parsing some DAGs). What helped in that case was putting the code under: |
Beta Was this translation helpful? Give feedback.
8 replies
-
|
I went away and merged it anyway |
Beta Was this translation helpful? Give feedback.
-
|
Wow, thanks for the quick update @potiuk !!! Your suggestion did fix the issue with that migration, though I'm getting the same error now at the end of the migrations and cant seem to pinpoint where it's coming from. Running check-migrations seems to exit silently, albeit successfully. But it looks like all the migrations were successfully applied. But these are all problems for another time. Thank you so much for your detailed explanations and quick fix! |
Beta Was this translation helpful? Give feedback.
-
|
Yeah. I think airflow db init performs Dag importing after finishing migration I believe. I advise you to do bisecting. Remove half of your DAGs and run migration, Then either take those you have (if failed) or the other half (if not) and continue. You will quickly find the affected DAG this way and will be able to narrow down the search. |
Beta Was this translation helpful? Give feedback.
-
|
I'd love to find out what it is and maybe find a solution to warn better if others will have similar problem - I already saw it once so it can happen again |
Beta Was this translation helpful? Give feedback.
-
|
I've solved the issue. It was multiple problems. Putting a After removing the In conclusion:
|
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hey folks,
I've come across an issue where when attempting to run migrations against a clean database, but with existing dags defined in my dags folder, there's a silent failure and the migration exits with the
airflowcommand usage prompt saying "invalid command 'db'".I got to digging and the culprit was the following line.
I'm almost positive that this has to do with an erroneous
.pyfile in my dags folder (I have yet to dig deeper into the issue), but this begged the question of why is the migration is reading the dags to migrate existing data in the database in the first place? Shouldn't the migration simply migrate the data regardless of the state of the dag files themselves?I was able to fix the issue by changing the line in the link above to
dagbag = DagBag(read_dags_from_db=True)but I'd like to see if there was any context behind reading the dag file instead of the database before opening a PR.Beta Was this translation helpful? Give feedback.
All reactions