New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dynamic mapped tasks group arguments are interpreted as MappedArgument when provided to classic operators #39222
Comments
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval. |
@florian-guily Would you like to be assigned to this issue? |
I'd like to yes, i'll find the time to resolve it ! |
Hello there, I've just encountered the same issue.
I have the version 2.6.3 though. |
Interesting, not anything in a task group though, as python operator created with taskflow api seems to work well. Haven't tried with classic python operator. |
Yes, sorry. I meant while using dynamic task mapping. Like the original post.
|
Apache Airflow version
Other Airflow 2 version (please specify below)
If "Other Airflow 2 version" selected, which one?
2.8.4
What happened?
When using expand_kwargs on a task group, arguments of this task group will not get correctly interpreted when using them in classic operators inside this task group. They will be interpreted as MappedArgument instead of their real values.
What you think should happen instead?
Real value of the mapped task group's argument should be passed to the operators.
How to reproduce
This was originally done with a google cloud operator in a task group, but i managed to reproduce it with a bash operator.
Here are the associated logs. You can clearly see that multiple MappedArguments are passed in the echo command, which should not happen.
dag_id=airflow_issue_test_run_id=scheduled__2024-04-14T00_00_00+00_00_task_id=bq_to_gcs_tg.bash_task_map_index=0_attempt=2.log
Operating System
Mac OS Sonoma 14.2.1 (23C71)
Versions of Apache Airflow Providers
apache-airflow-providers-google==10.16.0
apache-airflow-providers-common-sql>=1.11.0
Deployment
Docker-Compose
Deployment details
No response
Anything else?
No response
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: