You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It would be great is each map_index in a set of dynamically mapped tasks could each update a different Dataset depending on the parameters sent to them.
Use case/motivation
Perform the same transformation on a number of similar datasets and then have a bunch of separate DAGs that do further analysis after each mapped task is finished. It seems relatively common to use mapped tasks to do the same thing to multiple datasets, so not being able to be aware of multiple datasets across a set of mapped tasks is something of a handicap.
Description
It would be great is each map_index in a set of dynamically mapped tasks could each update a different Dataset depending on the parameters sent to them.
Use case/motivation
Perform the same transformation on a number of similar datasets and then have a bunch of separate DAGs that do further analysis after each mapped task is finished. It seems relatively common to use mapped tasks to do the same thing to multiple datasets, so not being able to be aware of multiple datasets across a set of mapped tasks is something of a handicap.
Related issues
No response
Are you willing to submit a PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: