Join the Kedro community

M
M
M
D
M

Kedro run executes nodes out of order despite expected DAG behavior

Hi all,

When running uv run kedro run, the node in blue gets run before running the nodes upstream, while these are input for the blue node (it basicually unions two datasets back together). I would not expect this behavior, as I thought the entire pipeline should be executed as a DAG? Am I wrong in this assumption here? I have the following pipelines: ingestion, data_prep, feature, model_input, modeling and reporting .

L
T
7 comments

Could it be your somehow are missing out the connecting nodes due to how you connect the pipelines?

Thanks for your quick answer Laurens! I am not explicitly connecting the pipelines, as I assumed the pipeline autodiscovery will resolve the order by looking at the inputs and outputs of the first and and last nodes of each pipeline.

What does your default pipeline look like?

Are you sure all the nodes in Kedro viz are part of the default pipeline?

When putting a breakpoint in the registry, I found an error in one of the pipelines

Probably, kedro viz was showing an older version, so from now on I will always use uv run kedro viz run -a πŸ™‚

Thanks for your help!

Add a reply
Sign up and join the conversation on Slack
Join