Join the Kedro community

Updated 5 months ago

Knowing pipeline run status in a Kedro project

At a glance

The community member asks how to know if a pipeline ran successfully in a Kedro project, and how to record the status of each pipeline run as success or failure. The first comment suggests that the pipeline is successful if there are no errors and the logging message "Pipeline execution completed successfully" is displayed. However, the second community member notes that this may not be sufficient for a production environment, and asks how to save the success status in a database.

The other community members then introduce the "Hook" feature in Kedro, specifically the after_pipeline_run hook, which they suggest is likely what the original community member needs to achieve their goal of recording the pipeline run status.

Hey Team a very naive question. How can we know if a pipeline ran successfully. So basically, I would like to record the status of each pipeline run as success or Failure. I wanted to know the best way to have it in a kedro project.

M
V
N
4 comments

It's successful if there's no errors and you'll get the following logging message: " Pipeline execution completed successfully."

Yeaah i can see this message on my console, but if the pipeline is running in a production environment , how do i declare the run was successful and save the Success status in database. Do you suggest to look for this specific message in the log file and then take the action.

May I introduce you the Hook feature? This is something very useful when you want to extend on Kedro.

For example, there is a hook called `after_pipeline_run , which is probably exactly what you need :)

Add a reply
Sign up and join the conversation on Slack