Join the Kedro community

M
M
M
D
M

Databricks workflow performance optimization

Hi all! In my organization, we run Databricks Workflow based on GitHub repository source code using kedro pipeline daily basis.

I’d like to know which nodes take most of the time to process. What would be the best practices to know how long each nodes require to run in this scenario?

d
S
3 comments

Kedro hooks are your friends!

I just tried and succeeded to print out how long each nodes process by implementing hooks. Thank you!

Add a reply
Sign up and join the conversation on Slack
Join