Hi all! In my organization, we run Databricks Workflow based on GitHub repository source code using kedro pipeline daily basis.
I’d like to know which nodes take most of the time to process. What would be the best practices to know how long each nodes require to run in this scenario?
I just tried and succeeded to print out how long each nodes process by implementing hooks. Thank you!