Join the Kedro community

Updated 2 months ago

Documenting data pipelines

Hello folks
Wanted to hear from all of you, how do you document your data pipelines.
I recently started building data pipeline and feels like there are many things which shall be documented. But is there any tool / template to achieve that in a more structured manner.

d
R
V
4 comments

so we used to include a sphinx template in Kedro but we took it away since it wasn't very widely adopted.

I think there are two types of docs:

  • Code - Here you can use docstrings to generate code API docs using tools like Sphinx and Markdown.
  • Data - You can use the metadata tag in the catalog to annotate datasets however you'd like, this is something longer term I'd like to see rendered in kedro-viz (cc ) is that something you'd use?

thanks .

, as Joel said; it is on our roadmap to build a feature that will allow you to document your data pipelines via Kedro-viz which makes it easier for onboarding, communication with stakeholders etc. We would love to get on a call and understand from you on how would you see this.

Sure let me create some initial draft for what all things we would like to document and I can let you know.

Add a reply
Sign up and join the conversation on Slack