Join the Kedro community

Updated 4 weeks ago

Leveraging Delta Lake Functionality in Kedro Datasets

Hello Kedro community,👋

We had a question in our team:
We are using currently dataset databricks.ManagedTableDataset (that are working with databricks table using delta).

We though that dataset.save would use delta function (ex: WhenNotMAtch or when Match), but it is not the case (in source code, we saw that a merge in a sql query is hardcoded).

Is it a reason for that? Is it a PR we should propose?

Thank you and good week-end 🙂

d
T
2 comments

This is an evolving dataset, a PR would be welcome!

Ok, that's what we thought. We ll do it (we ll need some time :))

Add a reply
Sign up and join the conversation on Slack