Hi all, I'm having trouble getting kedro viz to run, even for an example repo. Here's the steps I've taken:
uvx kedro new
(uses kedro 0.19.11
)1-5,7
(everything but pyspark)uv venv
and uv sync
kedro viz
, I get the following output: Error: No such command 'viz'.
Question, if we have a dataset in the catalog that updates incrementally with an upsert
/append
, and then is an input to another node, is there a risk that the full dataset will get loaded from the catalog and input to the downstream node rather than the increment of data?
Hi team, are there any best practices for optimizing spark code within Kedro pipelines? I have a large pipeline that executes at the last node due to lazy eval. I would like to look at execution plans, etc.
Any suggestions? I suppose this would apply to Polars/Ibis/other similar frameworks.