Hi Kedroids! :kedro:
Is there a way to override global parameters through a cli kedro pipeline trigger?
kedro run --pipeline <my_pipeline> --params "<my_list_of_global_params>"
Hi Abhishek, yes, you can overwrite your parameters with CLI, it doesn't depend on a pipeline, like in that manual:
https://docs.kedro.org/en/stable/configuration/parameters.html#how-to-specify-parameters-at-runtime
In my testing, global params do not get overriden. Other parameters do get overriden.
Kedro allows the use of global parameters which you can interpolate in rest of the parameters file. This is generally globals.yml
but can be configured for your config loader by specifying any arbitrary glob pattern.
"globals_pattern": "**/globals*"
globals.yml
# Base path for the filepath defined in the catalog raw_base_path: data/ raw_base_path_spark: data/ base_path: data/ base_path_spark: data/
my_dataset: <<: *_spark_parquet filepath: ${base_path_spark}/my_dataset.parquet
I see, that's tricky case, have you tried that advice?
https://docs.kedro.org/en/stable/configuration/advanced_configuration.html#how-to-use-globals-and-runtime-params
ah I see, this makes it explicitly clear that they can't be overriden. Thanks! I will have to find a more intelligent way of doing this it seems ๐ง