Join the Kedro community

Updated 3 weeks ago

Overriding Global Parameters Through a Cli Kedro Pipeline Trigger

Hi Kedroids! :kedro:

Is there a way to override global parameters through a cli kedro pipeline trigger?

kedro run --pipeline <my_pipeline> --params "<my_list_of_global_params>"

D
A
6 comments

Hi Abhishek, yes, you can overwrite your parameters with CLI, it doesn't depend on a pipeline, like in that manual:
https://docs.kedro.org/en/stable/configuration/parameters.html#how-to-specify-parameters-at-runtime

In my testing, global params do not get overriden. Other parameters do get overriden.

what do you mean by global parameters?

Kedro allows the use of global parameters which you can interpolate in rest of the parameters file. This is generally globals.yml but can be configured for your config loader by specifying any arbitrary glob pattern.

"globals_pattern": "**/globals*"

So, as an example, in my case, I put base paths for files which I use in other catalog.yml files.

In globals.yml

# Base path for the filepath defined in the catalog
raw_base_path: data/
raw_base_path_spark: data/
base_path: data/
base_path_spark: data/

In any arbitrary catalog yml file

my_dataset:
  <<: *_spark_parquet
  filepath: ${base_path_spark}/my_dataset.parquet

ah I see, this makes it explicitly clear that they can't be overriden. Thanks! I will have to find a more intelligent way of doing this it seems ๐Ÿง 

Add a reply
Sign up and join the conversation on Slack