Kedro
Log in to community

Join the Kedro community

A
M
M
M
D
Join

Error with Global Variable Interpolation in Kedro OmegaConfigLoader

IIonut Barbu

good morning all!
We are facing an error in using global variable interpolation with the OmegaConfigLoader. The error occurs when lunching a jupyter notebook e.g with kedro jupyter lab
the issue seems very similar/identical to the one signaled here https://kedro-org.slack.com/archives/C03RKP2LW64/p1726216824633969

the full error stack is below. The global var is located in conf\globals.yml The issue also occurs for the location conf\base\globals.yml


Any help from the kedro team is very much appreciated

Traceback (most recent call last):
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\IPython\core\shellapp.py", line 322, in init_extensions
    self.shell.extension_manager.load_extension(ext)
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\IPython\core\extensions.py", line 62, in load_extension
    return self._load_extension(module_str)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\IPython\core\extensions.py", line 79, in _load_extension
    if self._call_load_ipython_extension(mod):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\IPython\core\extensions.py", line 129, in _call_load_ipython_extension
    mod.load_ipython_extension(self.shell)
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\kedro\ipython\__init__.py", line 62, in load_ipython_extension
    reload_kedro()
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\kedro\ipython\__init__.py", line 123, in reload_kedro
    catalog = context.catalog
              ^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\kedro\framework\context\context.py", line 187, in catalog
    return self._get_catalog()
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\kedro\framework\context\context.py", line 223, in _get_catalog
    conf_catalog = self.config_loader["catalog"]
                   ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\kedro\config\omegaconf_config.py", line 201, in __getitem__
    base_config = self.load_and_merge_dir_config(  # type: ignore[no-untyped-call]
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\kedro\config\omegaconf_config.py", line 341, in load_and_merge_dir_config
    for k, v in OmegaConf.to_container(
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\omegaconf.py", line 573, in to_container
    return BaseContainer._to_content(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\basecontainer.py", line 292, in _to_content
    value = get_node_value(key)
            ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\basecontainer.py", line 247, in get_node_value
    value = BaseContainer._to_content(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\basecontainer.py", line 292, in _to_content
    value = get_node_value(key)
            ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\basecontainer.py", line 244, in get_node_value
    conf._format_and_raise(key=key, value=None, cause=e)
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\base.py", line 231, in _format_and_raise
    format_and_raise(
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\_utils.py", line 899, in format_and_raise
    _raise(ex, cause)
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\_utils.py", line 797, in _raise
    raise ex.with_traceback(sys.exc_info()[2])  # set env var OC_CAUSE=1 for full trace
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\basecontainer.py", line 242, in get_node_value
    node = node._dereference_node()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\base.py", line 246, in _dereference_node
    node = self._dereference_node_impl(throw_on_resolution_failure=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\base.py", line 277, in _dereference_node_impl
    return parent._resolve_interpolation_from_parse_tree(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\base.py", line 584, in _resolve_interpolation_from_parse_tree
    resolved = self.resolve_parse_tree(
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\base.py", line 764, in resolve_parse_tree
    return visitor.visit(parse_tree)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\antlr4\tree\Tree.py", line 34, in visit
    return tree.accept(self)
           ^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\grammar\gen\OmegaConfGrammarParser.py", line 206, in accept
    return visitor.visitConfigValue(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\grammar_visitor.py", line 101, in visitConfigValue
    return self.visit(ctx.getChild(0))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\antlr4\tree\Tree.py", line 34, in visit
    return tree.accept(self)
           ^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\grammar\gen\OmegaConfGrammarParser.py", line 342, in accept
    return visitor.visitText(self)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\grammar_visitor.py", line 301, in visitText
    return self._unescape(list(ctx.getChildren()))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\grammar_visitor.py", line 389, in _unescape
    text = str(self.visitInterpolation(node))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\grammar_visitor.py", line 125, in visitInterpolation
    return self.visit(ctx.getChild(0))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\antlr4\tree\Tree.py", line 34, in visit
    return tree.accept(self)
           ^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\grammar\gen\OmegaConfGrammarParser.py", line 1041, in accept
    return visitor.visitInterpolationResolver(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\grammar_visitor.py", line 179, in visitInterpolationResolver
    return self.resolver_interpolation_callback(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\base.py", line 750, in resolver_interpolation_callback
    return self._evaluate_custom_resolver(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\base.py", line 694, in _evaluate_custom_resolver
    return resolver(
           ^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\omegaconf\omegaconf.py", line 445, in resolver_wrapper
    ret = resolver(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\kedro\config\omegaconf_config.py", line 384, in _get_globals_value
    raise InterpolationResolutionError(
omegaconf.errors.InterpolationResolutionError: Globals key 'model_to_use' not found and no default value provided.
    full_key: performance_metrics_best_model.filepath
    object_type=dict

N
I
R
16 comments
NNok Lam Chan

Can you try open a regular notebook and run this as the top cell

%load_ext kedro.ipython?

IIonut Barbu

thanks Nok for the prompt reply!

Opened a new notebook just with jupyter lab

the command gives the same error.

IIonut Barbu

NNok Lam Chan

alright. Did it works previously until you add some new thing, or did you pick up an old codebase? I will need these information:

  • How did you define your global configs & where is the config
  • How did you reference the global config & where it the config

NNok Lam Chan

I left a comment there - but if you are using a class out of a kedro run context, you need to either:

  • make sure bootstrap_project are run, so that you load your project settings, and these settings have to be pass to the class correctly. (usually you don't have to handle this because kedro handles it, but you may need to to this when you write test)
  • Pass the argument to the certain class. For example, you may see CONFIG_LOADER_ARGS in settings.py, if you are creating them explicitly, you need to pass those argument to the constructor.

IIonut Barbu

yes, it worked before :

  1. we added global configuration stored in conf/globals.yml. The definition is 
## choose from LightGBMModel, SARIMAXModel
model_to_use: SARIMAXModel
No updates made to settings.py

2. the global config is referenced in the catalog
performance_metrics_best_model:
  type: pandas.ParquetDataset
  filepath: "data/08_reporting/performance_metrics_best_model_${globals:model_to_use}.pq"

timeseries_model:
  type: pickle.PickleDataset
  filepath: "data/06_models/forecasting_model_${globals:model_to_use}.pkl"

energy_consumption_forecast:
  type: pandas.ParquetDataset
  filepath: "data/08_reporting/forecast_${globals:model_to_use}.pq"

if you read the error message, it is the first dataset that is referenced therein.

Important to mention that kedro run works perfectly and beautifully (personally very happy with this improved approach to global configs wrt kedro 0.18).

RRashida Kanchwala

thank you, can you share the version of kedro you are using. You mentioned it worked before; can you clarify if it stopped working after upgrading Kedro ?

IIonut Barbu

by working before, i meant before we added the global config.

we have kedro 0.19.5

NNok Lam Chan

Can you move the globals.yml to, i.e. conf/base/globals.yml?

IIonut Barbu

this fixes the jupyter notebook issue however makes running the pipeline with kedro run fail

The end of the stack trace is

  File "C:\Users\IonutBarbu\miniconda3\envs\EIT-Epsilon\Lib\site-packages\kedro\config\omegaconf_config.py", line 398, in _get_globals_value
    raise InterpolationResolutionError(
omegaconf.errors.InterpolationResolutionError: Globals key 'model_to_use' not found and no default value provided.
    full_key: model_to_use
    object_type=dict

In the code it fails on the last code line below, when trying to load the parameters programmatically in the code
# Initialize weather data extraction class based on given settings
conf_path = str(Path.cwd() / settings.CONF_SOURCE)
conf_loader = OmegaConfigLoader(conf_source=conf_path)
parameters = conf_loader["parameters"]

NNok Lam Chan

Is this piece of code coming from your own code? Why do you need to create a separate config loader ?

IIonut Barbu

we need to load the params programmatically in the code. We follow the guidelines here
https://docs.kedro.org/en/stable/configuration/parameters.html#how-to-load-parameters-in-code

NNok Lam Chan

https://docs.kedro.org/en/stable/configuration/advanced_configuration.html#read-configuration
It may be helpful to read this section of the configuration to understand how environment work

NNok Lam Chan

I can see the confusion from that section. When you are creating the class directly, you need to make sure you pass in the environment as well (assuming you want the same behavior as your kedro project). By default it assume you are not working with any environment (think just one yaml file in a folder)

NNok Lam Chan

You need to pass in the right environment to the class. default base_env=base, default_run_env=local.

# Initialize weather data extraction class based on given settings
But soon you will find new issue, because this does not taken into account your runtime parameters, i.e. kedro run --env <your_specific_env>. I would challenge the idea that initialising a class base on parameters settings, do you really need to do this? This create a circular dependencies in pipeline and is much harder to reason about.

The architecture overview can be found here: https://docs.kedro.org/en/stable/extend_kedro/architecture_overview.html

If you want to use config loader, it would be easier to get this via hook instead of creating your own separate copy
https://docs.kedro.org/en/stable/hooks/introduction.html

IIonut Barbu

thanks a lot Nok! specifying the environments in the call to the OmegaConfig class have solved the problem

also thank you for the additional suggestions regarding our current implementation for class initialization from param settings. I will have a look at the resources you have shared

Add a reply
Sign up and join the conversation on Slack
Join