The omegaconfigloader scans the conf/ for subdirectories like base/ and local/ . Suppose , we have a catalog.yml file in both base/ and local/ .
Can we have the same top level keys in catalog.yml for base/ and local/ ??
Hi I've seen you ask many questions this week, did you know about the Kedro Award Winning Docs? https://docs.kedro.org/en/stable/index.html They are super high quality and actually explain a lot of what you have been asking.
The functionality of config environments is described here: https://docs.kedro.org/en/stable/configuration/configuration_basics.html#configuration-environments and https://docs.kedro.org/en/stable/configuration/configuration_basics.html#configuration-loading
For some reason the below code throws error
import logging.config from kedro.config import OmegaConfigLoader from kedro.framework.project import settings project_path = "/home/vishal/Documents/workspace/mlops/data-pipelines/" conf_path = str(project_path + settings.CONF_SOURCE) conf_loader = OmegaConfigLoader(conf_source=conf_path, env="local") conf_catalog = conf_loader["catalog"] conf_params = conf_loader["parameters"]
ValueError: Duplicate keys found in /home/vishal/Documents/workspace/mlops/data-pipelines/conf/base/catalog.yml and
/home/vishal/Documents/workspace/mlops/data-pipelines/conf/local/catalog.yml: case_data_insert_records,
case_data_update_records, case_master, doc_table, dummy_csv_resolver, dumm...
"However, when you use the OmegaConfigLoader
directly, it assumes <i>no</i> environment."