Join the Kedro community
M
M
M
D
M
👋 Hello, team!
I want to use env variable in a node of a kedro pipeline
I have set up the credentials.yml as this, however it doesn't populate it with the env var inside the node (works for fetching kedro SnowParktableDataset though)
snowflake_creds:
user: ${oc.env:snowflake_svc_user}Any other ways to do this?Do you want to use any environment variable or specifically credentials? Credentials are purposefully abstracted in Kedro for security reasons, so I would advice using any sensitive data directly in a node if you can prevent it.
Specifically credentials
I found a way using OmegaConfigLoader like this:
config_loader = OmegaConfigLoader(conf_source=".") credentials = config_loader["credentials"] openai_client_creds_dict = None if "azure_openai_client_creds" in credentials: openai_client_creds_dict = credentials["azure_openai_client_creds"]But open to better suggestions 😄
We have an experimental dataset for openai with langchain: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langchain/_openai.py this might give some inspiration on how to handle this.