Join the Kedro community

M
M
M
D
M
Members
Fazil Topal
F
Fazil Topal
Offline, last seen 1 hour ago
Joined September 12, 2024

hey everyone,

Is there a way to run kedro-viz on docker without actually installing the lib? I am asking because i wanted to keep the env a bit clean and I thought docker for viz would be nice. Did anyone do that before?

44 comments
1
J
F
N
d
R

Hi everyone,

I have the following files:
settings.py

CONFIG_LOADER_ARGS = {
    "base_env": "base",
    "default_run_env": "local",
    "config_patterns": {
        # Also include models.yml in the catalog
        "catalog": [
            "catalog*",
            "models*",
            "catalog*/**",
            "models*/**",
            "**/catalog*",
            "**/models*",

        ],
    }
}
conftest.py
from kedro.framework.project import settings


@fixture(scope='session')
def config_loader():
    kwargs = settings.CONFIG_LOADER_ARGS
    kwargs.update(env="test", base_env="base", default_run_env="test")

    return OmegaConfigLoader(
        conf_source=str(PROJECT_PATH / settings.CONF_SOURCE), **kwargs,
    )

I expect my settings valued to loaded but this doesnt seem to be the case as I can't get config patters in the dict itself. Any idea why is that?

3 comments
F
N

Hey guys,

Me again 😄 I had a question regarding parquet dataset itself. I often encounter issues with custom datatypes during saving. For instance if i have a custom class in my dataframe i would like to still keep this as is - the reason why i use parquet -. I know there needs to be customer serializer/deserilizar code required to do this. I can for sure do it in my code but since it's io related, i believe it should be done in the dataset definition where i can somehow point to my custom which gets serialized before writing to file. I will work on the extended version now, i was wondering if it was discussed before?. i am happy to push this as a PR later

16 comments
F
N

hey all,

I noticed something that when i define the following node:

node(
    func=sample_func,
    # name="sample_func"    -> does not work when this is commented
    inputs="epubdf",
    outputs="result"
)

and then run this test:
kedro_session.run(node_names=['sample_func'])
I get an error saying name doesnt exists but when i specify name parameter in node itself it works. I thought if i don't provide it, name would be equal to func name itself, no?

9 comments
N
F
M
J

Hey everyone,

I was wondering if there was a way to do the node declaration as follows:

pipelines/data.py

@node(inputs=..., outputs=...)
def preprocess(...):
    ...
The reason why i ask is because my nodes are growing and it is becoming a bit tiring to go import my function, do the kedro wiring and occasionally I have typos as i forgot about the function signature. Something like this would be quite nice to declare.

4 comments
J
F

Hey guys,

Do we know how to pass credentials to a node in kedro? Is it only meant to be accessed by a dataset loader? I have a code that makes API calls (LLMs) and either I get them set as env or pass them from my local credentials? I can manually load for sure but i was looking for a better way. Maybe similar to parameters, credentials: openai?

25 comments
I
N
F
J