Join the Kedro community

Updated 2 days ago

the customer resolver dilemma

Am I being dumb - is there no way to do this without doing a customer resolver like this?

Attachment
image.png
1
L
N
d
20 comments

Feels defined a little recursively, or is this me? 😄

is user password a backend specific arguments? Usually credentials is the argument and you don't need to use a resolver but the built-in Kedro credentials.

I see ibis.TableDataset doesn't have credentials as argument, it is an implementation issue but maybe there are some consideration for this?

My main point is that this feels quite ergonomic for me, why don't we make this default resolver?

I think the Deepayman's implementation is purposefully not opinionated since that way it will work with every Ibis backend

I see https://github.com/kedro-org/kedro/issues/4259 now, so may be we are moving forward with this finally?

It’s not like https://github.com/kedro-org/kedro/issues/4259 blocks https://github.com/kedro-org/kedro/discussions/4320#discussioncomment-11306477 but vice versa. We start as soon as we decide there’s enough evidence to make credentials a resolver.

what's the actual blockers in the discussion? The only thing I can recall is the credential leak

hmm, I think the one that you mentioned above

Oh I see what you mean, it would be nice if the resolver made the catalog repr / dict mask

that is annoying

My point is the so called leakage exist today, so I don't see resolver make it worse🫠

though we can also argue we should fix that? But if that's not a problem for years, is that really important

Actually I've come up with a solution I'm really proud of

Attachment
image.png

Like it's not fool proof, but it's pretty good for interactive workflow and 1 line of code

There’s also one more thing: people inject credentials with hooks like so: https://docs.kedro.org/en/stable/hooks/common_use_cases.html#use-hooks-to-load-external-credentials

It will be slightly more difficult if we resolve them with config loader and make them part of dataset config

though we can also argue we should fix that? But if that's not a problem for years, is that really important

that was the main argument 🙂

https://docs.python.org/3/library/collections.html#collections.UserString

there are always weird Python thing with subclass the native Python type, so you may want to use this one instead.

  1. I'd argue this hook workflow can coexist for situations where this is needed
  2. I think the credentials workflow was designed in 2017 and drastically needs a rethink, it's is broken and clunky.
  3. Overriding the stdlib- you're the software engineers - I'm the ideas guy 😎 / imposter

Add a reply
Sign up and join the conversation on Slack