Join the Kedro community

Updated 2 months ago

Intercepting Kedro Logging System for Pulumi Compatibility

Bit of a long shot but could use some help on how to do this. My team is using an infrastructure as code tool called pulumi. I bootstrap and running a kedro project in a pulumi project is easy enough by running

bootstrap_project(Path("."))
with KedroSession.create() as session:
     session.run()
But pulumi restricts logs of anything that isn't explicitly a pulumi asset. Pulumi does allow you to log arbitrary python code https://www.pulumi.com/docs/iac/concepts/logging/ by running for example pulumi.info(message) pulumi.warning(message) etc..

Is there any way I could intercept the kedro logging system so that instead of it running something like logger.info(message) to emit a message, it could run pulumi.info(message)?

N
2 comments

But pulumi restricts logs of anything that isn't explicitly a pulumi asset. Pulumi does allow you to log arbitrary python code https://www.pulumi.com/docs/iac/concepts/logging/ by running for example pulumi.info(message) pulumi.warning(message) etc..
I am not familiar with Pulumi, but it sounds strange if it prevent Python logging since Kedro won't be the only problem you have.

Alternatively, you can implement a custom logger instead of using Kedro one (defined in your logging.yml, check the docs for how to customise the logging).

Add a reply
Sign up and join the conversation on Slack