Join the Kedro community

Home
Members
João Dias
J
João Dias
Offline, last seen 3 days ago
Joined February 18, 2025

Hi everyone! I'm having trouble using tensorflow.TensorFlowModelDataset with an S3 bucket. The model saves fine locally, but when I configure it to save/load directly from S3, it doesn't work.
Some key points:

  • Credentials are fine – I can load other datasets (preprocessing outputs and split data) from S3 without issues.
  • Uploading manually works – If I explicitly upload the model file using boto3 or another script, I can access it in S3 just fine.
  • Had issues with .h5 models – Initially, I could retrieve .h5 files from S3 but loading was not working properly, so I switched to the .keras format, which works fine when handling files manually.
Has anyone successfully used tensorflow.TensorFlowModelDataset with S3? Is there a recommended workaround or configuration to get it working? Any insights would be much appreciated!

To make it clearer: I am only having problems when the node output - the model - is pointed to s3. I am getting access denied even after checking credentials , IAM policies, and testing with manual script

8 comments
E
J