Join the Kedro community

M
M
M
D
M

Error pushing data to S3: Bad Request and DatasetError

Hey Everyone

I am getting below errors while the pipeline is trying to push some data to s3. Any headsup ?

ClientError: An error occurred (400) when calling the HeadObject operation: Bad Request
The above exception was the direct cause of the following exception:

DatasetError: Failed while saving data to data set CSVDataset(filepath=ml-datawarehouse/warehouse/extraction/doc_table_insert.csv, load_args={},
protocol=s3, save_args={'index': False}, version=Version(load=None, save='2024-10-15T15.35.46.341Z')).
[Errno 22] Bad Request

R
V
3 comments

Hi Vishal, Is this happening for all the datasets that are saved to s3 or this particular entry ?

Hey Thanks for responding. So it was a simple credential issue, the aws session got expired . Although I am getting a new issue now even after saving the data to s3.

INFO Saving data to training_data_insert_records (CSVDataset)... data_catalog.py:581
INFO Saving data to case_data_insert_records (CSVDataset)... data_catalog.py:581
INFO Saving data to s3_insertion_records (CSVDataset)... data_catalog.py:581
[10/15/24 21:18:18] INFO Completed 2 out of 5 tasks sequential_runner.py:93
INFO Loading data from doc_table (TableDataset)... data_catalog.py:539
INFO Loading data from case_table (TableDataset)... data_catalog.py:539
INFO Loading data from user_table (TableDataset)... data_catalog.py:539
INFO Loading data from parameters (MemoryDataset)... data_catalog.py:539
[10/15/24 21:18:26] ERROR Future exception was never retrieved base_events.py:1758
future: <future>
ClientOSError: [Errno 32] Broken pipe

The above exception was the direct cause of the following exception:

ConnectionError: Connection lost

This might be related to a network interruption or may be an S3 timeout

Add a reply
Sign up and join the conversation on Slack
Join