Join the Kedro community

Home
Members
Júlio Resende
J
Júlio Resende
Offline, last seen 2 months ago
Joined October 29, 2024

Hello everyone!
I'm having some troubles using the geopandas.GenericDataset. Here is my dataset:

raw_line:
  type: geopandas.GenericDataset
  filepath: "data/01_raw/lines/lines.shp"
  file_format: file

I'm facing the error:
DatasetError: Failed while loading data from dataset GenericDataset(file_format=file,
filepath=C:/MyCodes/my_project/data/01_raw/lines/lines.shp, load_args={}, protocol=file, save_args={}).
Failed to open dataset (flags=68): /vsimem/6485f3632b634505a3cf8c07708393b2

It looks like there is an old issue related to fsspec + geopandas:
https://github.com/kedro-org/kedro/issues/695#issuecomment-973953139

My libs:
kedro==0.19.9
kedro-datasets==5.1.0
fiona==1.10.1
fsspec==2024.10.0
geopandas==1.0.1

Is anyone able to use geopandas.GenericDataset with .shp files?

7 comments
J
D
J

Hello everyone

I'm using kedro inside a notebook on Databricks (unfortunately I need to use notebooks and I don't have authorization to use the new kedro-databricks plugin yet). It turns out that I'm having a very bad experience with the logs. The notebook cell is constantly blinking and freezing, so I can't view the logs. Below is my log configuration file. Could anyone suggest a better log configuration for Databricks notebooks?

version: 1

disable_existing_loggers: False

formatters:
  simple:
    format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"

handlers:
  console:
    class: logging.StreamHandler
    level: INFO
    formatter: simple
    stream: <a target="_blank" rel="noopener noreferrer" href="ext://sys.stdout">ext://sys.stdout</a>

  info_file_handler:
    class: logging.handlers.RotatingFileHandler
    level: INFO
    formatter: simple
    filename: info.log
    maxBytes: 10485760 # 10MB
    backupCount: 20
    encoding: utf8
    delay: True

  rich:
    class: kedro.logging.RichHandler
    rich_tracebacks: True

loggers:
  kedro:
    level: INFO

  grp_global:
    level: INFO

root:
  handlers: [console]

14 comments
L
J
Y