Join the Kedro community

Home
Members
Nicolas P
N
Nicolas P
Offline, last seen 2 months ago
Joined September 18, 2024

Hello team,

I'm trying to upsert into a databricks ManagedTable and have issues due to a space in the name of one of the primary key:

DatasetError: Failed while saving data to data set ManagedTableDataset(catalog=dev-catalog, database=my_database, dataframe_type=pandas, primary_key=['content_id', '"From yyyy-mm-dd hh:mm"'], table=my_table, version=None, write_mode=upsert).

[PARSE_SYNTAX_ERROR] Syntax error at or near '"From yyyy-mm-dd hh:mm"'. SQLSTATE: 42601 (line 1, pos 212)

== SQL ==
MERGE INTO `dev-catalog`.`my_database`.`my_table` base USING update ON base.content_id=update.content_id AND base."From yyyy-mm-dd hh:mm"=update."From yyyy-mm-dd hh:mm""

I tried to set the primary key using the following syntaxes:
  primary_key:
    - content_id
    - "From yyyy-mm-dd hh:mm"

  primary_key:
    - content_id
    - "`From yyyy-mm-dd hh:mm`"

  primary_key:
    - content_id
    - '"From yyyy-mm-dd hh:mm"'
but none seems working.

Would you have any ideas?

Thx!

1 comment
N