I have a fairly simple objective: I have data in Cloud SQL (GCP managed SQL server), in response to cloud pub/sub messages I want to run pipelines to extract data from cloud sql, to some transformation, and write results to BigQuery. I'm looking through the docs, but don't see who I'd trigger a pipeline in response to a pub/sub message (whatever the orchestrator); or is this this kind of thing that's outside kedro's remit - I'd have to do that bit separately?
Hi Paul, I think you right, Kedro can’t directly trigger pipelines from Pub/Sub messages. Instead, for example you can use a Cloud Function to listen to Pub/Sub and trigger Kedro.
