Scheduling CloudQuery Syncs with Google Cloud Run and Cloud Scheduler
Google Cloud Run (opens in a new tab) is a managed compute platform that enables you to run stateless containers that can be invoked via web requests or Pub/Sub events. It provides a serverless experience for developers to deploy and run highly scalable applications without worrying about the underlying infrastructure. While not a perfect fit for CloudQuery syncs, it provides a relatively simple way to run CloudQuery as a serverless container on Google Cloud. If your use case is more complex but still requires deployment on GCP, you may want to consider deploying to a Virtual Machine, Cloud Composer (opens in a new tab) or Google Data Flow (opens in a new tab) instead.
Note that Cloud Run quotas (opens in a new tab) impose a 1 hour time limit per execution (i.e. CloudQuery sync)
Deploying with Cloud Run
See the cloudquery/cloudrun-example (opens in a new tab) repository for an example Docker file and instructions on how to deploy CloudQuery using Cloud Run and Cloud Scheduler. Cloud Run containers must accept incoming connections on a given port (8080 by default). The Docker image above wraps CloudQuery with a small web server that listens for incoming requests and starts a sync whenever a request is received. You should keep the HTTP endpoint private, making it available only to the Cloud Scheduler that triggers the sync on a schedule.
Improve this Page
If you have successfully deployed CloudQuery with Google Cloud Run and would like to help us improve this guide, please consider opening a Pull Request (opens in a new tab).