Deploy walkerOS to GCP Cloud Run
This guide deploys a walkerOS flow to Google Cloud Run, forwarding events to BigQuery.
Prerequisites
- Docker installed (for local testing)
- Node.js 22.12+ installed
1. Set Up BigQuery
Follow the GCP BigQuery setup to:
- Create a service account with write permissions
Step 2 creates the dataset and the events table with walkeros setup.
For local testing, download a service account key. Production uses Workload Identity.
Enable Cloud Run API
gcloud services enable run.googleapis.com
2. Bundle the Flow
The example flow at https://www.walkeros.io/flows/gcp-bigquery.json receives events with the Express source on /collect and writes them to BigQuery:
{
"version": 4,
"flows": {
"server": {
"config": { "platform": "server" },
"sources": {
"express": {
"package": "@walkeros/server-source-express",
"config": {
"settings": { "paths": ["/collect"] }
}
}
},
"destinations": {
"bigquery": {
"package": "@walkeros/server-destination-gcp",
"import": "destinationBigQuery",
"config": {
"setup": { "location": "$env.BQ_LOCATION:EU" },
"settings": {
"projectId": "$env.GCP_PROJECT_ID",
"datasetId": "$env.BQ_DATASET:walkerOS",
"tableId": "$env.BQ_TABLE:events"
}
}
}
}
}
}
}
Install the CLI and bundle the flow into dist/:
npm install -g @walkeros/cli
walkeros bundle https://www.walkeros.io/flows/gcp-bigquery.json -o dist/
This creates dist/flow.mjs, dist/package.json, and dist/node_modules/. The CLI adds the packages named in package to the bundle, so the flow needs no separate package list.
"setup" lets walkeros setup create the dataset and the events table, in the region BQ_LOCATION names (default EU). Run it once. It finds Google credentials the way the runtime does, from GOOGLE_APPLICATION_CREDENTIALS (the key from step 1) or your gcloud application default credentials, and the account needs the setup permissions listed under Create service accounts:
export GCP_PROJECT_ID=your-project-id
export GOOGLE_APPLICATION_CREDENTIALS=$(pwd)/sa-bigquery.json
walkeros setup destination.bigquery -c https://www.walkeros.io/flows/gcp-bigquery.json
In a server flow, $env.NAME is read from the environment when the container starts, not at bundle time. Required: GCP_PROJECT_ID; without it the first event fails with Config settings projectId missing in the log. Optional with defaults: BQ_DATASET (walkerOS) and BQ_TABLE (events). Only walkeros setup uses BQ_LOCATION, for the dataset it creates. Only web flows bake $env values in, from config.bundle.env. See Reference Syntax.
3. Test Locally with Docker
Before deploying, verify everything works locally. Replace <version> with the walkeros/flow version matching your CLI (see image tags):
# Run locally (uses key file for auth)
docker run --rm \
-p 8080:8080 \
-v "$(pwd)/dist:/app/flow:ro" \
-v $(pwd)/sa-bigquery.json:/app/sa-bigquery.json:ro \
-e GOOGLE_APPLICATION_CREDENTIALS=/app/sa-bigquery.json \
-e GCP_PROJECT_ID=your-project-id \
walkeros/flow:<version>
The image runs as the user walker (uid 1001), which must be able to read the
mounted key. On a Linux host, a key file only your user can read (mode 600) is
unreadable in the container; for this local test, make it readable with
chmod 644 sa-bigquery.json, and delete it when you clean up.
In another terminal, send a test event:
curl -X POST http://localhost:8080/collect \
-H "Content-Type: application/json" \
-d '{
"name": "page view",
"data": {
"title": "Test Page",
"path": "/test"
}
}'
Expected response:
{ "success": true, "timestamp": 1700000000000 }
The Express source answers a POST before delivery, so the response says nothing about BigQuery. A failed write shows up as a Push failed line in the container log.
Check BigQuery for the event:
bq query --use_legacy_sql=false \
"SELECT * FROM walkerOS.events WHERE DATE(timestamp) = CURRENT_DATE() ORDER BY timestamp DESC LIMIT 5"
4. Deploy to Cloud Run
Create dist/Dockerfile so Cloud Run builds an image from the runtime image plus the whole dist/ directory. The runtime image starts /app/flow/flow.mjs on port 8080 by default:
FROM walkeros/flow:<version>
COPY . /app/flow/
To bundle inside Docker instead, use the canonical two-stage Dockerfile. Cloud Run uses Workload Identity, so no credentials file is needed in the image.
Deploy with the service account attached and the project ID set as an environment variable:
cd dist
gcloud run deploy walkeros-flow \
--source . \
--port 8080 \
--region europe-west1 \
--allow-unauthenticated \
--service-account=walkeros-flow@YOUR_PROJECT_ID.iam.gserviceaccount.com \
--set-env-vars GCP_PROJECT_ID=YOUR_PROJECT_ID \
--memory 512Mi \
--cpu 1 \
--min-instances 0 \
--max-instances 10
Note the service URL from the output (e.g., https://walkeros-flow-xxx-ew.a.run.app).
5. Verify Deployment
Health Check
curl https://walkeros-flow-xxx-ew.a.run.app/health
Send Test Event
curl -X POST https://walkeros-flow-xxx-ew.a.run.app/collect \
-H "Content-Type: application/json" \
-d '{
"name": "page view",
"data": {
"title": "Production Test",
"path": "/"
}
}'
Query BigQuery
bq query --use_legacy_sql=false \
"SELECT name, data, timestamp FROM walkerOS.events WHERE DATE(timestamp) = CURRENT_DATE() ORDER BY timestamp DESC LIMIT 5"
Read the Logs
Cloud Run reads JSON lines from stdout and stderr as structured logs, with
severity and message. Ask the flow for them with format: "json" in its
collector block, then bundle and deploy again:
"collector": {
"logger": { "level": "INFO", "format": "json" }
}
A destination that fails to deliver shows up as one ERROR entry per cause and
minute, with the step (destination.bigquery), the cause code and, when the
vendor answered, the HTTP status in the context. See Logger for the codes.
Next Steps
- Configure event mapping to transform events before sending to BigQuery
- Add more destinations to forward events to multiple services
Cleanup
To remove all created resources:
- Delete the Cloud Run service (
gcloud run services delete walkeros-flow --region europe-west1) - Clean up BigQuery resources
- Delete local files (
dist/,sa-bigquery.json)