Cache
Cache is a built-in config property available on every step type (source, transformer, destination). It prevents redundant processing by caching results and serving them on subsequent matching requests.
Configuration
Add cache to any step in your flow config:
"sources": {
"express": {
"package": "@walkeros/server-source-express",
"config": {
"ingest": { "map": { "method": "method", "path": "path" } }
},
"cache": {
"rules": [
{
"match": { "key": "ingest.method", "operator": "eq", "value": "GET" },
"key": ["ingest.method", "ingest.path"],
"ttl": 300
}
]
}
}
}
ingest.method and ingest.path exist because config.ingest lifts them from
the request. Without it the rule's match and key find nothing, and nothing
is cached (see Key Resolution).
Cache Rule Properties
| Property | Type | Description |
|---|---|---|
match | MatchExpression | When to apply this cache rule |
key | string[] | Dot-paths to build the cache key from |
ttl | number | Time-to-live in seconds |
update | Record<string, Value> | Source caches only: changes to the cached response, applied on HIT and MISS |
match is optional, omit it for an always-match rule.
Cache Properties
| Property | Type | Default | Description |
|---|---|---|---|
stop | boolean | false (true for sources) | Stop flow on cache HIT and skip remaining steps, returning the cached value |
store | string | Built-in memory | Reference to a store in flow.stores |
namespace | string | (none) | Optional key prefix. Omit to write keys directly to the store. Same store + same key + same namespace = same cache entry. |
rules | CacheRule[] | Required | At least one cache rule |
walkeros validate does not reject an unknown key in cache or in a rule;
such a key has no effect.
Behavior by Step Type
Source Cache (stop by default)
Source cache intercepts requests before the pipeline runs. Sources default to stop: true. On cache HIT, the cached respond value is returned immediately and the pipeline is skipped, as in the example above. With stop: false, a HIT changes nothing: the pipeline runs as on a MISS, and update is not applied.
Transformer Cache (step-level)
Transformer cache skips the transformer's execution on HIT but continues the chain with the cached event. This helps with expensive computations such as API enrichment. With stop: true, the remaining chain stops on HIT and downstream transformers and destinations are skipped.
Transformer caches with stop: true in a source chain (source.before, source.next) or in the collector chain (collector.next) halt the event on HIT: no destination sees it. In a destination's before chain, a HIT skips the rest of that chain and the destination receives the cached event. Destination caches with stop: true halt only the per-destination delivery; sibling destinations still receive the event. To drop an event by rule rather than by cache, use a route stop (see Dropping an event with stop).
The dedup step of
flow-complete.json
has no code of its own, only a cache. It runs in the server source's next
chain, so a hit with stop: true drops a resent event before the collector:
{
"stop": true,
"namespace": "dedup",
"rules": [
{
"key": [
"event.id"
],
"ttl": 3600
}
]
}quality/flows/server/transformers/dedup/cache/stop: true. stop: true turns a cache hit into a drop. Guide chapter quality
/flows/server/transformers/dedup/cache/namespace: "dedup". namespace keeps cache keys of different steps apart. Guide chapter quality
[
{
"key": [
"event.id"
],
"ttl": 3600
}
]quality[
"event.id"
]quality/flows/server/transformers/dedup/cache/rules/0/ttl: 3600. ttl in seconds. Guide chapter quality
The first event with this id passes and its id is remembered for an hour. A resend with the same id is stopped (second request, proven in the flow test).
{
"name": "order complete",
"entity": "order",
"action": "complete",
"data": {
"id": "ORD-100",
"value": 149.8,
"currency": "EUR",
"coupon": "WELCOME10"
},
"id": "ev-order-complete",
"trigger": "load",
"timestamp": 1700000000000,
"consent": {
"functional": true,
"marketing": true
},
"globals": {
"language": "en",
"pageGroup": "checkout"
},
"user": {
"device": "d3v1c3",
"session": "s3ss10n",
"id": "cust-42",
"email": "jane@example.com"
},
"source": {
"type": "express",
"platform": "server",
"url": "https://www.example.com/checkout/thanks"
},
"nested": [
{
"entity": "product",
"data": {
"id": "SKU-1",
"name": "Trail Runner",
"price": 129.9,
"quantity": 1
}
},
{
"entity": "product",
"data": {
"id": "SKU-2",
"name": "Running Socks",
"price": 9.95,
"quantity": 2
}
}
]
}return {
"event": {
"name": "order complete",
"entity": "order",
"action": "complete",
"data": {
"id": "ORD-100",
"value": 149.8,
"currency": "EUR",
"coupon": "WELCOME10"
},
"id": "ev-order-complete",
"trigger": "load",
"timestamp": 1700000000000,
"consent": {
"functional": true,
"marketing": true
},
"globals": {
"language": "en",
"pageGroup": "checkout"
},
"user": {
"device": "d3v1c3",
"session": "s3ss10n",
"id": "cust-42",
"email": "jane@example.com"
},
"source": {
"type": "express",
"platform": "server",
"url": "https://www.example.com/checkout/thanks"
},
"nested": [
{
"entity": "product",
"data": {
"id": "SKU-1",
"name": "Trail Runner",
"price": 129.9,
"quantity": 1
}
},
{
"entity": "product",
"data": {
"id": "SKU-2",
"name": "Running Socks",
"price": 9.95,
"quantity": 2
}
}
]
}
}firstDelivery of step dedup. Guide chapter qualityDestination Cache (deduplication)
Destination cache skips the push on HIT, preventing duplicate events from reaching the destination. Other destinations still receive the event. With stop: true, the before transformer chain is also skipped on HIT.
A thank-you page reload sends the same order again with a new event id, so the
dedup step above lets it through. Meta dedups on the order id instead, and the
rule's match limits the cache to order complete:
{
"stop": true,
"rules": [
{
"match": {
"key": "event.name",
"operator": "eq",
"value": "order complete"
},
"key": [
"event.data.id"
],
"ttl": 86400
}
]
}quality{
"key": "event.name",
"operator": "eq",
"value": "order complete"
}qualityKey Resolution
Cache keys use dot-path resolution from a structured context:
ingest.method: request method from source ingestingest.path: request path from source ingestevent.name: event nameevent.data.id: nested event data fieldevent.user.loggedIn: user properties
Ingest holds only the fields the source's config.ingest maps. A source cache
sees only ingest: event. paths resolve in transformer and destination
caches. The resolved paths are joined with :; when every path of the rule
resolves empty, the event is not cached, and nothing reports it.
Update Rules
In a source cache, use update to modify the response on both HIT and MISS:
"cache": {
"rules": [{
"key": ["ingest.method", "ingest.path"],
"ttl": 300,
"update": {
"headers.X-Cache": { "key": "cache.status" },
"headers.Cache-Control": { "value": "public, max-age=300" }
}
}]
}
cache.status resolves to "HIT" or "MISS". Static values use { "value": "..." }. Transformer and destination caches do not apply update, and walkeros validate does not flag it there.
Custom Store
By default, cache uses a shared in-memory store. Reference a custom store for persistence:
{
"stores": {
"files": {
"package": "@walkeros/server-store-fs",
"config": { "settings": { "basePath": "./cache" } }
}
},
"sources": {
"express": {
"package": "@walkeros/server-source-express",
"config": {
"ingest": { "map": { "method": "method", "path": "path" } }
},
"cache": {
"store": "files",
"rules": [{ "key": ["ingest.method", "ingest.path"], "ttl": 300 }]
}
}
}
}
The filesystem store keeps entries on the local disk of one instance. To share cache entries across instances, use @walkeros/server-store-s3 or @walkeros/server-store-gcs instead. Do not use the Sheets store as a cache tier (a cache.store target): nothing rejects it, but every cache read and write then costs a Sheets request. A cache can still wrap it.
The filesystem store rejects a key that begins with / (a WARN line, Path traversal rejected, and no entry), so start key with a field such as ingest.method, or set namespace.
A cache write that fails writes no log line and the push still reports ok: true, so a dedup cache on a store that refuses writes lets duplicates through.
A store can carry a cache of its own. In the complete example, reads from the
customers store are memoized for five minutes:
{
"rules": [
{
"ttl": 300
}
]
}state-storesCustom stores can be either synchronous (in-memory) or asynchronous (file system, S3, GCS, the cache wrapper). Cache lookups await the underlying store.get internally, so HIT/MISS semantics behave identically regardless of which backing you wire.