Redis Streams
Server-side event streaming to Redis Streams via the ioredis client. Each event is appended to a configurable stream via XADD, with optional approximate MAXLEN trimming for bounded memory usage, JSON or flat serialization modes, and graceful shutdown via destroy().
Redis Streams is a server destination in the walkerOS flow:
Receives events server-side from the collector, serializes them as JSON (or flat fields), and appends them to a Redis Stream for downstream consumers (stream processors, worker queues, pub/sub fan-out, XREAD consumer groups).
Installation
- Integrated
- Bundled
Configuration
This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see destination configuration. Package-specific fields live under config.settings and are listed below.
Settings
| Property | Type | Description | More |
|---|---|---|---|
redis | object | Redis Streams configuration |
Mapping
Per-event rules under config.mapping. For the standard rule fields (consent, condition, data, batch, name, policy) see mapping.
| Property | Type | Description | More |
|---|---|---|---|
streamKey | string | Override Redis stream key for this rule. Takes precedence over settings.redis.streamKey. |
Examples
ignored event
json default
order complete
stream key override
with exact trim
with max len
The destination creates a single long-lived ioredis client during init(). On flow hot-swap or server shutdown, destroy() calls client.quit() to flush in-flight commands and close the TCP connection gracefully. User-provided clients (wired in via _client) are left untouched.
Stream entry format
Each event is appended via XADD <streamKey> [MAXLEN [~] <n>] * <field> <value> ...:
- streamKey: from
settings.redis.streamKey(ormapping.settings.streamKeyoverride) *: auto-generated entry ID (ms-timestamp + sequence)- fields: in
jsonmode, a singleeventfield with the full event JSON-stringified; inflatmode, top-level event keys as separate fields (nested objects JSON-encoded) - MAXLEN: when
settings.redis.maxLenis set, trims the stream to ~N entries. Approximate (~) by default; setexactTrimming: trueto trim exactly (slower)
Use mapping.settings.streamKey to route specific events to dedicated streams (e.g. orders to walkeros:orders, identities to walkeros:identities).
Serialization modes
JSON (default)
Stores the full walkerOS event as a single event field. Easiest to consume, decode with JSON.parse().
XADD walkeros:events * event '{"entity":"page","action":"view",...}'
Flat
Stores top-level event fields as separate stream entry fields. Nested objects are JSON-encoded. Useful when downstream consumers want to filter or project specific fields without parsing the full event.
XADD walkeros:events * entity page action view timestamp 1700000100 data '{"id":"Home"}'
Configure via settings.redis.serialization: 'flat'.
Authentication
Redis Cloud / Upstash (TLS + password)
AWS ElastiCache (in-VPC, no auth)
Custom options (ACL user, non-default db)
MAXLEN trimming
Redis Streams grow unbounded by default. For high-volume pipelines, cap stream length with maxLen:
Approximate trimming (~) is used by default. Redis trims to "about N" entries with minimal overhead. For exact trimming, set exactTrimming: true (slower, use only when strict bounds are required).