File
Local filesystem sink for walkerOS server flows. Appends events to a file as JSON Lines (JSONL), tab-separated values (TSV), or comma-separated values (CSV). Useful for debug logging, audit trails, replay sources, and lightweight local persistence without standing up a database or external service.
File is a server destination in the walkerOS flow:
Receives events server-side from the collector, serialises each event, and appends a line to a local file. It makes no network calls and uses no SDK, only `fs.createWriteStream` with `flags: 'a'`.
Installation
npm install @walkeros/collector @walkeros/server-destination-file- Integrated
- Bundled
import { startFlow } from '@walkeros/collector';
import destinationFile from '@walkeros/server-destination-file';
await startFlow({
destinations: {
file: {
code: destinationFile,
config: {
settings: {
filename: 'events.jsonl',
},
},
},
},
});
A complete flow.json with this destination:
{
"version": 4,
"flows": {
"default": {
"config": {
"platform": "server"
},
"destinations": {
"file": {
"package": "@walkeros/server-destination-file",
"config": {
"settings": {
"filename": "events.jsonl"
}
}
}
}
}
}
}Configuration
This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see destination configuration. Package-specific fields live under config.settings and are listed below.
Settings
| Property | Type | Description | More |
|---|---|---|---|
filename | string | object | Output filename. Static string or Mapping.Value (e.g. { fn: "$code:..." } for daily rotation, { key: "data.tenant" } for sharding). | |
format | 'jsonl' | 'tsv' | 'csv' | Serialisation format. Defaults to jsonl. | |
fields | Array<string> | Event paths used as columns for tsv/csv formats. Object values are JSON-stringified. Required when format is tsv or csv. |
Mapping
This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see mapping.
Examples
CSV with object cell
An event is written as a CSV row with the data object JSON-stringified and properly quoted for embedded commas and quotes.
{
"name": "page view",
"data": {
"title": "Hello, \"World\"",
"count": 3
},
"context": {
"dev": [
"test",
1
]
},
"globals": {
"pagegroup": "docs"
},
"custom": {
"completely": "random"
},
"user": {
"id": "us3r",
"device": "c00k13",
"session": "s3ss10n"
},
"nested": [
{
"entity": "child",
"data": {
"is": "subordinated"
}
}
],
"consent": {
"functional": true
},
"id": "f0cc5f6fec3d8eeb",
"trigger": "load",
"entity": "page",
"action": "view",
"timestamp": 1700000000000,
"timing": 3.14,
"source": {
"count": 1,
"trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9",
"type": "collector",
"schema": "4"
}
}fs.writeFile("events.csv", "1700000000000,page view,\"{\"\"title\"\":\"\"Hello, \\\"\"World\\\"\"\"\",\"\"count\"\":3}\"\n")Daily rotation
A mapping function derives a date-stamped filename from the event timestamp to rotate JSONL files daily.
{
"name": "order complete",
"data": {
"id": "ORD-1"
},
"context": {
"shopping": [
"complete",
0
]
},
"globals": {
"pagegroup": "shop"
},
"custom": {
"completely": "random"
},
"user": {
"id": "us3r",
"device": "c00k13",
"session": "s3ss10n"
},
"nested": [
{
"entity": "product",
"data": {
"id": "ers",
"name": "Everyday Ruck Snack",
"color": "black",
"size": "l",
"price": 420
},
"context": {
"shopping": [
"complete",
0
]
},
"nested": []
},
{
"entity": "product",
"data": {
"id": "cc",
"name": "Cool Cap",
"size": "one size",
"price": 42
},
"context": {
"shopping": [
"complete",
0
]
},
"nested": []
},
{
"entity": "gift",
"data": {
"name": "Surprise"
},
"context": {
"shopping": [
"complete",
0
]
},
"nested": []
}
],
"consent": {
"functional": true
},
"id": "235d0d7bd7a213f2",
"trigger": "load",
"entity": "order",
"action": "complete",
"timestamp": 1776256496000,
"timing": 3.14,
"source": {
"count": 1,
"trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9",
"type": "collector",
"schema": "4"
}
}fs.writeFile("events-2026-04-15.jsonl", "jsonl:event")JSONL append
An event is appended as a single JSON line to a static filename using default JSONL formatting.
{
"name": "page view",
"data": {
"domain": "www.example.com",
"title": "walkerOS documentation",
"referrer": "https://www.walkeros.io/",
"search": "?foo=bar",
"hash": "#hash",
"id": "/docs/"
},
"context": {
"dev": [
"test",
1
]
},
"globals": {
"pagegroup": "docs"
},
"custom": {
"completely": "random"
},
"user": {
"id": "us3r",
"device": "c00k13",
"session": "s3ss10n"
},
"nested": [
{
"entity": "child",
"data": {
"is": "subordinated"
}
}
],
"consent": {
"functional": true
},
"id": "f0cc5f6fec3d8eeb",
"trigger": "load",
"entity": "page",
"action": "view",
"timestamp": 1700000000000,
"timing": 3.14,
"source": {
"count": 1,
"trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9",
"type": "collector",
"schema": "4"
}
}fs.writeFile("events.jsonl", "jsonl:event")Tenant sharding
The filename is resolved from an event field so events are partitioned into per-tenant JSONL files.
{
"name": "custom event",
"data": {
"tenant": "acme"
},
"context": {
"dev": [
"test",
1
]
},
"globals": {
"lang": "elb"
},
"custom": {
"completely": "random"
},
"user": {
"id": "us3r",
"device": "c00k13",
"session": "s3ss10n"
},
"nested": [
{
"entity": "child",
"data": {
"is": "subordinated"
}
}
],
"consent": {
"functional": true
},
"id": "e6517aa4a8c81e02",
"trigger": "test",
"entity": "custom",
"action": "event",
"timestamp": 1700000000000,
"timing": 3.14,
"source": {
"count": 1,
"trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9",
"type": "collector",
"schema": "4"
}
}fs.writeFile("acme", "jsonl:event")TSV log
An event is written as a tab-separated line selecting specific fields for a compact access-log style file.
{
"name": "page view",
"data": {
"title": "Docs"
},
"context": {
"dev": [
"test",
1
]
},
"globals": {
"pagegroup": "docs"
},
"custom": {
"completely": "random"
},
"user": {
"session": "sess-1"
},
"nested": [
{
"entity": "child",
"data": {
"is": "subordinated"
}
}
],
"consent": {
"functional": true
},
"id": "f0cc5f6fec3d8eeb",
"trigger": "load",
"entity": "page",
"action": "view",
"timestamp": 1700000000000,
"timing": 3.14,
"source": {
"count": 1,
"trace": "0a1b2c3d4e5f60718293a4b5c6d7e8f9",
"type": "browser",
"platform": "web",
"url": "https://example.com/docs",
"referrer": "https://example.com/"
}
}fs.writeFile("logs/access.tsv", "1700000000000\tsess-1\tpage view\thttps://example.com/docs\tDocs\thttps://example.com/\n")The destination opens one long-lived WriteStream per resolved filename during init() (for static filenames) or on first matching event (for dynamic filenames). On flow hot-swap or server shutdown, destroy() closes all cached streams.
Filename templating
filename accepts either a static string or a standard walkerOS Mapping.Value. Common patterns:
Tenant sharding
{
"filename": { "key": "data.tenant" }
}Each event lands in a file named after the tenant value (e.g. acme, globex), without an extension. An event whose mapping resolves to no name fails with a missing-input error and is not written.
A mapping value runs one operator, so fn next to key never runs. For a .jsonl extension, build the whole name in fn:
{
"filename": {
"fn": "$code:(event) => event.data.tenant && `${event.data.tenant}.jsonl`"
}
}Daily rotation
{
"filename": {
"fn": "$code:(event) => `events-${new Date(event.timestamp).toISOString().slice(0, 10)}.jsonl`"
}
}The function receives the event being processed. Each UTC day gets its own file, created by that day's first event, without logrotate or cron.
Formats
JSONL (default)
One JSON object per line. The entire event is serialised with JSON.stringify. Easy to ingest with jq, DuckDB (read_json_auto), ClickHouse JSONEachRow, BigQuery external tables, and Athena.
TSV / CSV
Specify fields: string[] listing the event paths to extract as columns, in that order. Object values are JSON-stringified into a single cell, a missing value is an empty cell, and no header row is written. CSV output follows RFC 4180 quoting. TSV writes each value as it is, so a tab inside a value shifts the following columns and a line break splits the row: use CSV for free text. format: "tsv" or "csv" without fields fails at init with a config error.
{
"filename": "events.csv",
"format": "csv",
"fields": ["timestamp", "name", "data"]
}Limits
- One file handle is opened per resolved filename and kept open until
destroy(). Sharding by high-cardinality keys (e.g.user.session) can exhaust the OS file descriptor table. Be deliberate about cardinality. - External rotation (e.g.
logrotate) leaves the cached handle pointing at the rotated inode. Use the date-token pattern above instead. - There is no batching: each event is one write, and its push resolves once the line is written. A
batchsetting has no effect, and the collector logs an ERROR line that names it. - A file that cannot be opened or written (for example
EACCES,ENOSPCorEISDIR) fails the event with atransporterror that names the code, and the event goes to the dead-letter queue (when delivery fails). The process keeps running, and the next event opens the file again.