Last updated:
Every Lunora function call produces telemetry across all three OpenTelemetry
signals: an RPC event (one per query/mutation/action, carrying path,
duration, ok/error, shard, and fan-out), log events
(ctx.log),
spans (ctx.trace), and metrics
(ctx.metrics). A sink receives them.
Locally, none of it needs setup: the Studio groups error events into an
Issues view, streams log lines in Logs, and renders ctx.trace
waterfalls in Traces. To watch a deployed app you point a sink at an
OpenTelemetry collector (your own, a vendor's, or
the Lunora cloud), and the worker and any container ship the same telemetry
over one protocol.
Sinks
A sink is passed to createWorker as the observability option. Lunora ships
several; combine them with combineSinks.
| Sink | Ships to |
|---|---|
consoleSink() | console (local dev) |
otlpSink({ endpoint }) | any OTLP-over-HTTP collector |
webhookSink({ url }) | an arbitrary HTTP endpoint (your own JSON shape) |
sentrySink({ dsn }) | Sentry |
analyticsEngineSink({ … }) | a Cloudflare Analytics Engine dataset |
pipelineLogSink({ pipeline }) | a Cloudflare Pipeline → R2 (durable log store, read via R2 SQL) |
import { analyticsEngineSink, combineSinks, otlpSink } from "@lunora/runtime";
export default createWorker({
// …schema, functions…
observability: combineSinks(
otlpSink({ endpoint: env.LUNORA_OTLP_ENDPOINT, token: env.LUNORA_OTLP_TOKEN }),
analyticsEngineSink({ dataset: env.ANALYTICS }),
),
});Sinks that carry RPC events accept onlyErrors: true to drop successful RPCs and
export only failures (pipelineLogSink is log-only, so it has no such option;
log events always pass through regardless). All four network/log sinks
(otlpSink, webhookSink, pipelineLogSink, plus sentrySink when you wire
its captureLog) forward ctx.log lines; webhookSink takes a transformLog
redactor mirroring its RPC transform.
:::caution[Upgrading a webhookSink]
webhookSink now ships ctx.log lines (message and structured fields,
which may carry user input) in addition to RPC events; previously it shipped
neither. Two consequences for an existing config:
onlyErrorsdoes not gate log lines (only RPC events); everyctx.logcall egresses to the endpoint.- The RPC
transformredactor does not cover log lines, so add atransformLog(same fail-closed contract) if you scrub PII before it leaves the worker.
Set transformLog (or point the sink at a trusted endpoint) before upgrading if
your handlers log user data. sentrySink (opt-in via captureLog) and
otlpSink don't have this expansion.
:::
otlpSink
otlpSink({
endpoint: env.LUNORA_OTLP_ENDPOINT, // required — the collector base URL
token: env.LUNORA_OTLP_TOKEN, // optional — sent as `Authorization: Bearer <token>`
headers: { "x-lunora-deployment": env.LUNORA_DEPLOYMENT_ID }, // optional correlation headers
serviceName: "my-app", // optional — the `service.name` resource attribute (default `"lunora"`)
onlyErrors: false, // optional — export only error spans
});Read endpoint/token from the environment so the platform can inject them at
deploy time with no code change. Each event is one fire-and-forget POST; on a
Worker it is registered with waitUntil so it outlives the response.
Resource attributes
A resource is the identity of the thing producing telemetry: the service, its version, where it runs. It rides on every span, log record, and metric, so a collector can group "all signals from v1.2.3 of the checkout API in Frankfurt" without you stamping it on each call.
otlpSink({
endpoint: env.LUNORA_OTLP_ENDPOINT,
serviceName: "checkout-api", // service.name
serviceVersion: env.CF_VERSION_METADATA, // service.version
serviceNamespace: "storefront", // service.namespace — when several services share a name
deploymentEnvironment: "production", // deployment.environment
resourceAttributes: { "service.instance.id": env.INSTANCE_ID }, // anything else
});resourceAttributes is a free-form bag and wins over every convenience field
above it, so it can also override one.
Auto-detection is opt-in with detectResources: true. The worker then reads
service.version (from SERVICE_VERSION / CF_VERSION_METADATA /
VERCEL_GIT_COMMIT_SHA / GITHUB_SHA / COMMIT_SHA), deployment.environment
(DEPLOYMENT_ENVIRONMENT / ENVIRONMENT / NODE_ENV), and cloud.provider /
cloud.region (the Cloudflare colo the request landed in). A value it cannot
determine is omitted rather than guessed, and anything you set explicitly always
wins.
It is off by default because detection is a guess about your deployment: a stray
NODE_ENV in a binding would otherwise silently label every span. Detection runs
at most once per request, so turning it on costs nothing per log line or span.
:::note[Shard-side signals carry the explicit attributes only]
ctx.log, ctx.trace, and ctx.metrics originate inside the shard, which has
no env of its own, so host-detected attributes are not attached to them. Set
the values explicitly if you need them to match across the worker span and the
shard spans of the same trace.
:::
pipelineLogSink
Persist every ctx.log line durably to a Cloudflare Pipeline → R2, which
gives you a queryable log store (read back with R2 SQL) in your own account,
with no cloud required. It is the durable counterpart to otlpSink (which
streams to a collector). Only log lines are stored; RPC-span metrics belong in
analyticsEngineSink.
import { pipelineLogSink } from "@lunora/runtime";
pipelineLogSink({ pipeline: env.LOG_PIPELINE });Each record carries message, level, functionPath, fields, traceId,
spanId, shardKey, userId, and ts. The send is registered with the
request's waitUntil so it survives isolate teardown.
Reading the durable archive back
pipelineLogSink only writes. To read those records back, with no cloud
involved, use createPipelineLogReader, which builds a safe, keyset-paginated R2
SQL query over the Iceberg table the Pipeline lands in, or run the
lunora logs --durable CLI command.
This needs operator setup; it is not code-only. The reader queries a table that only exists once you have wired the Pipeline to an R2 Data Catalog (Iceberg) table whose schema matches the written columns below, and provided read credentials. Until then the CLI fails closed with a clear message.
1. Create the destination table with a column per written field. The reader
owns these names (its DEFAULT_LOG_COLUMNS), matching exactly what the sink
writes:
| Column | Type | Always? |
|---|---|---|
functionPath | string | yes |
level | string | yes |
message | string | yes |
ts | long (epoch-millis) | yes |
fields | string (JSON) / struct | when set |
shardKey | string | when set |
userId | string | when set |
traceId | string | when set |
spanId | string | when set |
If your table types fields as a string column (so R2 SQL can query it),
turn on serializeFields so the sink stores it as a JSON string; the reader
parses it back to an object on read:
pipelineLogSink({ pipeline: env.LOG_PIPELINE, serializeFields: true });Renamed a column in your Iceberg schema? Pass a columnMap to realign the
reader without touching the writer.
2. Provide the R2 SQL credentials as env vars (also used by ctx.r2sql):
R2_SQL_ACCOUNT_ID: the Cloudflare account that owns the bucket/catalogR2_SQL_TOKEN: an API token scoped to R2 SQL read + R2 Data Catalog + R2 storageR2_SQL_BUCKET: the R2 bucket (warehouse) the catalog runs against
3. Read it back, from code:
import { createPipelineLogReader } from "@lunora/runtime";
import { createR2Sql } from "@lunora/bindings/r2sql";
const reader = createPipelineLogReader(createR2Sql({ accountId, apiToken, bucket }), { namespace: "default", table: "logs" });
const page = await reader.query({ minLevel: "warn", sinceTs: Date.now() - 3_600_000, limit: 200 });
// page.rows — newest first; page.nextCursor — pass as `cursor` for the next pageor from the CLI:
# newest 200 warn+ lines from the last hour, as a table
lunora logs --durable --table logs --namespace default --min-level warn --limit 200
# one JSON object per line, filtered to a function-path prefix (pipeable to jq)
lunora logs --durable --table logs --function-prefix "messages:" --ndjson
# resume the next page with the ts the previous run printed
lunora logs --durable --table logs --cursor 1737460000000The reader paginates by keyset on ts DESC (WHERE ts < cursor), not OFFSET,
so deep pages stay cheap over Iceberg. Filters (sinceTs/untilTs, level,
minLevel, functionPathPrefix, traceId, shardKey, userId) are all
inlined as escaped SQL literals, so user values can never inject.
…or from the Studio. The Studio Logs → Archive feed reads the archive back through an admin-gated route (the worker holds the R2 SQL credentials and runs the reader, so the token never reaches the browser).
For a Vite-first / scaffolded app there's nothing to wire in code: the
generated worker entry already calls createWorker({ logArchive: … }) for you,
resolved from env. Set one more variable alongside the R2_SQL_* credentials:
LUNORA_LOG_ARCHIVE_TABLE: the Data Catalog table (e.g.logs)LUNORA_LOG_ARCHIVE_NAMESPACE: the Iceberg namespace, optional (e.g.default)
That's the opt-in. With the table unset, the Archive feed stays "not configured".
For a hand-written worker (or to pass a columnMap), set logArchive on
createWorker directly:
import { createWorker } from "@lunora/runtime";
export default createWorker({
// …your shardDO, adminToken, etc.
logArchive: { namespace: "default", table: "logs" },
});The Archive feed offers the same function-prefix / user / min-level filters and a
Load more button that pages via nextCursor. Until logArchive and the
R2_SQL_* env vars are set, it shows a "not configured" panel that links to the
setup above, never an error. (The live Requests and Errors feeds keep
reading the bounded in-DO log, which resets on hibernation; the Archive feed is
the durable, unbounded history.)
Structured logging with ctx.log
ctx.log spans the full OpenTelemetry severity ramp (trace, debug, info
and its log alias, warn, error, fatal) and takes either console-style
values or a structured message + fields object:
export const placeOrder = mutation({
handler: async (ctx, args) => {
// Console-style: any number of values, joined into the message.
ctx.log.debug("placing order", args);
// Structured: a message plus a fields object. The fields become
// filterable/indexable log-record attributes.
ctx.log.info("order placed", { orderId: order._id, total: order.total });
// Bind context once with `.with(...)`; every line inherits it
// (per-call fields win on a key clash).
const log = ctx.log.with({ orderId: order._id });
log.warn("inventory low", { sku });
log.fatal("charge failed", { code: err.code });
},
});The (string, object) shape is the structured form; every other shape is
console-style. The rendered message and the structured fields reach the dev
terminal, Workers Logs, the Studio Logs panel, and any sink; the raw positional
args of a console-style call reach only the in-process onLog sink you control.
:::caution[Behavior change]
A two-argument call whose second argument is a plain object, such as ctx.log.info("saved", user), is now the structured form: the message is "saved" and user becomes fields, instead of being rendered into the message as saved {…}. Calls with a non-object second argument, or three or more arguments, are unchanged. If you relied on the object being folded into the message text, pass it as a third argument (ctx.log.info("saved", "-", user)) or pre-render it.
:::
Tracing sub-operations with ctx.trace
Every dispatch is already one span, named after the function path. That tells
you a request took 900ms; it doesn't tell you which part took 900ms. ctx.trace
wraps a sub-operation so it becomes its own span nested under the request:
export const checkout = action({
handler: async (ctx, args) => {
const cart = await ctx.trace("cart.load", () => loadCart(args.cartId));
// Attributes are structured like log fields, and become span attributes.
const charge = await ctx.trace("stripe.charge", () => stripe.charges.create({ amount: cart.total }), { cartId: cart._id });
// Nesting is explicit: the body receives a tracer bound to its own span,
// and calling that is what makes a child.
await ctx.trace("fulfil", async (trace) => {
await Promise.all([trace("reserve.stock", () => reserve(cart)), trace("email.receipt", () => sendReceipt(charge))]);
});
},
});The body's value is returned unchanged, and a throw is recorded as an error span and then re-thrown: this is instrumentation, never flow control. Recording is best-effort, so a failing sink can't turn a working handler into a broken one.
:::note[Why the tracer is passed in, rather than nesting being implicit]
It would read nicer if a bare ctx.trace inside another span's body were
automatically its child. That can't be done correctly here: with
Promise.all([trace("a", …), trace("b", …)]), b starts while a is still
open, so an "innermost currently-open span" rule records b as a child of
a rather than its sibling, and parallel fan-out is one of the main things a
tracer is for. Telling "called inside a" apart from "called concurrently with
a" needs AsyncLocalStorage, which Lunora's Durable Objects deliberately
don't require. Passing the parent is correct in every case, and visible where it
happens.
Calling ctx.trace inside a body instead of the passed tracer isn't an error;
that span is just parented to the request rather than the enclosing span.
:::
A span created inside a function invoked via ctx.runQuery / runMutation /
runAction is attributed to the outer entrypoint's function path, since the
composed call reuses its context. ctx.log follows the same rule.
Spans share the dispatch's trace id with its ctx.log lines and with any
container the handler calls (the same traceparent is propagated), so one trace
stitches together worker, shard, and container.
Joining a trace from upstream
Everything above happens inside one Lunora deployment: the worker mints a trace
and propagates it down. If something in front of the worker already started a
trace (an API gateway, a service mesh, another service), joining it needs one
more decision, because the W3C traceparent
header arrives from the caller.
By default Lunora ignores it and starts its own trace. Continue an upstream
trace with trustInboundTraceContext:
export default createWorker({
// …schema, functions…
trustInboundTraceContext: true, // nothing untrusted can reach this worker
});| Value | Continues the upstream trace when… |
|---|---|
false (default) | never; every request starts a fresh trace |
true | always |
"mtls" | the caller presented a client certificate Cloudflare verified at the edge |
(request) => … | your predicate returns true |
When the upstream is trusted, the dispatch adopts its trace id, parents its span
under the upstream span, and carries tracestate onward, so the waterfall
stitches end to end across services.
:::caution[Why this is off by default]
traceparent is supplied by whoever called the worker. On a worker an untrusted
client can reach directly, honouring it lets that client choose which trace its
spans and ctx.log lines join, grafting entries into another tenant's waterfall
in a shared collector. Because the head-sampling decision is derived from the
trace id, it also lets that client choose its own sampling outcome.
true is the right answer when the worker is genuinely unreachable except
through a front door you control. Check that it really is: a *.workers.dev
route left enabled, or a hostname outside your Access policy, is a second front
door with no gate on it. Use "mtls" when the check itself has to carry the
proof: cf.tlsClientAuth is set by the Cloudflare edge and a caller cannot
forge it, unlike a header.
Error traces are unaffected either way: a trace that produced an error is kept whole regardless of the sampling decision, and that decision is the worker's, never the caller's. :::
If an inbound traceparent is dropped while this option is unset, Lunora logs a
one-time hint naming it, because a broken waterfall shouldn't be silent. Setting
the option to false keeps the behaviour and silences the hint.
:::tip[Keep span names low-cardinality]
Put the varying part in the attributes, not the name: ctx.trace("stripe.charge", …, { orderId }), never a name interpolated from the order id. A name
built from an id makes every span its own group in a collector, which is exactly
what attributes exist to avoid.
:::
Locally, the Traces panel in the Studio renders recent waterfalls from an
in-memory ring on the shard: recent activity on this instance, reset on
hibernation. It is a development readout, not a trace store. For retention and
cross-instance search, point otlpSink at a real collector, where
each span is exported as an OTLP INTERNAL span carrying its parentSpanId.
Expanding a trace gives an elapsed-time ruler over the bars, a per-span detail
block (span id, kind, the full attribute bag, the error, and any
span.addEvent / span.recordException events), and the ctx.log lines the
same dispatch emitted, joined by trace id. Errors only narrows the list to
traces where something threw; the search box matches a span name or span id as
well as the trace's own identifiers, so an id pasted from an error report finds
the trace containing it. Both Logs views carry a Trace link on rows
emitted inside a dispatch, opening the waterfall on the shard it came from.
:::caution[The Requests view's Trace link is best-effort]
The durable request log records each dispatch's trace id, but it outlives the
span ring it points at: the log survives hibernation and the ring does not. A
Trace link on a row older than the current DO instance therefore lands on an
empty Traces panel, because the trace has aged out locally. The id itself stays valid
wherever otlpSink ships spans, which is where a deployed app's
traces are retained; it is also emitted on the Logpush console event, so a SIEM
can join a request to its spans.
:::
Generation spans (AI)
An AI model call is a span with the OpenTelemetry
gen_ai.* attributes on
it: the model and, once the call returns, its token usage. Two paths emit them:
@lunora/aitraces everydefineRagembedding call as a generation span (gen_ai.operation.name: "embeddings",gen_ai.request.model) whenever the bound context carriesctx.trace. Wrap your owngenerateText/streamTextcalls inctx.trace("…", fn, { "gen_ai.request.model": model })for the same.@lunora/agentemits a generation span per model turn (and a span per tool call), with token usage, through itsotlpTelemetryintegration. Add it todefineAgent({ telemetry: { integrations: [...] } })to opt in.
A generation span carries gen_ai.request.model and, from the model result,
gen_ai.usage.input_tokens / gen_ai.usage.output_tokens; with the agent
integration's recordInputs / recordOutputs it also carries the prompt /
completion (off by default). The Lunora cloud reads these to show the model,
token counts, and cost on the Traces waterfall.
Trace sampling
At real traffic, exporting every trace is mostly paying to store the same
successful request a million times. sampling keeps a fraction of them,
mirroring Cloudflare Workers' head_sampling_rate:
export default createWorker({
// …schema, functions…
sampling: {
headRate: 0.1, // keep ~10% of traces (default 1 — keep everything)
alwaysSampleErrors: true, // …but never drop a trace that errored (default)
},
});It governs trace spans only: the per-dispatch SERVER span and the
ctx.trace spans beneath it. ctx.log lines and ctx.metrics measurements are
never sampled: logs are the thing you go looking for when something broke, and a
sampled metric is a wrong metric.
:::tip[sampling vs. onlyErrors]
Both reduce what you export, differently. sampling is deployment-wide and keeps
a representative fraction of whole traces, so you retain a true picture of
normal traffic at a lower bill. A sink's onlyErrors keeps no successful RPC
spans at all, which is cheaper still, but you lose the baseline you would compare
a regression against. Reach for sampling first; use onlyErrors on a sink whose
only job is failures (a Sentry or alerting destination sitting alongside a
sampled otlpSink).
:::
Traces are kept or dropped whole
The decision is deterministic per trace, derived from the trace id rather
than a coin flip. That matters because a trace spans processes: the worker, the
shard, and any container all reach the same verdict for the same trace, so you
never get half a waterfall: the worker span kept and its ctx.trace children
missing, or the reverse.
The worker decides once and propagates it: the sampled bit rides the
traceparent it forwards, so the shard drops the matching spans without
re-deciding.
Errors are never sampled away
alwaysSampleErrors (default true) is the tail bias: a trace that produced
an error is exported whole even when the head decision dropped it. The shard
holds a sampled-out trace's spans until the dispatch settles, then flushes them
if anything failed and discards them otherwise.
So headRate: 0.01 does not mean "1% of your errors". It means 1% of your
successful traffic, plus every failure. That is usually the shape you want:
aggressive sampling on the boring path, complete traces on the interesting one.
:::note[headRate and the trace id]
headRate >= 1 keeps everything and <= 0 drops every non-error trace; in
between, the first 8 hex characters of the trace id are mapped onto [0, 1) and
compared against the rate.
Because the trace id decides, which id is used matters. When the inbound trace context is untrusted (the default), Lunora keys the decision on a freshly-minted server-side id, so a caller cannot choose its own sampling outcome by picking a trace id. When you have opted into trusting the upstream, the upstream's trace id is used, which is what keeps a distributed trace whole across services. :::
Application metrics with ctx.metrics
The third signal, alongside logs and traces. A trace tells you what one request did; a metric tells you what a million requests did.
export const checkout = action({
handler: async (ctx, args) => {
const started = Date.now();
// "How many" — summed over time.
ctx.metrics.count("orders.placed", 1, { plan: user.plan });
// "How many right now" — replaces the previous reading.
ctx.metrics.gauge("cart.items", cart.items.length);
// "What's the distribution" — percentiles, not just a mean.
ctx.metrics.record("checkout.latency_ms", Date.now() - started);
},
});| Method | Instrument | Use for |
|---|---|---|
count | counter | requests, retries, bytes: things you add |
gauge | gauge | queue depth, cache size: current readings |
record | histogram | latency, payload size: distributions |
Each becomes an OTLP metric at POST {endpoint}/v1/metrics: a monotonic Sum, a
Gauge, or a Histogram.
:::caution[Keep attributes low-cardinality]
Attributes are the metric's dimensions, and every distinct combination is a
separate time series. { plan: user.plan } is a handful of series; { userId }
is one per user, which is how a metrics bill gets out of hand. Identifiers belong
on a log line or a span, which is what those are for.
:::
Nothing is pre-aggregated: one call is one exported measurement, with counter and histogram values carrying delta temporality for the collector to aggregate. That keeps the sink model identical to logs and spans, at the cost of one export per call, so in a hot loop, sum locally and record once at the end rather than calling per iteration.
:::caution[Durable per-minute history is opt-in]
The Durable Object can also keep a per-minute rollup of every measurement in a
reserved per-shard SQLite table, so the Studio can chart a 24h local trend without
an external collector. It is off by default because it is not free: with it on,
each ctx.metrics.* call becomes a durable SQLite write on the request path,
a billed, rate-limited storage op that competes with your app's own data for the
shard's write budget. Turn it on with metricHistory: true on the sink you pass to
createShardDO (or an object, { maxSeries, retentionBuckets }, to tune the
caps). The live, cross-instance path is onMetric → your collector, and is
unaffected either way.
:::
Metrics have no local buffer and no Studio panel: their value is in the aggregate
over time, which an in-memory ring on a hibernating instance can't represent.
consoleSink prints them in dev; for anything real, point otlpSink at a
collector. (The Studio's Metrics page shows framework-level per-function
metrics, which are collected separately and always on.)
Delivery metrics (notify)
@lunora/notify counts every notification send onto
ctx.metrics for you, with no code, as long as the send runs on a ctx that
carries ctx.metrics (every handler does). Two low-cardinality series:
| Metric | Dimensions | Meaning |
|---|---|---|
notify.send | channel, provider, status | attempted sends. status is accepted (the provider took it), failed, or gone (the endpoint is unregistered and was pruned). |
notify.skipped | channel, reason | a send that reached nobody. reason is no-subscriptions-matched (an empty broadcast) or channel-not-configured. The "sent 0 because…" signal. |
A single send emits one notify.send measurement; a broadcast aggregates its
outcomes into one measurement per (provider, status) bucket (its value is the
bucket's count), not one per recipient. Each ctx.metrics.count is a durable
write, so a large fan-out must not pay it per subscription.
accepted means the push/chat provider accepted the message, not that it was
delivered or opened. Web Push and FCM give no delivery/open receipts, so the
status stops at the send attempt. A failed send also emits one ctx.log.warn
line ("notify <channel> delivery failed") carrying the error and, for push, the
subscription and user ids, trace-correlated to the enclosing action and durably
archived by the log sink. Successful sends and prunes stay off the log; they live
on the metric.
Container telemetry
Code running inside a container can't use a worker
sink, because it is a separate process. @lunora/container/otel gives it a
zero-config exporter that speaks the exact same wire contract, so container spans
and worker spans land in the same collector side by side.
import { createContainerTelemetry } from "@lunora/container/otel";
// Reads LUNORA_OTLP_ENDPOINT / LUNORA_OTLP_TOKEN from the container env.
const telemetry = createContainerTelemetry();
// Time a unit of work — records an ok span, or an error span if it throws.
const result = await telemetry.trace("transcode", () => transcode(job), { jobId: job.id });
telemetry.emitLog({ level: "info", message: "done", attributes: { jobId: job.id } });
// Before the process exits, flush any in-flight sends.
await telemetry.flush();With no endpoint resolvable the exporter is a silent no-op (telemetry.enabled === false): trace still runs your work, it just records nothing. The same code
runs unchanged locally and in the cloud.
Each POST is bounded by timeoutMs (default 10s), so a hung collector aborts
instead of pinning a send in flight and stalling flush(); failures are handed to
the optional onError callback and never break the container.
It takes the same resource attributes as otlpSink
(serviceVersion, serviceNamespace, deploymentEnvironment,
resourceAttributes, and an opt-in detectResources), so container and worker
telemetry can be grouped by the same service and version:
createContainerTelemetry({
serviceName: "transcoder",
detectResources: true, // service.version, deployment.environment, host.name, k8s.pod.name, process.pid
});Detection here reads the container's environment, so unlike the worker it can
also resolve host.name, k8s.pod.name (only when actually running under
Kubernetes), and process.pid. As on the worker, it is opt-in and anything you
set explicitly wins, including the deploymentEnvironment and serviceVersion
env fallbacks, which only apply when detectResources is on.
Thread the endpoint/token into the container the same way any other config
reaches it, by declaring them on defineContainer:
defineContainer({
name: "transcoder",
// …
env: { LUNORA_OTLP_ENDPOINT: env.LUNORA_OTLP_ENDPOINT },
secrets: ["LUNORA_OTLP_TOKEN"],
// The collector host must be reachable from the container egress allow-list.
allowedHosts: ["collector.example.com"],
});The wire contract
Both the worker otlpSink and the container exporter conform to one contract, so
any OTLP-compatible collector (and the Lunora cloud ingest) accepts either
without special-casing.
Transport. OTLP over HTTP
with JSON encoding (not protobuf). Two endpoints, derived from the configured
base endpoint (trailing slashes are tolerated):
| Signal | Request | Emitted by |
|---|---|---|
| Spans | POST {endpoint}/v1/traces | each RPC dispatch (SERVER) and each ctx.trace (INTERNAL) |
| Logs | POST {endpoint}/v1/logs | each ctx.log.* call |
| Metrics | POST {endpoint}/v1/metrics | each ctx.metrics.* call |
:::note[The Lunora cloud ingest accepts more]
Lunora's own sinks emit JSON, but the cloud ingest also accepts standard OTLP
protobuf (Content-Type: application/x-protobuf, what most Collectors default
to) and gzip (Content-Encoding: gzip) on all three endpoints, so any
OpenTelemetry SDK or Collector can ship to it unchanged. A capped batch comes back
as OTLP partialSuccess. Its authorization bearer is a scoped ingest key,
telemetry-only so a leaked token can't deploy, which the platform mints per org
and injects into each tenant as LUNORA_OTLP_TOKEN at deploy time (alongside
LUNORA_OTLP_ENDPOINT).
:::
Headers.
| Header | Value |
|---|---|
content-type | application/json |
authorization | Bearer <token>, when a token is configured |
x-lunora-deployment (convention) | the deployment id, for the ingest to attribute the sender |
x-lunora-org (convention) | the organization id |
The x-lunora-* headers are a convention the cloud ingest reads to route
telemetry; pass them through the headers option. A collector that ignores them
still accepts the payload.
Encoding. Bodies are standard OTLP ExportTraceServiceRequest /
ExportLogsServiceRequest JSON. Per the OTLP/JSON spec:
traceIdis 16 random bytes as 32 lowercase hex chars;spanIdis 8 bytes as 16 hex chars (the documented exception to proto3 JSON's base64bytes).timeUnixNanofields are the nanoseconds since the epoch as a decimal string (Lunora works in millis, so this is the millisecond value followed by six zeros, and is exact).resourcealways carriesservice.name, plus any other resource attributes configured or detected; the instrumentationscope.nameis@lunora/runtime(worker) or@lunora/container(container).- Attribute values follow the OTLP
AnyValueunion: strings asstringValue, booleans asboolValue, integers asintValue(a decimal string), floats asdoubleValue.
Span shape. One span per RPC event (worker) or per trace/emitSpan
(container):
| Field | Worker (otlpSink) | Container |
|---|---|---|
kind | 2 (SERVER) | 1 (INTERNAL) |
startTimeUnixNano | end − durationMs | your startMs |
status.code | 1 ok / 2 error (with status.message) | same |
Worker spans carry these attributes:
| Attribute | When | Value |
|---|---|---|
lunora.function_path | always | the function path, e.g. messages:list |
lunora.ok | always | boolean |
http.route | always | the function path, i.e. the logical route |
http.response.status_code | always | 200, or the error's status (int) |
http.request.method | if known | the inbound HTTP method |
url.path | if known | the inbound URL path |
url.scheme | if known | https / http |
server.address | if known | the inbound host |
server.port | if known | the inbound port (int) |
user_agent.original | if known | the inbound User-Agent |
lunora.shard_key | if sharded | the shard key |
error.type | on error | the Lunora error code |
lunora.error_status | on error | the HTTP/RPC status (int) |
lunora.fanout.table | on a fan-out | the table fanned across |
lunora.fanout.shards | on a fan-out | shard count (int) |
lunora.fanout.failed | on a fan-out | failed-shard count (int) |
The http.* / url.* / server.* / user_agent.* keys are the OpenTelemetry
HTTP semantic conventions,
so a collector's built-in HTTP dashboards work against Lunora spans with no
mapping. http.route deliberately carries the function path rather than the
transport path (/_lunora/rpc), because that is the route actually being
invoked.
A span that errored also carries a standard OTel exception span event with
exception.type and exception.message, the canonical error representation
collectors look for, alongside the error.type attribute.
Worker spans additionally carry parentSpanId when the trace was
joined from upstream, and flags with the W3C
sampled bit.
Container spans carry whatever attributes you pass, plus error.type when the
work throws, and the same exception event on failure.
Log record shape. body.stringValue is the message; severityText is the
upper-cased level; severityNumber maps as:
| Level | severityNumber |
|---|---|
trace | 1 |
debug | 5 |
info / log | 9 |
warn | 13 |
error | 17 |
fatal | 21 |
Worker log records also carry lunora.function_path, and lunora.shard_key /
lunora.user_id when known. Structured fields (below) become additional
log-record attributes, and each record carries its dispatch's trace_id /
span_id so a line links back to its RPC span.
Privacy
Spans carry error.type and error messages, and log records carry the rendered
message, and either can include user-supplied input. Point endpoint only at a
collector you trust, and use onlyErrors to narrow what leaves the deployment.
url.path is the raw inbound path. On the RPC transport that is a constant
(/_lunora/rpc), but on the opt-in public REST surface a path can carry record
ids or user identifiers, which is worth knowing before pointing a sink at a third
party.
http.route is always the templated function path, so it is the safe one to
group and alert on.
Writing your own sink
A sink receives (event, context). The context is deliberately narrow: a
waitUntil for keeping a send alive past the response, and a
resourceAttributes() thunk resolving the small allowlisted bag described
above. It does not expose the Worker env or the raw
Request, because every registered sink (including your own) is handed the same
context, and a single debug console.log(context) would otherwise print every
secret binding and the caller's Authorization and Cookie headers.