@lunora/ai

@lunora/ai is a small helper over the Vercel AI SDK and Cloudflare's workers-ai-provider. Call generateText / streamText / generateObject / embed / tool from any function. Workers AI is the zero-config default, but every call is provider-agnostic: pass a Workers AI model id (a string) or any AI SDK model object (@ai-sdk/openai, @ai-sdk/anthropic, OpenRouter, …).

pnpm add @lunora/ai

When a function uses AI, the dev server / lunora prepare reconciles the ai binding into wrangler.jsonc for you ({ "ai": { "binding": "AI" } }), and codegen wires a typed ctx.ai onto your action contexts.

ctx.ai in an action

Inference is an external, non-deterministic call, so like ctx.fetch, ctx.ai lives on actions, not queries or mutations.

import { action, v } from "@/lunora/_generated/server";
import { generateText } from "@lunora/ai";

export const summarize = action.input({ text: v.string() }).action(async ({ ctx, args: { text } }) => {
    const { text: summary } = await generateText({
        model: ctx.ai.model("@cf/meta/llama-3.3-70b-instruct-fp8-fast"),
        prompt: `Summarize:\n\n${text}`,
    });

    return summary;
});

ctx.ai.model(id) resolves a Workers AI model from the binding. Pass the resolved model to the AI SDK functions re-exported from @lunora/ai (generateText, streamText, generateObject, streamObject, embed, embedMany, tool).

Any provider, same call

A string id resolves Workers AI; an AI SDK model object passes straight through.

import { streamText } from "@lunora/ai";
import { openai } from "@ai-sdk/openai"; // optional, bring-your-own

const result = streamText({ model: openai("gpt-5"), messages });

Install the provider you want (@ai-sdk/openai, @ai-sdk/anthropic, …) alongside @lunora/ai; route through a Cloudflare AI Gateway by passing gateway to createAi.

Structured output

import { generateObject } from "@lunora/ai";
import { z } from "zod";

const { object } = await generateObject({
    model: ctx.ai.model("@cf/meta/llama-3.3-70b-instruct-fp8-fast"),
    schema: z.object({ sentiment: z.enum(["positive", "neutral", "negative"]) }),
    prompt: review,
});

RAG — defineRag (@lunora/ai/rag)

defineRag composes ctx.ai (embeddings) with ctx.vectors (Vectorize) into a declared index → retrieve pipeline: chunk → embed → upsert on the write side, embed → query → assemble on the read side. It's a thin library over the two facades every action already has, with no new binding and no codegen.

// lunora/rag.ts
import { defineRag } from "@lunora/ai/rag";

export const docs = defineRag({
    embeddingModel: "@cf/baai/bge-base-en-v1.5", // declared once → index + retrieve embed identically
    index: "docs", // a ctx.vectors index binding key
});

// inside an action:
import { docs } from "@/lunora/rag";

await docs(ctx).index({ id: doc._id, metadata: { title: doc.title }, namespace: ctx.shardKey, text: doc.body });

const { chunks, context, sources } = await docs(ctx).retrieve(question, { namespace: ctx.shardKey, topK: 5 });
// `context` is prompt-ready; `chunks` are ranked; `sources` are deduped refs.

What you get beyond the manual loop:

• Deterministic chunk ids (${sourceId}#${n}): re-indexing a source replaces its chunks; shrinking documents have stale trailing chunks deleted automatically.
• Content-hash short-circuit: re-indexing unchanged text skips chunking, embedding, and every write ({ unchanged: true }), so periodic re-syncs are free.
• Tenant isolation: thread namespace (your shard/tenant key) through both sides; a namespace-less call gets a one-time dev warning (Vectorize indexes are account-global). Multi-tenant apps should set requireNamespace: true to turn the warning into a hard error; single-tenant apps suppress it with allowSharedNamespace.
• Ranking controls: minScore threshold, per-source importance weighting (0 to 1, multiplied into scores), and chunkContext: { before, after } to stitch neighbouring chunks around each match ("embed small, retrieve big").
• asTool(): expose retrieval as an AI SDK tool so a model can decide to search the index itself: tools: { searchDocs: docs(ctx).asTool() }.
• Traced: when the bound context carries ctx.trace, each embedding call is a generation span (gen_ai.operation.name: "embeddings", gen_ai.request.model), so RAG shows up on the trace waterfall. A hand-built context without ctx.trace embeds untraced.

Chunk text lives in vector metadata by default (returnMetadata: "all", topK capped at 20, 10 KiB of metadata per vector). For long documents or deeper retrieval, supply a textStore ({ put, getMany, remove? }: a DO table, KV, …): text moves out of metadata and the topK ceiling lifts to 100. The default chunker is a fixed 1000-char window with 200 overlap; pass chunk for token/sentence/semantic strategies.

The 10 KiB is Vectorize's, and it covers the whole metadata object: chunk text, Lunora's bookkeeping keys, and any metadata you attach. defineRag measures each chunk's metadata as it is assembled and refuses one that would not fit, rather than letting the upsert fail at Vectorize with nothing naming the cause. It also rejects an unworkable chunkSize up front, though that earlier check can only compare characters against a byte ceiling; multibyte text costs up to three bytes each, so the index-time measurement is the one that holds.

The topK 20 is Lunora's cap, not Vectorize's: Vectorize allows 50 with full metadata. Ours is a legacy-V1 holdover.

Bring your own embeddings — no env.AI binding

embeddingModel takes a Workers AI model id (a string, resolved through ctx.ai, so it needs the env.AI binding) or a ready-made AI SDK EmbeddingModel object. Pass an object and the helper embeds through it directly, never touching ctx.ai, so a RAG index over OpenAI (or any provider) needs no Workers AI binding at all. ctx.vectors (Vectorize) is still required: it's the store.

import { openai } from "@ai-sdk/openai";
import { defineRag } from "@lunora/ai/rag";

// A model *object*, not an id → embeds without ctx.ai / env.AI.
export const docs = defineRag({
    embeddingModel: openai.textEmbeddingModel("text-embedding-3-small"),
    index: "docs",
});

Because the object path skips ctx.ai, you can even bind a hand-built context carrying only vectors (e.g. in a test or a non-action caller): docs({ vectors: ctx.vectors }).retrieve(question). A model-id string with no ctx.ai present throws a directed error telling you to pass a model object or wire ctx.ai.

Embedding-model versioning

Embeddings from different models are not comparable: swap the model and every old vector becomes noise a nearest-neighbour query still happily returns. Set an opt-in embeddingModelVersion discriminator (^[A-Za-z0-9._-]{1,40}$) and it is folded into the Vectorize namespace, so bumping the tag re-partitions the index: new writes and reads share a fresh partition and the old vectors become unreachable to new queries (an empty result beats a wrong one). Chunk #0 also stamps the tag in metadata for auditability.

export const docs = defineRag({
    embeddingModel: "@cf/baai/bge-large-en-v1.5",
    embeddingModelVersion: "bge-large-v1.5", // bump when you change embeddingModel
    index: "docs",
});

Leaving embeddingModelVersion unset is byte-identical to before; existing indexes are untouched. The discriminator must live in the namespace (not just the id prefix): a Vectorize id prefix does not partition nearest-neighbour results.

Hybrid search (vector + lexical)

Dense retrieval misses exact keywords, rare tokens, and identifiers. Supply a lexicalStore and retrieve runs a keyword leg alongside the vector leg and fuses the two rankings with Reciprocal Rank Fusion. That combines semantic recall with lexical precision and needs no reranker call.

import { bm25LexicalStore, defineRag } from "@lunora/ai/rag";

export const docs = defineRag({
    embeddingModel: "@cf/baai/bge-base-en-v1.5",
    index: "docs",
    lexicalStore: bm25LexicalStore(), // Okapi BM25 keyword leg
    lexicalTopK: 20, // fanout of the lexical leg (defaults to the query topK)
});

bm25LexicalStore() is an in-memory Okapi BM25 reference adapter: it lives in the worker isolate, so it is not durable and not shared across isolates; use it for tests, local dev, and single-isolate workloads. In production plug a durable RagLexicalStore ({ index, remove?, search }) backed by a DO-SQLite inverted index, D1, or an external search service; the index/remove/hybrid-fusion plumbing is identical. Indexing mirrors chunks into the lexical store automatically, and removals fan out to it too.

RLS-filtered retrieval

rlsFilter derives a metadata filter from the retrieval identity so per-request row-level security applies without every call site remembering to pass it. It receives ctx.auth (an action's ctx satisfies RagContext.auth structurally) and returns a Vectorize metadata filter, which is merged over any explicit filter with the RLS keys winning, so a caller can never widen past the tenant/RBAC scope.

export const docs = defineRag({
    embeddingModel: "@cf/baai/bge-base-en-v1.5",
    index: "docs",
    rlsFilter: (auth) => ({ orgId: (auth as { orgId: string }).orgId }),
});

// the retrieval is transparently scoped to the caller's org:
const { chunks } = await docs(ctx).retrieve(question, { topK: 5 });

The filter applies to both the vector and lexical legs, and only to retrieval; indexing stays a trusted server path. The reference bm25LexicalStore holds no metadata, so it fails closed on a metadata filter it cannot evaluate (skips its leg and warns once); fold the RLS dimension into namespace, or plug a filter-aware lexical store, to keep a lexical leg under metadata-based RLS.

Outside an action

ctx.ai is only wired onto action contexts. In the worker entry, a Durable Object, or a queue / scheduled handler, build the helper directly from the binding:

import { createAi } from "@lunora/ai";

const ai = createAi({ binding: env.AI });

The raw binding escape hatch, ctx.ai.run(model, inputs) (or ai.run(...)), covers Workers-AI-only model families (image, ASR, translation) that aren't surfaced through the AI SDK provider.