Production architecture and safety

Zhivex is a runtime library. Your application still owns authentication, tenancy, billing, provider credentials, rate limits, HTTP contracts, and retention policy.

Recommended boundary

Browser or client
  -> your authenticated API route
    -> tenant and model policy
    -> Agent, Runner, or Workflow
    -> durable session / run / workflow store
    -> provider

• Browser: input, display state, local interaction.
• Application server: authorization, SDK calls, provider credentials, tools, policy.
• Database or object store: durable state, artifacts, audit records, retention.

Choose durable state deliberately

Runtime	Recommended state
Unit tests	In-memory
Local scripts and demos	File-backed or SQLite
Long-running service	Postgres or SQLite
Serverless or multiple replicas	Postgres
Large binary artifacts	Object storage plus references

File-backed stores are inspectable but are not shared across serverless instances and may disappear between invocations.

Partition generation caches

TypeScript 1.7 hashes default generation-cache keys and requires a stable authentication scope before a file-backed cache reads or writes. Give different credentials, tenants, and upstream base URLs different opaque scopes; never put a raw secret in the scope.

import {
  createCachedGenerateMiddleware,
  createFileGenerateCache,
  wrapLanguageModel
} from "@zhivex-ai/sdk";
import { createOpenAI } from "@zhivex-ai/openai";

const openai = createOpenAI({ apiKey: process.env.OPENAI_API_KEY });

const cachedModel = wrapLanguageModel(openai("gpt-5.6-luna"), [
  createCachedGenerateMiddleware({
    cache: createFileGenerateCache({ dir: ".cache/zhivex" }),
    scope: "tenant-acme:openai-prod-v2"
  })
]);

Without an explicit scope or a fully partitioned custom key, persistent cache middleware bypasses caching. The file cache also applies private atomic storage, bounded reads, entry-size limits, and optional expiry; those controls do not replace tenant authorization or retention policy.

Map identity before the SDK call

Resolve the authenticated user and tenant in application code. Pass application-owned identifiers into the runner or run scope, and use the same scope for run, resume, cancellation, query, and retention operations.

Never treat a client-supplied tenant id, model id, tool name, session id, or run id as authorization.

Make side effects resumable

• claim an idempotency key before model or tool side effects;
• use revision checks to reject stale resumes and cancellations;
• pass cancellation signals into networked tools;
• forward the SDK idempotency key to downstream services;
• reconcile a timed-out side effect before retrying it.

Durable SDK state does not make an external API call atomic. Payments, email, deployments, and writes still need destination idempotency, an outbox, fencing, or reconciliation.

Apply safety and approval policy

For TypeScript tool-using agents, start with the production preset:

import {
  applySafetyPolicyToAgent,
  createProductionSafetyPolicy
} from "@zhivex-ai/sdk";

const safeAgent = applySafetyPolicyToAgent(
  agent,
  createProductionSafetyPolicy()
);

Approval is not authorization. The application must verify who can approve, bind the decision to the pending request, enforce expiration, consume it once, and write an audit record.

Export safe observability

Record identifiers, provider/model, status, steps, latency, token usage, tool names, tool errors, and approval counts. Redact before data leaves the API process and keep full prompts, tool inputs, tool outputs, approval arguments, and output text disabled unless the destination is approved for sensitive payloads.

Know the release boundary

Prefer documented public package entrypoints and stable APIs. Beta surfaces may change between minor releases; experimental provider-native features should be isolated behind an application-owned service.

Before a release, keep these forms of evidence separate:

• API and documentation alignment;
• offline tests and build gates;
• authenticated provider smoke;
• committed exact SHA and remote CI;
• published package and deployed application.

Use the TypeScript production guide or Python production guide for the full operational contract.

Use observability and evaluations to build evidence, errors and troubleshooting for safe failure handling, and stability and upgrades before changing SDK versions.