#Model providers

Model providers connect the kernel to model APIs. @lite-agent/provider offers direct anthropic() and openai() adapters, local endpoint presets, and the AI SDK 7 bridge for mainstream vendors. Provider 0.11.0 requires Node.js >=22.

pnpm add @lite-agent/provider

#Quick start

Hand a provider to query from @lite-agent/sdk (or to createAgent from core):

import { anthropic } from "@lite-agent/provider";
import { query } from "@lite-agent/sdk";

// Reads ANTHROPIC_API_KEY and BASE_URL from env by default.
const claude = anthropic();

for await (const ev of query({ prompt: "Hello!", model: claude, modelName: "claude-sonnet-4-6" })) {
  if (ev.type === "text_delta") process.stdout.write(ev.text);
}

import { openai } from "@lite-agent/provider";

// OpenAI or any OpenAI-compatible endpoint (Ollama, vLLM, LM Studio, ...).
// Reads OPENAI_API_KEY and OPENAI_BASE_URL from env by default.
const gpt = openai({ baseURL: "http://localhost:11434/v1" });

#anthropic(options?)

Creates a ModelProvider (id: "anthropic") that maps normalized requests to the Anthropic Messages API, wrapping @anthropic-ai/sdk.

#Options — AnthropicProviderOptions

The returned provider also advertises context capabilities: automatic prompt caching, countTokens, and — when the client's beta.messages.create surface is present — Anthropic-native context edits (clearToolUses, clearThinking, compact).

import Anthropic from "@anthropic-ai/sdk";
import { anthropic } from "@lite-agent/provider";

const client = new Anthropic({ defaultHeaders: { "x-trace-id": "..." } });
const claude = anthropic({ client });

#openai(options?)

Creates a ModelProvider (id: "openai") that maps normalized requests to OpenAI Chat Completions, wrapping the openai SDK. Because the whole protocol family shares one wire format, the same adapter works against OpenAI-compatible and local endpoints.

#Options — OpenAIProviderOptions

#OpenAI-compatible and local endpoints

openai() accepts any server that speaks Chat Completions. Point baseURL at the server's /v1 root:

import { openai } from "@lite-agent/provider";

// Ollama — https://ollama.com
const ollama = openai({ baseURL: "http://localhost:11434/v1", apiKey: "ollama" });

// vLLM — `vllm serve <model>`
const vllm = openai({ baseURL: "http://localhost:8000/v1", apiKey: "local" });

// LM Studio — local server tab
const lmStudio = openai({ baseURL: "http://localhost:1234/v1", apiKey: "lm-studio" });

// llama.cpp — `llama-server`
const llamaCpp = openai({ baseURL: "http://localhost:8080/v1", apiKey: "local" });

#Stream translation: SSE → ModelChunk

Each adapter's core job is translating the provider's SSE event stream into the normalized ModelChunk union from @lite-agent/core:

type ModelChunk =
  | { type: "text_delta"; text: string }
  | { type: "message_done"; message: AssistantMessage; usage: Usage };

What the translators guarantee:

• Text deltas stream in source order; their concatenation equals the final text block.
• Tool calls are accumulated from provider-native argument fragments and emitted as normalized { type: "tool_call", id, name, input } blocks with parsed JSON input.
• A successful stream ends with exactly one message_done, carrying the full AssistantMessage and Usage (inputTokens / outputTokens).

Adapter-specific behavior worth knowing:

• Anthropic also surfaces cacheReadTokens / cacheCreationTokens in usage when reported, emits compaction content blocks, and wraps unknown block types as { type: "native", provider: "anthropic", data }. Malformed tool-call JSON throws.
• OpenAI assembles tool calls by index and falls back to {} when tool-call arguments fail to parse.

#Error handling

Both adapters catch SDK errors and rethrow them as ProviderError (from @lite-agent/core), preserving the numeric HTTP status when available:

import { ProviderError } from "@lite-agent/core";

try {
  for await (const ev of query({ prompt: "Hi", model: anthropic(), modelName: "claude-sonnet-4-6" })) { /* ... */ }
} catch (e) {
  if (e instanceof ProviderError) {
    console.error(e.message, e.status); // e.g. 401, 429, 500 — or undefined for non-HTTP failures
  }
}

This applies to failures before output and mid-stream failures alike, so the core retry() middleware can classify them uniformly.

#Retries

maxRetries defaults to 0 on both adapters on purpose: retry policy is owned by the core retry() middleware, and SDK-level retries would compound with it. Set maxRetries explicitly only if you want the SDK to retry on its own.

#Offline testing with an injected client

Both factories accept a client that only has to satisfy a small structural type — AnthropicClientLike (messages.create returning an async iterable of raw stream events) or OpenAIClientLike (chat.completions.create returning chunks). That makes tests fully deterministic and network-free:

import { openai } from "@lite-agent/provider";
import type { OpenAIClientLike } from "@lite-agent/provider";

async function* fakeChunks() {
  yield { choices: [{ delta: { content: "Hello" } }] };
  yield { choices: [{ delta: { content: ", world" } }], usage: { prompt_tokens: 3, completion_tokens: 2 } };
}

const client: OpenAIClientLike = {
  chat: { completions: { create: async () => fakeChunks() } },
};

const provider = openai({ client });
const chunks = [];
for await (const c of provider.stream({ model: "fake", messages: [] })) chunks.push(c);
// chunks: [{ type: "text_delta", text: "Hello" }, { type: "text_delta", text: ", world" }, { type: "message_done", ... }]

The same seam is how the repo's own conformance suite runs against both adapters without any network access.

#Compatibility levels

The repo distinguishes these levels of support — "OpenAI-compatible" is a protocol claim, not a certification:

#Probing an endpoint

An opt-in smoke test verifies a real OpenAI-compatible endpoint on demand (never in default CI):

LITE_AGENT_COMPAT_BASE_URL=http://localhost:11434/v1 \
LITE_AGENT_COMPAT_MODEL=qwen3 \
pnpm --filter @lite-agent/provider test:compat

The base profile checks text deltas, a single final message_done, delta/final-text consistency, and usage shape. Passing means verified for this endpoint/runtime/model profile — not that every model on that runtime behaves identically.

#API summary

#See also

• The nine strategies — the ModelProvider strategy interface these adapters implement.
• Tool-call codecs — pair nativeCodec() with these providers, or a prompt codec with local models.
• Deployment composition — maintained presets for local runtimes (Ollama, vLLM, LM Studio, llama.cpp).
• Testing utilities — providerConformance and fakeProvider for network-free tests.