A Vercel AI SDK v6 Provider for Ollama, offering type-safe, cross-provider compatible, and native Ollama feature support.
AI SDK Ollama is a Vercel AI SDK v6 Provider designed for Ollama local LLMs, built on top of the official ollama npm package. It fully implements core AI SDK APIs including generateText, streamText, embed, and Output.object, allowing developers to swap in Ollama as a drop-in replacement for OpenAI/Anthropic providers without modifying upper-layer logic.
The project provides enhanced capabilities across multiple dimensions: it improves the response completeness issues of Ollama's native tool calling; builds in ollama.tools.webSearch() and ollama.tools.webFetch() for web search integration; natively supports Reranking for RAG pipelines; and offers a ToolLoopAgent for autonomous agent systems with MCP protocol support (stdio/HTTP transport, including OAuth).
On the engineering reliability front, the project includes a Middleware system (global default parameter injection, <think/> reasoning chain extraction), cascading automatic JSON repair (jsonrepair + Ollama-specific fallback), and streaming utilities like smoothStream and createStitchableStream. It also supports Ollama advanced sampling parameters (mirostat, repeat_penalty, num_ctx) and vision models such as llava and bakllava. The code auto-adapts across Node.js (≥22) and browser environments, managed via a pnpm monorepo with Turborepo, and enforces code quality through ESLint, Prettier, and TypeScript strict mode.
Installation & Quick Start
npm install ai-sdk-ollama ai@^6.0.0
import { ollama } from 'ai-sdk-ollama';
import { generateText } from 'ai';
const { text } = await generateText({
model: ollama('llama3.2'),
prompt: 'Write a haiku about coding',
temperature: 0.8,
});
console.log(text);
Key Configuration Examples
- Embedding model:
ollama.embedding('nomic-embed-text') - Reranking model:
ollama.embeddingReranking('nomic-embed-text') - Advanced parameters:
ollama('llama3.2', { options: { mirostat: 2, repeat_penalty: 1.1, num_ctx: 8192 } }) - Web Search requires env var:
OLLAMA_API_KEY
Unconfirmed Information: NPM package publish status not directly verified; Web Search API depends on Ollama cloud, public availability and pricing unconfirmed; AI SDK v6 official release status pending confirmation; browser-to-local-Ollama connection method (CORS or proxy) not detailed. Current version line v3+ targets AI SDK v6; AI SDK v5 users should use ai-sdk-ollama@^2.2.0.