Spice.ai

A portable accelerated SQL query, search, and LLM inference engine unifying data federation and AI gateway, built for data-driven AI applications and agents.

Core Positioning#

Spice.ai is a high-performance data and AI infrastructure engine built in Rust. It uses Apache DataFusion as the core query engine and Apache Arrow as the in-memory format, providing unified SQL interfaces (HTTP / Arrow Flight / Arrow Flight SQL / ODBC / JDBC / ADBC) and an OpenAI-compatible LLM inference API. The design philosophy is not to replace underlying storage, but to serve as a lightweight compute and cache layer (Database CDN / Query Mesh) between data sources and AI applications, offering unified federated querying, local materialized acceleration, and native inference gateway capabilities.

Interfaces & Protocols#

SQL & Search Interfaces: HTTP, Arrow Flight, Arrow Flight SQL, ODBC, JDBC, ADBC; built-in vector_search and text_search UDTFs.
OpenAI-Compatible API: HTTP interface supporting seamless OpenAI SDK integration, local model inference (CUDA/Metal acceleration), and hosted model gateway.
Iceberg Catalog REST API: Unified Iceberg REST Catalog access endpoint.
MCP HTTP+SSE API: External tool integration via Model Context Protocol (MCP).

Data Federation & Query Optimization#

Supports standard SQL across arbitrary databases, data warehouses, and data lakes with advanced query pushdown optimization. Scales from single-node to distributed multi-node query execution via Apache Ballista integration.

Data Acceleration & Materialization#

Multi-engine support: Arrow (in-memory), DuckDB, SQLite, PostgreSQL, Spice Cayenne (Vortex columnar format + SQLite metadata).
Storage modes: memory and file (persistent).
Cold start optimization: Acceleration snapshot bootstrapping from S3.
Data sync: Debezium-based CDC with full / append / changes refresh modes.

Enterprise Search#

Keyword search: Tantivy-powered BM25 full-text search.
Vector search: PB-scale vector similarity search with backends including Amazon S3 Vectors, pgvector, DuckDB Vector, SQLite Vec.
Hybrid search: Built-in RRF (Reciprocal Rank Fusion) strategy.

AI Applications & Agents#

RAG workflows: Built-in embedding generation pipeline supporting AWS Bedrock, HuggingFace, Model2Vec (static embeddings), etc.
Text-to-SQL: NSQL model support.
Inference & observability: LLM memory management and inference observability for semantic knowledge layers.

Model & Embedding Providers#

LLM providers: OpenAI (compatible), local files, HuggingFace, Azure, Amazon Bedrock, Anthropic, xAI.
Embedding providers: OpenAI, local files, HuggingFace, Model2Vec, Azure, Bedrock.
Format support: ONNX, GGUF, GGML, SafeTensor.

Data Connectors (30+)#

Stable: PostgreSQL, MySQL, Databricks/Delta Lake, DuckDB, S3 (Parquet/CSV), File, GitHub, Dremio, Spice.ai Cloud.
Beta/RC: GraphQL, DynamoDB, Iceberg, Snowflake, Spark, FlightSQL, MSSQL, ODBC.
Alpha: Oracle, ClickHouse, MongoDB, Kafka, Debezium CDC, GCS, Azure BlobFS, FTP/SFTP, IMAP, etc.
Catalog integrations: Spice.ai Cloud, Unity Catalog, Databricks, Apache Iceberg, AWS Glue.

Architecture Highlights#

Core language: Rust (Cargo workspace with multiple sub-crates under crates/).
Query engine: Apache DataFusion (SQL parsing, optimization, execution).
Data formats: Apache Arrow (in-memory columnar), Parquet, CSV.
Transport protocols: Arrow Flight, Arrow Flight SQL, HTTP, ODBC, JDBC, ADBC.
Distributed execution: Apache Ballista.
Inference mechanism: Local ONNX Runtime for quantized models; remote via OpenAI-compatible endpoints.
Disaggregated storage: Compute-storage separation with local materialized working sets and remote source data.
Deployment: Single-node / distributed cluster / Kubernetes Sidecar / edge-to-cloud native.

Installation & Quick Start#

curl https://install.spiceai.org | /bin/bash
# or
brew install spiceai/spiceai/spice

spice init my_project
cd my_project
spice run

Core configuration is declaratively defined via spicepod.yml, including data sources (from), acceleration engines (acceleration.engine), refresh strategies (refresh_mode), model definitions, and secret references.

Typical Use Cases#

Data-driven Agentic AI: Cross-database querying via MCP or OpenAI-compatible API with federated SQL.
RAG: Vector + full-text hybrid retrieval, semantic knowledge layers, Text-to-SQL.
Database CDN / Query Mesh: Local data materialization for sub-second query response.
Real-time dashboards: Accelerated data refresh with BI tool integration.
Legacy system migration: Unified endpoint federated querying across heterogeneous sources.
Distributed data mesh: Multi-node distributed querying, edge-to-cloud native deployment.

Version Status#

v1.1.1-stable is released (1.0 GA achieved), v2.0-rc.3 is available. Apache-2.0 licensed, developed and maintained by Spice AI, Inc.