any-llm

Project Overview#

any-llm is a Python SDK by Mozilla AI designed to solve LLM provider API fragmentation. Through a single unified interface, developers can seamlessly access 30+ major LLM providers, including cloud services like OpenAI, Anthropic, Mistral, Google Gemini, AWS Bedrock, Azure OpenAI, as well as local deployment options like Ollama, llama.cpp, llamafile, LM Studio, and vLLM.

Core Design Principles:

• Uses Official SDKs: Calls provider official SDKs at the底层, ensuring maximum compatibility and stability
• No Proxy Server: Direct connection to LLM services from the application, reducing network hops and architectural complexity
• Standardized Output: Maps all return data to standard OpenAI Pydantic models (ChatCompletion)
• Framework Agnostic: Can be integrated into any Python project

Core Capabilities#

Supported Providers#

Cloud Major: OpenAI, Anthropic, Mistral, Google Gemini, AWS Bedrock, Azure OpenAI, Cohere

High Performance/Inference: Cerebras, Groq, Fireworks, Together, Perplexity, Hugging Face

Local/Self-hosted: Ollama, llama.cpp, llamafile, LM Studio, vLLM

Installation & Usage#

Requirements: Python 3.11+

# Install specific provider support
pip install 'any-llm-sdk[openai]'

# Install multiple providers
pip install 'any-llm-sdk[mistral,ollama]'

# Install all supported providers
pip install 'any-llm-sdk[all]'

Quick Start Code:

from any_llm import completion
import os

assert os.environ.get('MISTRAL_API_KEY')

response = completion(
    model="mistral-small-latest",
    provider="mistral",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

Production Use (AnyLLM Class):

from any_llm import AnyLLM

llm = AnyLLM.create("mistral", api_key="your-mistral-api-key")
response = llm.completion(
    model="mistral-small-latest",
    messages=[{"role": "user", "content": "Hello!"}]
)

Use Cases#

• Rapid Prototyping: Compare different models horizontally in a single codebase
• Production: Manage connection pools via AnyLLM class, combine with any-llm-gateway for budget control and multi-tenant management
• Hybrid Deployment: Seamlessly switch between cloud and local models
• Avoid Vendor Lock-in: Maintain code flexibility, switch providers based on cost or performance

Enterprise Features (via any-llm-gateway)#

• Budget management (daily/weekly/monthly auto-reset)
• API key management (virtual keys, no provider credential exposure)
• Usage analytics (complete token counting, cost, and metadata)
• Multi-tenant support (cross-user and team access and budget management)

Comparison with Alternatives#

• vs LiteLLM: any-llm uses official SDKs instead of reimplementing interfaces, no proxy server needed
• vs AISuite: any-llm provides more complete type support and active maintenance
• vs Framework-specific Solutions: Framework agnostic, can integrate into any Python project
• vs Proxy Solutions: No hosted proxy server required

Project Attribution#

Officially maintained by Mozilla AI, integrated into their any-agent project, providing underlying LLM call support for the agent building framework.

PyPI Package: any-llm-sdk Latest Version: 1.8.5 License: Apache License 2.0