Integrations

If it speaks the OpenAI API, it works with NemoRouter.

NemoRouter is a managed LLM gateway with a fully OpenAI-compatible REST API. Every framework, SDK, and tool that already talks to OpenAI talks to NemoRouter — change two lines and keep one key, one bill, and built-in guardrails across 20+ models.

Start free — no credit card View the docs

integrations · openai-compatible

The whole integration

API formatOpenAI

Base URL change1 line

API key change1 line

SDK rewrite needednone

FrameworksLangChain · LlamaIndex · …

ObservabilityLangfuse · Datadog · S3

OpenAI-compatibleOne keyGuardrails included

API surface: OpenAI
SDK languages: 7
Frameworks tested: 7
Lines to switch: 2

How it works

Three steps, two lines of code

There is no NemoRouter SDK you are forced to adopt. Compatibility is the integration — point any OpenAI client at our endpoint.

Create a key

Point your client

Set base_url to the NemoRouter endpoint and api_key to your key. That is the entire integration.

Ship

Your framework, SDK, and observability tooling all run unchanged — now with one key, one bill, and built-in guardrails.

Quickstart

Copy, paste, run

These snippets are generated from the canonical NemoRouter SDK examples — the exact same source the playground and keys page use. Set NEMOROUTER_API_KEY in your environment and go.

Installpip install nemoroutersdk

1	`# Cache: enabled (org default). Pass nemo_cache: false to skip.`
2	`from nemoroutersdk import NemoRouter, NemoGuardrailBlockedError`
3
4	`client = NemoRouter() # reads NEMOROUTER_API_KEY from env`
5
6	`try:`
7	`response = client.chat.completions.create(`
8	`model="gemini-2.5-flash",`
9	`temperature=1,`
10	`max_tokens=1024,`
11	`top_p=1,`
12	`messages=[`
13	`{"role": "user", "content": "Hello! What models do you support?"},`
14	`],`
15	`# nemo_cache=False, # Uncomment to skip cache`
16	`)`
17	`print(response.choices[0].message.content)`
18
19	`# Auto-captured metadata`
20	`meta = client.last_response`
21	`if meta:`
22	`print(f"Cost: ${meta.cost}")`
23	`print(f"Guardrails: {meta.guardrails_applied}")`
24
25	`except NemoGuardrailBlockedError as e:`
26	`print(f"Blocked by guardrail: {e}")`

Base URL is always https://api.nemorouter.ai/v1. Works with the official OpenAI SDKs and any OpenAI-compatible client.

Frameworks

The frameworks your team already uses

Each of these is a standard OpenAI-API client. NemoRouter does not ship a bespoke plugin for them — it does not need to. They work because the API surface is identical.

LangChain

Chains, agents, and RAG pipelines.

Use ChatOpenAI with base_url set to the NemoRouter endpoint. Every chain, tool, and agent runs unchanged.

pip install langchain-openaiProvider docs

LlamaIndex

Retrieval and document indexing.

Point the OpenAI LLM class at NemoRouter. Query engines and retrievers keep working — only the endpoint moves.

pip install llama-index-llms-openaiProvider docs

Vercel AI SDK

Streaming UIs for Next.js & React.

Configure the @ai-sdk/openai provider with a custom baseURL. streamText and generateText route through NemoRouter.

npm install ai @ai-sdk/openaiProvider docs

Haystack

Production RAG and search pipelines.

OpenAIGenerator and OpenAIChatGenerator accept an api_base_url — set it to NemoRouter and your pipelines are unchanged.

pip install haystack-aiProvider docs

CrewAI

Multi-agent orchestration.

CrewAI reads OpenAI-style env vars. Set OPENAI_API_BASE to the NemoRouter endpoint and your crews run on managed models.

pip install crewaiProvider docs

AutoGen

Conversational agent frameworks.

AutoGen LLM configs take a base_url field. Drop in the NemoRouter endpoint and every agent message routes through the gateway.

pip install autogen-agentchatProvider docs

Google ADK

Agent Development Kit.

ADK speaks the OpenAI-compatible interface via LiteLLM-backed models — point it at NemoRouter to keep one key and one bill.

pip install google-adk

Any OpenAI client

Instructor, Marvin, DSPy, Semantic Kernel…

If a library accepts an OpenAI base URL or honors OPENAI_API_BASE, it works with NemoRouter. No SDK fork, no shim, no waiting on us.

Don't see your framework? If it accepts an OpenAI base URL or honors OPENAI_API_BASE, it already works.

SDK matrix

One endpoint, every language

NemoRouter ships a first-party Python SDK and works with the official OpenAI SDKs everywhere else. Set the base URL, set the key — the request and response shapes are unchanged.

Language	Install	Client
Python	`pip install nemoroutersdk`	`NemoRouter SDK or openai`
Python (OpenAI)	`pip install openai`	`openai.OpenAI(base_url=…)`
Node.js / TypeScript	`npm install openai`	`new OpenAI({ baseURL })`
Go	`go get github.com/sashabaranov/go-openai`	`openai.ClientConfig{ BaseURL }`
Java	`Maven / Gradle — openai-java`	`OpenAIOkHttpClient.builder()`
Ruby	`gem install ruby-openai`	`OpenAI::Client.new(uri_base:)`
PHP	`composer require openai-php/client`	`OpenAI::factory()->withBaseUri()`
cURL / HTTP	`Any HTTP client`	`POST /v1/chat/completions`

Live, runnable snippets for every language are on the playground and your dashboard keys page.

Observability & tooling

Logging callbacks, built in

NemoRouter forwards request logs, traces, and metrics to the tools your platform team already runs — configured from Observability → Callbacks, with zero changes to your application code.

Langfuse

Stream traces, spans, and token usage to Langfuse for prompt analytics and evals. Configure it once under Observability → Callbacks — no code change in your app.

Per-request traces with model + latency
Token + cost attribution
Org / team / key scoping

Datadog

Ship request logs and latency metrics to Datadog so LLM traffic sits alongside the rest of your service telemetry.

Latency + error-rate metrics
Structured request logs
Dashboards + monitors in Datadog

Amazon S3

Archive raw request logs to an S3 bucket you control — for retention, audit, or downstream batch analysis.

Your bucket, your retention policy
JSON log objects
Feeds warehouse / SIEM pipelines

Slack

Route alerts — LLM errors, budget thresholds, provider outages — into a Slack channel so the right people see them fast.

8 configurable alert types
Channel-level routing
No webhook plumbing to maintain

Alerts can also fan out over generic webhooks — see the observability overview.

Why route through Nemo

The integration buys you more than a base URL

Pointing your client at NemoRouter is two lines — but those two lines move every request behind a managed gateway.

One key for every model

Switch models by changing a string — no per-provider keys, no SDK swaps. NemoRouter manages all provider credentials.

Guardrails on every request

PII redaction, prompt-injection detection, and secret scanning run at the gateway — your framework code stays clean.

One bill, real cost tracking

Per-key, per-team spend tracked in real time. The cost header is authoritative — credits reserve and settle automatically.

Observability without plumbing

Langfuse, Datadog, S3, and Slack callbacks are toggles in the dashboard — not webhook code you write and maintain.

On the roadmap

Terraform provider & CLI — coming soon

These are full product builds, not marketing pages — so they are not live yet. We are tracking them as committed roadmap items, not shipped features.

Upcoming

Terraform provider

Manage keys, teams, budgets, and guardrails as code. Planned — not yet published to the Terraform registry.

Declarative key + team management
Budgets and guardrails as code
CI-friendly drift detection

Upcoming

CLI tool

A nemo command for keys, usage, and config from the terminal. In design — no install command to publish yet.

Scriptable key rotation
Usage + spend from the shell
Local config + profiles

Want early access? sales@nemorouter.ai

Two lines · five minutes

Plug NemoRouter into your stack today

Keep your framework, keep your SDK, keep your observability tooling. Change the base URL and the key — get one bill, built-in guardrails, and real cost tracking.

Start free — no credit card Migrate from another gateway