# FlexInference

> FlexInference is an OpenAI-compatible LLM router. Bring your own provider key, or go managed and pay from a prepaid wallet with no key of your own. It routes OpenAI, Anthropic, and Gemini direct, reaches the same providers through Amazon Bedrock, Google Vertex AI, and Azure AI Foundry on your own cloud credential, and runs open models on Cloudflare Workers AI and DeepInfra, with deadline-aware flex routing via start_within.

Point your existing OpenAI SDK (or curl) at https://api.flexinference.com/v1, add the start_within field (a duration like 00h-00m-30s, from 5 seconds to 10 minutes, or default/priority/auto) that every request needs unless the API key carries a default for it, and for flex-capable OpenAI and Gemini models FlexInference races the cheaper flex tier against standard for that request using your own key. It speaks six request/response shapes - OpenAI Responses (/v1/responses), OpenAI Chat Completions (/v1/chat/completions), Google Interactions (/v1/interactions), Anthropic Messages (/v1/messages), Google GenerateContent (/v1beta/models/{model}:generateContent), and Amazon Bedrock Converse (/model/{modelId}/converse) - and every endpoint reaches every model, so you keep the client library you already use. Anthropic (Claude) races a cheaper tier of its own, which FlexInference reaches on managed keys only, for a non-streaming request carrying a duration between 3 and 10 minutes; current constraints put that floor at 3 minutes. Outside those rules a duration start_within on a claude-* model returns 400: flex_unsupported_for_anthropic on your own Anthropic key, flex_anthropic_streaming_unsupported when the request streams, or flex_anthropic_deadline_too_short under 3 minutes. Use default/priority/auto instead.

## Performance

- More than 30,000 requests asked for flex and saved about 48 percent, for about 20 percent more time to first token.
- Flex routing trades a little extra time to first token for a large cut in blended cost. The live median cost, savings, and latency across real traffic are shown on https://www.flexinference.com.
- Per-model figures - which models run the flex race, what each realizes as a saving, and what it changes in time to first token - are in the dashboard on the Savings tab, recomputed daily across all traffic.

## Pricing

- BYOK: free and unlimited - no credit card, no subscription, no per-request charge. Free routing, the flex race, and centralized cost tracking are all included.
- BYOK means provider usage bills to your own provider account - OpenAI, Gemini, Anthropic, or a cloud route like Amazon Bedrock, Google Vertex AI, Azure AI Foundry, or DeepInfra - and draws down your existing credits; FlexInference never marks it up.
- Managed Keys: instead of bringing your own key, you can have FlexInference run a dedicated, isolated provider credential for your workspace and pre-fund a cash balance. Model cost is a zero-markup pass-through at the real provider rate; the only FlexInference charge is a flat 10% fee on each top-up. Managed requests are unlimited, with no per-request charge; when the balance runs out, serving pauses with a 402 until you add funds. The cash you add is refundable; promotional credits are not. Managed serves OpenAI, Anthropic, and Gemini direct, plus Cloudflare Workers AI for open models.
- [Pricing](https://www.flexinference.com/#pricing) has the plan comparison.
- [Per-model rates](https://www.flexinference.com/pricing): the rate table for every model, direct and through the cloud routes.

## Privacy

- BYOK provider keys are encrypted at rest with AES-256-GCM and decrypted only transiently at the edge to make a proxied request.
- On BYOK, FlexInference keeps no prompt or completion content in its database, logs, or analytics, only metadata like token counts and cost.
- On Managed Keys the default is the same: we store no prompt or completion content. If your organization turns trace storage on, we store the request and response encrypted so your workspace can read them in the dashboard logs, decrypted server-side behind your own login.

## Languages

- The website (https://www.flexinference.com) and docs (https://docs.flexinference.com) are in English.

## Demos

- Three integration demos route real open-source agents through FlexInference and report the measured cost and time-to-first-token deltas: OpenCode (https://www.flexinference.com/transcripts/opencode.txt), OpenScience (https://www.flexinference.com/transcripts/openscience.txt), and OpenWork (https://www.flexinference.com/transcripts/openwork.txt).
- Each integration demo's setup instructions are an agent skill in markdown: https://www.flexinference.com/skills/opencode.skill.md, https://www.flexinference.com/skills/openscience.skill.md, and https://www.flexinference.com/skills/openwork.skill.md. The general FlexInference skill is https://www.flexinference.com/flexinference.skill.md.
- Three more recorded demos compare the same model with a start_within deadline against default routing: Gemini image classification (https://www.flexinference.com/transcripts/gemini-image-classification.txt), OpenAI browser agent (https://www.flexinference.com/transcripts/openai-browser-agent.txt), and OpenAI deep research (https://www.flexinference.com/transcripts/openai-deep-research.txt).
- Each link is the demo's full cleaned transcript as plain text; the videos themselves are embedded on https://www.flexinference.com under Demos.

## Blog

- Every post is also available as plain markdown at its URL plus .md, for agents and readers that fetch raw text.
- [Why use FlexInference](https://www.flexinference.com/blog/why-use-flexinference.md): pay about half on tasks that can wait, test models for the 10x, and read a bill that matches provider rates.
- [Great software wrestles for you](https://www.flexinference.com/blog/why-flexinference.md): why we built FlexInference and the trade it makes.
- [When to use agent keys](https://www.flexinference.com/blog/when-to-use-agent-keys.md): keys that store the deadline, the logs, and the bill for tools like Cursor, Claude Code, Codex, and OpenCode.
- [When to use the SDK](https://www.flexinference.com/blog/when-to-use-the-sdk.md): putting start_within in your own code for pipelines, background work, and side projects.
- [Errors have a new reader, and it isn't you](https://www.flexinference.com/blog/errors-for-agents.md): every error is written as the fix so a coding agent can act on it.
- [Blog index](https://www.flexinference.com/blog): all posts.

## Docs

- [Docs home](https://docs.flexinference.com/): overview and navigation for all docs pages.
- [Quickstart](https://docs.flexinference.com/quickstart): sign up, add a BYOK key, make your first deadline-aware request.
- [Authentication](https://docs.flexinference.com/authentication): how FlexInference keys and BYOK provider keys work together.
- [Flex Race Routing](https://docs.flexinference.com/deadline-routing): the start_within values and when fallback to standard happens.
- [Models](https://docs.flexinference.com/models): which OpenAI, Gemini, and Anthropic models are supported and which are flex-capable.
- [Coding agents](https://docs.flexinference.com/coding-agents): pointing Codex at FlexInference with a key that carries the deadline, for clients that cannot add a field to the request body.
- [Errors](https://docs.flexinference.com/errors): the error envelope, FlexInference-specific codes, and pass-through OpenAI errors.
- [Docs llms.txt](https://docs.flexinference.com/llms.txt): the full, auto-generated index of the docs subdomain (this file only covers the marketing site).
- Add .md to any docs URL for the same page as plain markdown, like https://docs.flexinference.com/deadline-routing.md; [docs llms-full.txt](https://docs.flexinference.com/llms-full.txt) is the whole docs corpus in one text file.

## MCP

- [MCP server](https://mcp.flexinference.com/mcp): the hosted FlexInference MCP server. Docs search and the try-it tools need no login; account tools are OAuth-gated.
- [MCP server discovery](https://www.flexinference.com/.well-known/mcp.json): machine-readable pointer to the MCP server for clients that auto-discover it.

## Legal

- [Privacy Policy](https://www.flexinference.com/privacy): what we collect, how BYOK keys are handled, and your choices.
- [Terms of Service](https://www.flexinference.com/terms): the terms governing use of the API, dashboard, and SDKs.

## Contact

- adi@flexinference.com