APIUp to 28% cheaper than official pricesTry the API →
Hermes
Back to News

Perplexity's Orchestrator API Routes 45+ Models Without Provider Outages

Perplexity launched the Orchestrator API : 45+ open and proprietary models behind one key. If a provider fails, Orchestrator retries on another deployment of the same model, e.g. Claude moves from Anthropic to Bedrock.

Perplexity's Orchestrator API Routes 45+ Models Without Provider Outages
Source
Perplexity
Published
Author
Perplexity
Read
6 min read

Takeaways

  • Perplexity launched the Orchestrator API : 45+ open and proprietary models behind one key.
  • If a provider fails, Orchestrator retries on another deployment of the same model, e.g. Claude moves from Anthropic to Bedrock.
  • Billing uses each model's published per-token rates, with no platform fee; requests that fail before producing output aren't charged.
  • Works with OpenAI Chat Completions, Responses and Anthropic Messages SDKs via a new base URL .
  • Preview perplexity/auto routing sends each request to a model based on plain-English task descriptions.
  • Routing around refusals raised Terminal-Bench subset solves from 38 to 41 of 63, at higher cost.

Perplexity has launched the Orchestrator API , a single endpoint for proprietary and open-weight models. It selects a provider for each request based on availability, capacity, and latency, then retries failures on another deployment of the requested model. The private-preview catalog includes more than 45 models from OpenAI, Anthropic, Google, and xAI, alongside models hosted by Perplexity.

The API exposes two routing layers. Provider routing chooses where to run a requested model, while optional automatic routing chooses a model from task descriptions supplied by the developer. Perplexity says the underlying infrastructure handles hundreds of millions of daily requests across more than 85 models and a dozen providers.

One model, several hosts

An application tied directly to one model provider inherits that provider’s outages, rate limits, and latency spikes. Orchestrator separates the requested model from its host, allowing a Claude request that fails on Anthropic, for example, to run through AWS Bedrock when the same model is available there. This can reduce user-visible failures and replace provider-specific retry code.

  • Traffic is distributed across deployments according to observed error rates, available capacity, and latency. Degraded deployments receive less traffic until they recover.
  • Provider errors, rate limits, and timeouts trigger a retry on another deployment of the same model.
  • Each attempt has limits for time to first token and total duration. Exceeding either limit counts as a failure.
  • If every deployment fails, the API returns HTTP 429 with a Retry-After header.
  • Requests rejected for rate-limit errors are not charged.

Perplexity reports that Claude and GPT requests sent through Orchestrator succeeded more often over a 30-day period than requests sent to any single provider it tracked. Daily success rates also varied less during provider incidents.

MeasurementPerplexity’s reported result
Fastest Claude Opus 5.5 providerAnthropic led on 15 days and AWS Bedrock led on 15 days.
GPT-5.6 Sol output speedAzure’s median was 27% higher than OpenAI’s.

These vendor-supplied measurements show why static provider selection can become stale, although they do not establish how Orchestrator will perform for a particular workload. Production evaluations should track success rate, first-token latency, total latency, and cost using representative prompts.

Swap the base URL

Existing clients can send OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages requests without adopting a new SDK. OpenAI-compatible clients use https://api.perplexity.ai/orchestrator/v1 , and existing Perplexity API keys work without modification.

Anthropic SDKs require https://api.perplexity.ai/orchestrator as the base URL because those clients append /v1/messages . Perplexity documents the difference in its FAQ .

Every request must include the full conversation because Orchestrator is stateless. A stable prompt_cache_key or x-session-id gives the conversation best-effort affinity with one deployment, improving the chance of reusing cached prompt prefixes. Failover can still move the request elsewhere.

The model catalog depends on the account’s usage tier. Calling GET /orchestrator/v1/models returns the models available to the current project.

Describe the job, then route it

Automatic routing, currently in preview, selects a model for each conversation turn. Developers set model to perplexity/auto and provide a routing object containing a default_model plus one to five named routes. Each route combines a plain-language task description with a model ID.

A lightweight decision model compares the conversation with those descriptions. It sees the route descriptions without seeing their assigned model names, then selects a route or returns none , which invokes the default model.

Support routeAssigned modelExample work
DefaultGLM-5.3 FlashRoutine requests such as changing an account email
billing_disputeClaude Opus 5.5Refunds, disputed charges, and escalated customers
technical_issueGPT-6.1 SolBugs, error messages, and broken integrations

Response headers expose the routing decision. x-perplexity-autoroute-route identifies the matched route, x-perplexity-autoroute-outcome records whether routing succeeded or used the default, and a confidence header reports the decision model’s probability. Setting "reasoning_effort": "auto" also lets the decision model choose low, medium, or high reasoning effort for the request.

Routing runs again on every turn, allowing one conversation to move among models as its task changes. In Perplexity’s coding example, Opus diagnoses a bug, Sol plans a multi-module refactor, and GLM Flash writes the docstring from the accumulated conversation. Each model maintains its own prompt cache, so returning to a previously used model can reuse its cached prefix.

Three more solves, at a higher cost

Perplexity also tested automatic routing as a response to predictable model refusals. Anthropic has documented cases in which Claude Fable 5.2 safeguards block legitimate cybersecurity and life-sciences work. Perplexity ran Claude Code against a self-selected 63-task subset of Terminal-Bench 4.1, a benchmark of agentic terminal tasks. Fable remained the default, while security, biology, and after_refusal routes used a GPT-6 model.

ConfigurationTasks solvedReported cost
Fable only38 of 63Lower
Either routed setup41 of 63Higher

The three additional solves represent a modest gain on a subset chosen by Perplexity, and the company labels its cost figures as estimates. The result supports a narrower use case: keep a preferred model as the default and route known refusal categories to another model. Broader cost and quality claims require evaluation against an application’s own traffic.

Published rates replace provider contracts

Orchestrator charges each model’s published per-token rate regardless of which provider serves the request. Perplexity adds no platform fee, and usage draws from the same prepaid credits as other Perplexity API products. Customers can also purchase credits through AWS Marketplace.

Project administrators can review spending by model, project, and API key, then apply model allowlists and spending caps.

Teams cannot supply OpenAI, Anthropic, or other provider keys to Orchestrator. Requests use Perplexity’s provider relationships and billing, so customers with negotiated enterprise rates lose those rates for traffic sent through this API.

Preview edges

  • Media bypasses automatic routing. Conversations containing images, audio, video, or files go directly to the configured default model.
  • Each configuration supports five routes. Route descriptions are limited to 1,000 bytes, which constrains detailed task taxonomies.
  • Model selection does not use live speed data. Provider routing already considers latency after a model has been chosen. Perplexity says automatic model selection based on speed remains in development.
  • Interrupted streams are not retried. A stream that fails after output begins ends with an in-band error. The delivered tokens remain billable.
  • Access remains gated. The Orchestrator API is in private preview, with access available by emailing api@perplexity.ai . Automatic routing has a separate preview, and its built-in default policy requires usage Tier 1 or higher.

The teams it suits

OpenRouter already offers unified access, billing, fallbacks, and routing across hundreds of open and proprietary models. Perplexity has a smaller catalog and draws on infrastructure built for its own production traffic. Its distinguishing features are same-model failover across hosts, Perplexity-hosted open-weight models, and task routing configured through plain-language descriptions.

Teams already maintaining multiple model integrations could use Orchestrator to consolidate authentication, retries, provider selection, and billing behind one endpoint. Workloads with predictable refusal categories or a mix of simple and complex requests could also benefit from the automatic-routing preview.

Provider contracts, multimodal routing, and recovery from interrupted streams remain clear constraints. A production trial should measure route accuracy, model quality, latency, cache reuse, failure rates, and total cost against the existing stack before moving critical traffic.

Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.

Read original report

Sources

  1. 01x.com
  2. 02perplexity.ai
  3. 03docs.perplexity.ai
  4. 04docs.perplexity.ai
  5. 05docs.perplexity.ai
  6. 06perplexity.ai