
LiteLLM vs Portkey vs OpenRouter: Production AI Gateway Comparison (2026)
Compare LiteLLM, Portkey and OpenRouter without invented benchmarks
Choosing an AI gateway affects credential ownership, data routing, operational control and the failure path between an agent and a model provider. No gateway is the best choice for every workload, and a managed router does not automatically provide an enforceable data-retention guarantee.
Evidence correction, November 13, 2026: The earlier comparison included an unmeasured sub-0.5 ms network overhead claim, an unverified RAM footprint, model-deprecation/successor statements, a universal Redis eviction policy, asserted hard spend caps, and untested LiteLLM/Portkey configuration snippets. It also described vendor privacy routing as a contractual guarantee without checking the particular provider agreement. Those claims and code were removed rather than presented as measurements.
This is an architecture and deployment responsibility comparison, based on current first-party product descriptions. No provider accounts, production routes, vendor price plans or real token responses were used in a laboratory comparison.
1. Three distinct gateway models
The LiteLLM Proxy documentation describes a proxy server that can expose a unified LLM API interface. Its open-source repository contains the implementation and configuration references. An operator hosting a LiteLLM proxy is also responsible for securing the deployment and upstream credentials.
The Portkey AI Gateway open-source repository describes a gateway for routing requests to AI services. Distinguish a self-hosted gateway engine from optional managed platform features. Their licensing, host requirements and feature availability should be confirmed for the exact product/release before an architecture decision.
The OpenRouter API quickstart describes a hosted API for accessing models through the OpenRouter service. It is not equivalent to running a private proxy exclusively inside the user’s own virtual private network.
| Decision question | LiteLLM Proxy | Portkey Gateway | OpenRouter |
|---|---|---|---|
| Where does the gateway run? | Operate the proxy yourself when self-hosting | Review self-hosted versus managed configuration | Managed API access to provider routing |
| Who manages gateway deployment updates? | Your deployment team | Depends on self-hosted/managed choice | Hosted service manages its gateway |
| How does a local model fit? | Investigate configured local/OpenAI-compatible endpoints | Verify applicable self-hosted routes | Do not assume direct access to a private VPC address |
| Where is the control plane? | Depends on deployed features and storage | Depends on gateway and optional platform | Provider-hosted routing and account policies |
| Who is responsible for upstream credentials? | Operator/approved secret store | Depends on route and deployment | Provider key/account policy |
| What is required before a production decision? | Verify version-specific config and auth | Verify gateway edition and features | Verify current terms, provider routing and privacy policy |
This table contains design questions, not measured latency, token throughput, RAM consumption or price comparisons. A product’s feature list does not establish that a specific configuration is enabled on the user’s account.
2. Test the actual failure and fallback policy
A gateway may route to more than one backend, but resilience is not automatic. Decide how the caller and the gateway should behave under each failure:
| Scenario | Verification the team should perform | Common wrong assumption |
|---|---|---|
| HTTP rate limit or temporary provider failure | Record exact error code, request identity, retry budget and response | Every 429/5xx must be retried indefinitely |
| Streaming response interrupted mid-output | Determine whether clients can retry without duplicate side effects | A new stream resumes from the same token |
| Fallback to a different model | Compare request parameter compatibility and required tool schema | Models support identical features and limits |
| Gateway or upstream credential revoked | Verify failures do not expose credentials and do not silently switch to a broader key | A configuration reference is equivalent to a tested secret store |
| Client request cancelled | Check whether gateway and upstream stop work or continue to bill | Cancellation guarantees zero charge |
| Partial outages in one region | Measure reachable endpoints and regulatory/data policy | A hosted edge route means data never leaves a region |
Do not paste a complete proxy YAML, SDK script or webhook configuration into production merely because a comparison claims it is supported. Current options must be checked against the pinned vendor release, and a controlled staging experiment must establish that the route succeeds.
3. Security and privacy are not one toggle
For a self-hosted gateway, secure the ingress API, upstream credentials, logs, request metadata, tenant boundaries and persistent configuration. For a hosted gateway, inspect service terms and provider-specific routing and retention settings, along with the organization’s obligations.
OpenRouter documents routing privacy controls such as a data_collection option in its privacy/routing guidance. Treat it as a documented routing control, not an independently audited promise that every applicable organization, upstream provider and logging component has the same contractual data handling. Confirm the exact account, model, provider choices and agreement before sensitive-data use.
The LiteLLM repository and Portkey repository can establish that the respective software is published, but GitHub pages alone cannot certify a particular deployment’s security, budget enforcement or audit-log retention.
4. Budget and caching controls require tests
An application may need per-user or per-project request limits and budgets, but a configuration field named “budget” is not enough to prove spending is hard capped at exactly that monetary value. Verify accounting delays, cost attribution, unknown models, cached requests, retries and any outage mode. The budget must be tested using a small authorized spend ceiling on the actual supported platform.
If Redis is used for both cache entries and durable router state, do not assume allkeys-lru is a safe general policy. Eviction of keys holding counters or leases can affect enforcement. Separate state with different durability requirements and validate eviction/failover behavior against the exact deployment.
Caching, fallbacks, streaming and request rewriting may alter data-retention expectations. Check each implementation and measure the actual application behavior before enabling it globally.
5. Reproducible comparison methodology
A serious comparison needs the same request mixture and environment, not a chart assembled from public feature pages.
- Record the exact versions, hosting topology, model/provider IDs and available features for each candidate.
- Establish a neutral, authorized workload with synthetic non-sensitive inputs: short/long prompts, streaming, tool-use schemas and controlled provider failures.
- Measure request success, retries, time to first response, end-to-end latency, output correctness and cost under the same offered load.
- Verify cancellation, partial stream recovery, rate limits and fallback including failed attempts.
- Check security separately: tenant boundaries, credential masking, logging, private routes and provider agreement.
- Publish raw test parameters and command versions if claiming benchmark outcomes; otherwise explicitly call the article an architecture comparison.
Suggested evidence table for a future real benchmark:
| Measurement | Required source | Current article |
|---|---|---|
| p50/p95 latency and streaming TTFT | Actual versioned test run | Not measured |
| RAM and CPU utilization | Process and container measurements | Not measured |
| Reliability of failover | Recorded controlled failure cases | Not measured |
| Effective budget enforcement | Authorized low-cap experiment | Not measured |
| Privacy and retention compliance | Contract/policy plus technical verification | Not established |
6. Practical selection
A team comfortable maintaining a Python proxy may evaluate self-hosted LiteLLM first; a team exploring a self-hosted gateway engine and optional platform functions may evaluate Portkey; a team seeking a hosted aggregation API may evaluate OpenRouter. These are starting points for tests, not a ranking of product quality or confirmed production performance.
Avoid universal prescriptions such as setting ASGI worker count equal to vCPU count, forcing Nginx buffering policies, indiscriminately stripping unknown model parameters or describing cloud traffic as an extra fixed latency. Each depends on deployment details and measured behavior.
Scope: The article cites first-party sources and provides a rigorous choice and test framework. It does not claim live gateway requests, measured model performance, vendor privacy guarantees or AdSense/SEO outcomes. Vendor features, pricing and terms can change.
Official primary sources
- LiteLLM Proxy product documentation
- LiteLLM open-source repository
- Portkey AI Gateway open-source repository
- OpenRouter API quickstart
- OpenRouter routing and data-collection controls
Architecture/source review: November 13, 2026. Original publication date, URL and representative image preserved.