
SearXNG vs Tavily vs Brave Search: Web Retrieval Architecture & Setup Guide for AI Agents
Compare SearXNG, Tavily and Brave Search without made-up costs or search benchmarks
Agent search systems can use a self-hosted metasearch application, a managed search/extraction API or an API backed by a provider’s own search index. The correct choice depends on upstream permissions, data routing, search quality, content needs and effective API plans, not an unmeasured provider ranking.
Technical review, November 13, 2026: Earlier versions contained plan prices and account allowances without complete current evidence, an unpinned Docker Compose stack presented as deployable, and provider SDK/request snippets whose versions and output fields were not verified. This revision removes those operational guarantees and keeps a verifiable offline comparison method. No search service was contacted for a live benchmark.
1. The products solve different tasks
| Consideration | SearXNG | Tavily | Brave Search API |
|---|---|---|---|
| Basic model | Self-hosted metasearch front end to configured external engines | Hosted retrieval API with content/search functions | Managed API backed by Brave’s web search service |
| Data path | Operator hosts SearXNG but enabled search requests leave for external engines | Queries and results pass through Tavily | Queries and results pass through Brave’s service |
| Full page content | Separate fetching/extraction may be required | Inspect selected endpoint’s returned content and extraction options | Web Search snippets differ from any separate context/extraction product |
| Key deployment responsibility | Secure host networking, upstream usage and instance configuration | API key, rate/credit policies, allowed response contents | API key, plan, request limits, response schema |
| What this article measures | Offline JSON fields only | Offline JSON fields only | Offline JSON fields only |
The SearXNG search API reference documents the / and /search endpoints, their supported GET/POST methods and optional response formats. SearXNG sends configured search queries to external services; self-hosting its web process does not make upstream requests air-gapped.
The Tavily credit documentation describes usage credits varying by operations and search depth. This is not an authorization to apply a fixed historical price to every current subscription. The Brave Search API page documents a managed search API, with pricing and free allowances determined by the current product/plan.
2. SearXNG’s JSON 403 is conditional, not a general outage
According to the official SearXNG Search API, requesting a response format not enabled under the instance’s search.formats settings returns HTTP 403. The settings reference lists the formats an administrator can enable, including HTML, JSON, CSV and RSS. A 403 from an upstream engine, a reverse proxy and a deliberately disabled JSON format are not interchangeable diagnoses.
For a SearXNG instance that is already authorized, installed and bound appropriately, a bounded inspection query is:
# Replace with the approved instance address after checking the listening port.
# This example requests the response, but does NOT start a container.
curl --fail --silent --show-error --get \
'http://127.0.0.1:8888/search' \
--data-urlencode 'q=SQLite WAL documentation' \
--data-urlencode 'format=json'
If it returns 403, first verify the actual instance’s search.formats, proxy policy and request logs. Do not disable the limiter, publish a new host port or replace authentication just to satisfy a tutorial. To set up your own instance, use the current upstream SearXNG repository and its versioned deployment/configuration instructions, then test the selected image digest and network boundary. This article does not provide an untested all-in-one Compose deployment.
A search engine’s CAPTCHA, terms of use or anti-bot restriction cannot be assumed removable by changing proxy IPs. Use only permitted upstream engines and appropriate rate limits.
3. Managed search requires endpoint-specific contracts
Tavily: The official Tavily credit guide differentiates operations and credit use by endpoint/method. For full content versus summaries, first consult the Tavily Search endpoint reference, check the current request parameters and record the actual returned JSON fields. This article intentionally provides no unverified universal SDK script, token estimate or claim about a particular HTML extraction library.
Brave: The official Brave Search API is a managed search product. A Web Search response, an LLM context product and fetching a complete publisher page are separate concerns. Before implementing a client, choose the exact supported endpoint and plan, then check request headers, API version, error handling and response schema in its current official documentation. No “strict SLA” or hard uptime guarantee is claimed without contract evidence.
Neither product automatically satisfies your organization’s privacy or data-retention obligations. Verify provider terms, request logging, region rules and credentials before sending anything sensitive.
4. Prices must be evaluated from the actual plan
Do not mix a historical headline dollar rate from one product with credits, service tiers or introductory allowances from another and call the resulting dollar total a measured benchmark.
A defensible monthly estimate requires:
- Number of requests by endpoint and search depth (basic, advanced or extraction as applicable).
- The exact dated price schedule and credit-consumption rule for each account/product plan.
- Included credits, reset periods, overage charges and applicable rate limits.
- Retries, failures, duplicate searches, pagination and content extraction as separate cost drivers.
- Conversion of credits into currency only where the applicable plan defines that conversion.
The arithmetic is simply billable operation units × applicable effective unit price, after accounting for allowances and excluded operations. This guide does not assert the earlier $0.008/credit, $5/1,000 Search requests or monthly free-credit figures as universally current for every account. Compare current invoices and official account terms, not guessed amounts.
5. A reproducible, source-backed offline JSON comparison
Our published offline response comparison source is a site-authored, Python-standard-library tool, also available from the tools index. Its exact source accepts three local JSON objects representing SearXNG, Tavily and Brave Web Search responses. It reports fields such as HTTP URL count, duplicate URLs, snippet length and pairwise URL overlap.
The source contains a --self-test that checks nine synthetic conditions, including response layouts, malformed input, overlap and local JSON loading. This evidence proves what the published code implements, not a live provider test or an independently measured search-quality comparison.
Inspect the downloaded source before running it:
curl --fail --silent --show-error \
'https://agentic-pulse.dev/tools/search_provider_offline_compare.py' \
-o search_provider_offline_compare.py
# Review the downloaded file and ensure it matches the version you approved.
less search_provider_offline_compare.py
# Run only after source review; this self-test does not contact providers.
python3 search_provider_offline_compare.py --self-test
To use previously saved authorized, non-sensitive JSON response files from the supported provider formats, the local tool accepts:
python3 search_provider_offline_compare.py \
--searxng ./searxng_response.json \
--tavily ./tavily_response.json \
--brave ./brave_response.json
The normalizer retains the URL query component, lowercases the hostname, removes www. and trims trailing path slashes. This is not search-provider relevance scoring or canonical URL verification. Do not use URL overlap as a substitute for a human answer-quality rubric.
For an actual fair benchmark, collect the same authorized public questions with the same country/language parameters and cutoff dates, then separately measure latency, successful responses, blocked requests, API credits, content coverage and evaluation by qualified reviewers. Never insert made-up request timings into a comparison table.
6. Which one should an agent project test first?
A team able to maintain host operations and comply with upstream search policies may start by evaluating SearXNG. An application requiring a hosted search/extraction interface may evaluate Tavily. A team that wants an API associated with Brave’s independent search service may evaluate Brave. These are starting hypotheses, not test-based ranks or promises of equal retrieval quality.
Record a decision based on your real approved workload, account terms, effective privacy boundary, search result relevance and maintenance cost. If a feature or price cannot be checked, mark it unverified, not “best”.
Primary evidence
- SearXNG Search API response format and 403 conditions
- SearXNG allowed search result formats
- Tavily API credit accounting
- Tavily Search endpoint
- Brave Search API product
- Published offline comparison code
Source review November 13, 2026. No real provider account, access key, API pricing invoice or production search benchmark is claimed.