
How to Build a Local MCP Memory Server with sqlite-vec for AI Agents
A local memory server can help an agent retrieve a previous engineering decision without placing every project note in every prompt. A typical design stores documents and their embeddings in SQLite, searches for nearby vectors, and exposes selected documents through an MCP tool.
The design has three separate questions: can the database recover its records, does retrieval find useful notes, and who receives those notes? A local database addresses only part of the last question. This guide provides a bounded implementation and validation plan; it does not provide a verified server implementation or claim an execution result.
Choose the boundary first
Need memory across sessions?
├─ No → Keep notes in the current session; a server may be unnecessary.
└─ Yes → Can the content be sent to the agent's model provider?
├─ No → Use a client and model arrangement approved for that restriction.
└─ Yes → Define what may be stored, retrieved, retained, and deleted.
Then validate storage → embeddings → MCP client integration.
Treat the database as sensitive: document text can be readable by anyone with sufficient filesystem access. Local storage does not establish end-to-end privacy. A model download, telemetry, backup, client integration, or cloud model prompt may cross the machine boundary. Inspect the actual components and their settings before storing real material.
Retrieved notes are untrusted data. A note can contain instructions that conflict with the user’s request or the agent’s operating rules. Limit what a tool returns, preserve the source and date of each note, and require the agent to treat retrieved text as evidence rather than authority to execute instructions.
Pin the storage contract
The sqlite-vec project describes a SQLite extension that stores and queries vectors in vec0 virtual tables, and warns that it is pre-v1 and may introduce breaking changes. Its sample demonstrates a nearest-neighbor query returning a distance. It does not establish the default distance metric for a particular release in the supplied evidence.
Before implementation:
- Select an exact sqlite-vec release and record its package source, integrity information, SQLite version, operating system, and architecture. Read the documentation and release notes for that release rather than assuming the repository’s current example applies unchanged.
- Verify how that release loads the extension, defines a vector column, selects a distance metric, and performs bounded nearest-neighbor queries. Do not label a returned distance as cosine similarity or assume a default metric without release-specific evidence.
- Choose a representation and dimension compatible with the embedding model. The API reference describes float32, int8, and bit vector types and provides vector length and type functions. Confirm the corresponding behavior in the pinned release.
- Plan a disposable database test with synthetic vectors whose expected ordering can be calculated independently. Close and reopen the database, repeat the query, and check rollback and invalid input behavior. Record the actual inputs, outputs, versions, and test code.
If extension loading is unavailable in the chosen SQLite build, stop and verify the build and the pinned extension’s loading instructions. Load only a trusted extension and keep generic extension loading enabled only as long as the documented procedure requires. Do not assume that a successful load in one connection establishes behavior for every connection.
Design documents, vectors, and migrations together
Give each note a stable identifier, content, project scope, provenance, and timestamps. Associate its vector with that identifier. Define how updates and deletions affect both records, and test that an interrupted write cannot leave a searchable vector pointing at missing or outdated content. Use transactions where the pinned implementation supports the required operations, and verify their behavior.
For an embedding model, record the exact model identifier and revision, library version, output dimension, preprocessing, and any normalization. Check the model publisher’s documentation and inspect actual output before fixing a schema dimension. A model name or dimension alone does not establish that two vector sets are compatible.
If the model, revision, preprocessing, or dimension changes, treat existing vectors as a migration:
- Preserve a recoverable backup and the old model metadata.
- Build replacement vectors from the authoritative documents in a separate, identifiable index.
- Validate counts, sample retrieval, and restart behavior before switching reads.
- Retain a tested rollback path until the replacement is accepted.
A cache that exists on one machine is not proof of offline startup on another. Provision the approved model artifacts in advance and test startup with network access deliberately unavailable in a controlled environment. Record whether initialization succeeds and whether any component attempts network access.
Protect the database location
Use a dedicated application-owned directory whose location and ownership are reviewed before deployment. Create new files with permissions appropriate to the operating system and threat model; verify the resulting permissions rather than assuming a process setting fixes existing files.
If a configured path points into an existing directory, inspect ownership, access controls, symlinks, and existing database files. Do not recursively change permissions or change the permissions of an arbitrary parent directory: other applications may depend on it. If the directory is unsuitable, choose or provision a dedicated one through the administrator’s normal process. Include SQLite’s associated files and backups in the access-control and deletion plan.
Expose a narrow MCP interface
The official Python SDK describes servers exposing tools, resources, and prompts, and lists stdio among supported transports. The supplied SDK excerpt documents an MCPServer example and requires Python 3.11 or later; it does not verify the earlier article’s FastMCP import, startup call, or compatible dependency versions.
For a pinned SDK release, verify its server and transport examples before writing code. A useful minimal interface has a controlled write operation and a bounded search operation. Validate input size, project scope, and result count; use parameterized database operations; and return errors without exposing sensitive paths or contents. Decide whether writes require explicit user confirmation and how notes can be inspected, corrected, exported, and deleted.
For client setup, follow the intended client’s own documentation for its installed version. Do not assume Claude Code, Claude Desktop, and Cursor share configuration syntax or paths. The Claude Code documentation says its /mcp panel shows connected servers’ tool counts and notes that local-scoped server configuration is specific to the project where it was added. Use those observations when diagnosing discovery, while checking the pinned client’s instructions for the actual stdio configuration.
Validate in layers
Use disposable, non-sensitive notes first. Save a note, stop and restart the relevant server and client, then search for it. Check the returned document, project scope, ordering, and error handling. Separately test deletion and confirm that the note no longer appears after restart.
A successful recall is a smoke test, not a retrieval-quality measurement. Build a representative labeled set with relevant and irrelevant queries, evaluate missed results and unwanted returns, and choose any distance threshold against that set for the pinned model and metric. Reevaluate after model, schema, or query changes. Measure latency and resource use on the intended workload before making capacity claims.
Primary sources
- sqlite-vec project:
vec0storage and search example, and the pre-v1 compatibility warning. - sqlite-vec API reference: vector types and inspection functions.
- MCP Python SDK: SDK capabilities, Python requirement, and documented server example.
- Claude Code MCP documentation: local scope and tool-discovery behavior.
Validation boundary: No independently versioned execution receipt, server implementation, downloadable test source, client integration result, or retrieval benchmark was supplied for this correction. Verify all release-specific procedures and record your own results before deployment.
Source excerpts reviewed November 13, 2026. Current source excerpts do not establish what documentation or software behavior was available on the original publication date.