Embeddings Are Not Anonymization

When a team builds their first RAG system, there is a moment where someone asks whether storing the company's internal documents in the vector database creates any compliance or security concerns. The answer they often get is some variant of "it's just embeddings" - as if transforming text into a 1536-dimensional floating point vector makes it legally or technically distinct from the original document.

It does not. What you put in is what is there. The embedding is a representation of the data, and that representation is increasingly reversible.

What Embeddings Actually Are

An embedding is a vector representation of a piece of text, produced by passing that text through an encoder model. The encoder has been trained to map semantically similar content to nearby regions of a high-dimensional space. Similar documents cluster together. Unrelated documents sit far apart.

This is genuinely useful for retrieval. A user query gets embedded, the vector database finds the nearest stored vectors, and the matching chunks get passed to the language model as context. RAG systems built on this architecture are practical and effective.

What the embedding is not: a hash. It is not a one-way function. The relationship between a piece of text and its embedding is not random, it is deterministic and structured. The encoder that produced the embedding captures the semantic content of the source text in ways that go well beyond retrieval similarity.

Embedding Inversion Is Not Theoretical

Research published since 2023 has consistently demonstrated that text embeddings can be inverted to recover approximations of the original text, often with meaningful accuracy for short to medium length inputs.

The intuition: an encoder trained on human language produces embeddings that reflect the statistical structure of that language. An attacker with access to an embedding, a copy of the encoder model (or a close substitute), and enough compute can run an optimization process that searches for the input text most likely to produce that embedding. The results are not perfect, but they are not noise either. Names, dates, account numbers, diagnosis codes, contractual terms - these are the kinds of tokens that tend to survive the inversion.

If you have stored embeddings of your customers' medical records because you wanted to build an intelligent document search, and an attacker exfiltrates the vector store, they are not looking at a safely obfuscated dump. They are looking at a starting point for inversion attacks against data that was probably regulated.

Access Controls in Vector Databases

Relational databases have had decades of development on access control primitives. Row-level security. Column masking. Auditable permission systems. The major vector database products were built to solve a retrieval problem first, and security was added later, at varying levels of completeness.

The typical RAG architecture loads all documents from a particular source into a single vector store, or at best segments by broad category. When a user queries the system, they retrieve based on semantic similarity, not based on what they are actually permitted to see. The marketing analyst who asks the internal assistant about Q3 performance may receive context chunks from board presentations, deal terms, or HR records, depending on what ended up in the same vector space and how the retrieval is configured.

Traditional databases enforce access control at query time. Vector retrieval enforces whatever filtering the developer remembered to implement, which is often insufficient for multi-tenant or multi-role environments. "Same namespace" is not a permission model.

The Retrieval Attack Surface

Vector databases create a new attack surface that sits between the user and the language model: the retrieval step.

If an attacker can influence what ends up in the vector store, they can influence what the language model sees when users query it. This is indirect prompt injection through the knowledge base. Malicious content embedded in a document that gets ingested into the store can surface as context during retrieval and attempt to alter the model's behavior for users who retrieve that chunk.

The scenario is straightforward: an attacker submits a document to an intake system that feeds the RAG pipeline. The document contains instructions styled to look innocuous to human review but interpretable as directives by the language model. When a user's query retrieves that chunk, those instructions appear in the model's context alongside the user's question.

This is a documented attack pattern, and it is particularly dangerous in systems where the RAG pipeline processes external or user-contributed content. The more trusted the retrieval source appears to the model, the more effective the injection.

What To Do About It

The practical mitigations are not exotic.

Treat your vector store as a database containing the sensitivity level of the data you ingested, because that is what it is. Apply data classification to documents before ingestion, and store embeddings with metadata that reflects that classification. Build retrieval filters that enforce access controls at query time, not just at ingestion.

Audit what is going in. Many RAG systems ingest broadly from internal document sources without a clear inventory of what sensitive data they are absorbing. Before you extend the ingestion pipeline to a new source, ask what the most sensitive document in that source is and whether a user should be able to retrieve it through the assistant.

For externally contributed or user-submitted content entering the pipeline, implement content review that looks for prompt injection patterns before documents are embedded and stored. This will not catch everything, but it raises the cost of the attack significantly.

Finally, do not build a security model on the assumption that users cannot reach the underlying vector store. Exfiltrating embeddings and running inversion attacks requires resources, but so does a lot of meaningful adversarial work. If the data in your vector store is regulated or sensitive, treat the store accordingly.

"It's just embeddings" is a comfortable assumption. It is also wrong.