Model Extraction Attacks: Your Proprietary AI Is Being Stolen Through Your Own API
You spent eighteen months and several hundred thousand dollars training a model. It is now powering your product, outperforming competitors, and embedded in your competitive moat. You exposed it through an API. An attacker is querying that API right now, systematically, and in a few weeks they will have a model that behaves almost identically to yours.
They paid for cloud compute. You paid for everything else.
What Model Extraction Actually Is
Model extraction - sometimes called model stealing - is the process of reconstructing a machine learning model's behavior by querying it through its API. The attacker does not need access to your weights, your training data, or your infrastructure. They only need an endpoint that accepts inputs and returns outputs.
The method is straightforward. Design a set of queries that systematically probe the model's decision boundaries. Feed them to the API. Record the inputs and outputs. Train a new model on that query-response dataset. If the queries cover the input space well enough and you run enough of them, the resulting model captures most of the original's behavior.
For classification models, the attacker needs to map the boundaries between output classes. For regression or generative models, the process is analogous: construct inputs that sample the behavior space, record outputs, train a substitute. The sophistication of the attack scales with the complexity of the model. Simpler models can be extracted more completely. Complex language models can be extracted well enough to be commercially useful.
Why This Matters and Who It Affects
The most direct impact is intellectual property theft. If your model represents proprietary training data, a novel architecture, or expensive domain expertise, a high-fidelity substitute represents the same competitive advantage without the cost. A competitor who cannot afford your training budget can afford a few weeks of API queries.
The secondary impact is privacy. Models trained on sensitive data sometimes memorize that data in recoverable form. An extracted substitute model may carry traces of that memorization. Extraction can be a vehicle for extracting training data you did not intend to expose, compounding the IP problem with a privacy one.
The third impact is adversarial attacks. Building adversarial examples - inputs crafted to cause misclassification - is much easier when you have local access to the model. An attacker who extracts your model can probe it offline for adversarial inputs, then deliver those inputs to your production API with high confidence they will work. Model extraction is often the precursor to adversarial attacks, not the end goal.
What Defenses Most Organizations Are Not Running
Rate limiting is the most commonly deployed control and the least effective on its own. A rate limit of a hundred requests per minute sounds restrictive. A sophisticated extraction campaign that needs a million queries spreads across multiple IP addresses, API keys, and time periods. A rate limit designed for abuse prevention is not designed to stop a determined extraction campaign.
Query logging exists in most deployments but is rarely reviewed with extraction in mind. Logging what comes in and what goes out is the raw material for detecting extraction. What organizations almost never do is analyze that log data for the patterns that indicate systematic probing: narrow distributions of inputs that gradually sweep a space, outputs that get recorded with unusual regularity, client behavior that looks nothing like a user but very much like an automated sampler.
Output confidence scores are a gift to attackers. When a classification model returns not just its prediction but how confident it is in that prediction, the attacker gets far more signal per query. Confidence scores directly reveal the model's probability estimates, making the extracted model substantially more accurate. Returning hard labels rather than probability distributions significantly degrades the quality of an extracted model. Most deployments return confidence scores because users find them useful. It is worth asking whether that tradeoff has been made deliberately.
Watermarking is underused. ML model watermarking embeds a detectable signature into the model's behavior - a set of specific input-output pairs that behave in a characteristic way for the watermarked model. If someone extracts your model and you later encounter it, you can query it with your watermark inputs and confirm it was derived from yours. This does not prevent extraction but it provides post-hoc attribution. Courts are beginning to see cases involving model theft where watermark evidence is being submitted. The technology exists. The deployment rate is low.
The Signals Worth Looking For
Extraction campaigns have detectable signatures if you know what to look for.
Query distribution anomalies: legitimate users query a model in ways that reflect actual use cases. The input distribution from real users is clustered around realistic scenarios. Extraction campaigns often produce inputs that look more uniformly distributed across the input space, because the attacker is trying to cover the space, not solve a real problem. Inputs that look synthetic, inputs that probe the edges of the input space, inputs with unusual statistical properties - these are worth flagging.
API key behavior: legitimate API keys belong to identifiable customers or internal services. A key that queries systematically, does not resemble any known use case, and queries volumes inconsistent with its stated purpose deserves attention. The same extraction campaign run from multiple keys still has correlated behavior if you analyze query timing, structure, and distribution.
Query-response correlation: an attacker building a training set records inputs with their corresponding outputs. They may re-query the same or very similar inputs to verify consistency. Repeated or near-duplicate queries that otherwise have no operational explanation are a signal.
None of these signals are perfect. An attacker who knows you are looking will structure the attack to avoid the obvious patterns. But most extraction campaigns are run by opportunists who are not expecting active monitoring. Basic anomaly detection on query patterns catches a surprising amount of the opportunistic end of the threat.
What Actually Helps
Treat your model API the same way you treat any sensitive data API. Access control, logging, anomaly detection, rate limiting. The fact that the model returns predictions rather than records does not change the threat model.
Return hard labels, not probabilities, when the use case permits. Some applications need confidence scores. Many do not, and shipping them by default because the framework produces them is a configuration choice worth revisiting.
Monitor for extraction-pattern behavior specifically. This requires query log analysis tooling that is different from what standard API observability covers. The metric to watch is not error rates or latency but the distribution and correlation of inputs over time.
Watermark models before exposing them. Especially high-value models, especially those exposed to external parties. The implementation cost is low relative to the protection value when a dispute arises.
Have a legal and IP framework for what to do when you detect extraction. The technical detection is the easy part. What your organization does with that finding - notification, legal action, rate-limiting the suspected actor - should be defined before you find an extraction campaign in progress, not after.
Your model is a product of work that cost real resources. The API that exposes it is the attack surface. Whether that attack surface is getting the same security attention as the rest of your infrastructure is a question worth having an answer to.