Most people don’t “use an LLM.” They use a product that uses an LLM.
That sounds like a pedantic distinction until you look at what disappears in the middle: model identity, cost visibility, context limits, retention rules, and—most importantly—what happens to the text you just poured into the box. We’ve managed to turn the most consequential part of the system into a rounding error in the user interface. It’s a strange cultural move: we’ve become exquisitely sensitive to cookie banners while casually handing our thoughts, drafts, client details, and half-formed strategies to an opaque relay race of vendors.
Call it convenience. Call it progress. But don’t call it paranoia when someone notices the missing labels on the bottle.
There is a particular modern kind of naivety that wears sophistication like a hoodie. People will argue passionately about which smartphone camera pipeline is “more natural,” then paste an entire internal memo into an “AI meeting notes” tool without asking which model it uses, where it runs, who logs it, and for how long it persists. They can tell you the wattage of their espresso machine, yet have no idea whether the service they’re using forwards their data to a third-party model provider, caches it for “quality improvements,” or stores it in an analytics warehouse accessible to a half-dozen contractors and a customer support team.
We used to call this “data processing.” Now we call it “magic.”
The industry has learned something basic about human psychology: if you put the words “AI-powered” on the front and make the interface feel like a friendly notebook, people will behave as though they are talking to a private assistant. In reality, they are often feeding a pipeline. A pipeline with metrics, logs, traces, fallbacks, retries, and incident reports. A pipeline with sub-processors and legal clauses and security exceptions. A pipeline that might route through multiple regions, multiple vendors, multiple model versions—sometimes in a single session—because cost optimization and latency matter more than your ability to audit what just happened.
The most telling detail is that many of these tools cannot even tell you the truth in a stable way. Ask: Which model did you just use? The answer is often fuzzy (“our best model”), or conditional (“depends on load”), or proudly noncommittal (“we automatically select the optimal model”). Imagine a bank that told you, “We automatically select the optimal currency.” Or a pharmacy that said, “We automatically select the optimal substance.” You would not accept that, and you shouldn’t accept it here either—because the model is not a paintbrush. It is a probabilistic engine with specific failure modes, specific data handling, and specific costs.
Costs are the second vanishing label. Token consumption is not just a billing detail; it is a limit on reasoning depth, context retention, and determinism. When a wrapper hides the model and the token budget, it also hides why outputs degrade, why the system forgets earlier constraints, and why a “simple” task yields a hallucinated certainty. Users interpret this as personality: “It’s moody today.” No. It’s running on a cheaper route, a smaller context window, or a different system prompt that quietly changed at 3 a.m.
And then there is the issue people like to wave away with a shrug: data.
“What do you think happens to the text you send?” is not a paranoid question. It is the only adult question in the room.
If you send a draft contract, a customer list, a pricing model, a proprietary design, a security finding, or even just a candid internal critique into an unknown LLM-backed service, you are not “asking for help.” You are exporting sensitive information to an environment you do not control. The vendor may be honest and careful. The vendor may also be sloppy. The vendor may not even be malicious; they may simply be normal—meaning they run logs, keep backups, pipe traces into observability platforms, and grant access to engineers who need to debug production issues quickly. Many data leaks happen not because someone twirled a villain moustache, but because data flows to where data always flows: into systems built for reliability and iteration, not for your privacy.
The typical defense is: “But they have a privacy policy.”
Yes. And most people do not read it. More importantly, even if they read it, they often cannot interpret it. “We may retain data to improve services.” “We may use sub-processors.” “We may share aggregated information.” “We may store for a limited time.” These phrases are legal fog. The practical meaning is: you don’t know. You don’t know if your text sits in a database for 30 days, 180 days, or indefinitely in backups. You don’t know if it’s used to fine-tune anything. You don’t know if a human can access it under “support.” You don’t know whether the model provider is a second controller with its own rules. You don’t know where it is stored. You don’t know whether “deleted” means deleted, or merely unlinked.
If you’re working with anything that matters, “you don’t know” is not a tolerable default.
Here’s the provocant part: we are recreating the early web, but with higher stakes. In the 2000s we trained people to click “I agree” so they could get a free email account. Now we train them to paste their inner life into a text box so they can get a better paragraph. We’re normalizing a behavior pattern: outsource cognition without demanding accountability. The interface encourages confessional writing—“tell me everything and I’ll fix it”—while the backend is built to maximize throughput and learning. It’s a perfect mismatch: intimacy on the front, industrial processing on the back.
To be clear, this is not an argument against using LLMs. It’s an argument against using them blindly through intermediaries that are allergic to disclosure.
If a service wants access to your text, it should be able to answer, in plain language, at least these questions:
Which model(s) are used, and under what conditions can that change?
What is the data retention period for prompts and outputs, including backups?
Is any of the data used for training or evaluation, and can you opt out?
Who can access your data internally, and under what controls?
Which sub-processors receive the data, and where are they located?
What encryption and segregation exists, and what happens on account deletion?
Can you export an audit trail that shows what ran where, with what settings?
If the answers are missing, evasive, or buried, that is not “modern.” That is negligent design.
So, are we being over-cautious?
Only if you believe that words are disposable.
But words are not disposable. In a business context they encode strategy. In a personal context they encode identity. In a technical context they encode vulnerabilities. In every context they encode relationships: names, responsibilities, mistakes, conflicts, plans. LLM wrappers invite you to feed these into a system that optimizes for convenience over clarity. The fact that it often works is not proof of safety. It’s proof that the incentives are aligned to make you forget the question.
There is a simple rule that feels old-fashioned because it is: treat unknown AI services like unknown cloud services. If you wouldn’t upload the file there, don’t paste the text here. If you must use them, sanitize aggressively: remove identifiers, replace names, generalize numbers, split tasks, and keep the sensitive core on systems you can actually audit. Use the “red/yellow/green” mental model: public content is green 🟢, anonymized internal content is yellow 🟡, anything sensitive is red 🔴. Red stays under your control.
This isn’t paranoia. It’s literacy.
The most dangerous trend isn’t that people use LLMs. It’s that they use them through glassy, friendly intermediaries that hide the operational facts. We should stop praising “frictionless” AI and start demanding “inspectable” AI. We should treat model identity, token budget, retention rules, and data pathways as first-class UI elements—not as secrets guarded by marketing copy.
Because the moment you can’t see what’s in the pipe is the moment you should assume the pipe is not built for you.