How enterprises can reduce generative-AI costs without slowing adoption
TL;DR
Enterprise AI becomes expensive when every request carries oversized prompts, files, histories and tool definitions through multiple agent loops. The biggest savings come from routing routine work to smaller models, engineering prompts for cache hits, retrieving only relevant context, limiting outputs and agent turns, and choosing the leanest file format that preserves the required information. Measure cost per completed task—not simply tokens per employee.
A few years ago, the enterprise AI strategy was simple: give everyone access and encourage everyone to use it.
That worked. Employees now draft documents, analyze data, write code, search internal knowledge and delegate entire workflows to AI agents. But the success has produced a new problem: the token bill.
Imagine an organization with 10,000 employees. Each employee starts 20 AI-assisted tasks per day, and each task triggers five model calls. That is one million requests every working day. If a request contains 6,000 input tokens and produces 700 output tokens, the organization processes six billion input and 700 million output tokens daily.
At August 2026 list prices for OpenAI’s GPT-5.6 Terra, that illustrative workload would cost roughly $20,400 per day—or more than $5 million across 250 working days. Tool fees, searches, retries, storage and failed agent runs would come on top. Current OpenAI pricing also illustrates an important detail: output tokens can cost several times more than ordinary input.
The answer is not to discourage AI use. It is to make AI architecture cost-aware.
The hidden prompt is larger than the visible prompt
An employee may enter a 20-word question, but the model often receives thousands of additional tokens before it sees those words. The real request can contain system instructions, conversation history, policy documents, retrieved passages, examples, tool descriptions, schemas, images and previous agent results.
This means that the user’s prompt is rarely the main problem. Persistent context is.
Agent and skill Markdown files are a good example. They often begin as concise instructions and gradually turn into internal manuals. Because they may be loaded repeatedly, every redundant explanation becomes a recurring cost. Root-level instructions should contain only rules that genuinely apply to most tasks. Specialized guidance, examples and reference material should be loaded only when needed.
OpenAI calls this pattern progressive disclosure: an agent initially sees a short skill description and loads the full instructions only after selecting that skill. The same principle should govern enterprise context generally—advertise broadly, load selectively.
Use the cheapest intelligence that passes the test
Not every request needs a frontier model with extensive reasoning.
Classification, extraction, formatting and simple summaries can often run on smaller models. Deterministic calculations and filtering may not need a language model at all. More capable models should be reserved for difficult, ambiguous or high-risk tasks, or invoked when a cheaper first attempt fails validation.
Reasoning depth should also be treated as a budget decision. Internal reasoning can consume billable output tokens even when users never see it. The right default is therefore not maximum reasoning, but the lowest setting that passes a representative quality evaluation.
Outputs need limits too. “Analyze this contract” offers no stopping point. “Return the five most material risks, each with a clause reference and recommended action, in no more than 700 words” gives the model a clear output contract. This saves expensive generation tokens while often making the result more useful.
Build prompts for reuse
Repeated context does not always have to be processed at full price. Prompt caching allows providers to reuse previously processed instructions, examples, tool definitions or documents.
But caching normally depends on a stable prefix. Shared instructions and reference material should appear at the beginning of the prompt; user-specific information, timestamps and request IDs should come later. A changing value inserted near the top can quietly destroy the cache hit for everything that follows.
For current GPT-5.6 models, OpenAI prices cached input at one tenth of ordinary input, while cache writes cost 1.25 times the standard rate. Caching can therefore pay off quickly—but only when entries are actually reused. Organizations should monitor cache reads, cache writes and hit rates rather than merely confirming that caching is enabled. OpenAI’s caching guide explains the exact-prefix behavior.
Send the information, not automatically the original file
File format can make a surprisingly large difference.
For OpenAI file inputs, a PDF may contribute both extracted text and images of its pages to the model context. PPTX and DOCX files are generally processed as text, without automatically including embedded charts and images. A PDF can therefore consume substantially more tokens than expected. The file-input documentation recommends considering these implications before processing files at scale.
If the task is to review one slide’s visual appearance, a cropped and resized PNG may be far cheaper than sending the entire presentation or a PDF export. If the task is only to check wording, clean extracted text will usually be more efficient than any visual format.
The rule is not “PNG is always cheaper.” It is: send the smallest representation that preserves the evidence required for the task.
This principle can also improve document-production workflows. Instead of generating and repeatedly revising a complete editable presentation, an agent might first create a low-cost visual preview. The native PPTX is produced only after the content and design direction have been approved.
Put agents on a budget
Agentic workflows multiply costs because planning, tool use, review and synthesis may each require another model call. Multi-agent systems add further handoffs, often carrying the same conversation history repeatedly.
Agents therefore need explicit limits: maximum turns, tool calls, retries, elapsed time and cumulative cost. Handoffs should contain a compact task description, relevant facts and constraints—not the full transcript.
Tools have a token footprint as well. Their descriptions and parameter schemas may be placed in the model context even when they are never called. Large tool catalogs should be loaded on demand, and tool outputs should return only the fields required for the decision. Filtering, joins, calculations and deduplication belong in ordinary code whenever possible.
Optimize money, not only tokens
Some of the easiest savings do not reduce token counts at all. Non-urgent extraction, translation, summarization and evaluation can move to asynchronous processing. OpenAI’s Batch API, for example, offers a 50 percent discount for workloads that can accept completion within 24 hours.
The final step is measurement. Tokens per employee reveal activity, but not value. The more meaningful metric is cost per successfully completed task, accompanied by model choice, fresh and cached input, output, reasoning, tool calls, retries and a quality signal.
The first phase of enterprise AI was about access. The next is about unit economics.
“Everyone should use AI” remains a useful ambition. It simply needs a second sentence:
Use as much intelligence as the task needs—and no more context, output or orchestration than the outcome justifies.
No comments yet