Modernization Monday is a field guide to what it actually takes to modernize your infrastructure for AI, published every other Monday. Each installment takes one piece of the puzzle and breaks it down in plain terms. This is post one of nine.
A builder never prices a house by guessing. They know what a nail costs, what a sheet of drywall costs, what an hour of framing labor costs. They know how many of each go into a house, and they know what the house sells for. That discipline has a name: unit economics. It is the practice of knowing what one unit of output costs to produce and what it earns.
The builder’s CFO isn’t worried about spending more on nails as long as more houses are going up. That same CFO is very worried if the nail bill rises because the price per nail went up, or because a growing share of nails is ending up on the floor and swept out with the trash. Keep those two problems in mind, the price of the nail and the wasted nail. We are going to see them again.
Now imagine that builder’s CEO standing in front of the board and saying, “We don’t really know what our materials cost, but the houses are beautiful.” That CEO wouldn’t be CEO for long. Yet that is how many organizations are running AI projects today.
The billable unit of AI is the token
The series introduction defined tokenomics as the unit economics of turning electricity into intelligence. A fair objection to this definition is that a power company can meter a house and bill the homeowner to the kilowatt-hour, but intelligence can’t be metered and billed, right? It can. In the world of artificial intelligence, intelligence is metered in tokens, on the way in and on the way out, and tokens can most certainly be billed. Just ask any large company whose token spend is skyrocketing (or look at my own monthly AI bill, for that matter).
Tokens are the small units of data an AI model uses to process and understand language, images, and other kinds of information. Depending on the context, a token can be part of a word, a whole word, or a single character. In English text, a token works out to roughly three-quarters of a word. Tokens are counted as they enter the model as input and counted again as they leave as output, which means they can be metered very precisely as long as the right systems are in place.
Every AI workload, whether it is training a model or answering one person’s question, follows the same pattern. It takes in human or machine input, breaks that input into tokens, runs the work, produces output as tokens, and delivers that output in a form a human or a machine can read.
Electricity and GPU time are the nails and lumber of AI. The token is the house. Electricity goes in one side of the factory. Intelligence, measured in tokens, comes out the other. Everything in between is your infrastructure, labor, and the facility that hosts it, and every layer of it either raises or lowers your cost per token. For a small project, tokenomics may not matter much as long as ROI is measured. At scale, small inefficiencies compound into millions, and tokenomics becomes the whole game.
Welcome to the AI Factory
There is a name for a building that takes in raw materials, runs them down an assembly line, and mass-produces goods for hopefully happy customers. The Industrial Revolution gifted humanity with factories. The AI revolution has gifted us the AI Factory: same idea, except the raw material is electricity and the product is intelligence.
A modern, properly equipped data center and its supporting infrastructure are the host for an AI Factory. The factory takes in electricity and data as raw materials, runs them through its machinery, the GPUs and the systems that host them, and produces intelligence in the form of tokens. The AI Factory is more than a marketing phrase. It is a change in the economics and purpose of a data center, and it is a big reason the data center business is absolutely booming. Take a drive through Ashburn, Virginia and you can see it for yourself. The data center used to be a cost center that housed applications. A data center with modern power and cooling, hosting an AI Factory, is a production facility with a specific product: efficient and accurate token output.
The reason this framing matters is that your executives most likely understand the concept of a factory even if they haven’t worked in one. Throughput. Utilization. Downtime. Unit cost. Yield. Staffing. Availability. Every question a plant manager would ask about a production line applies directly to an AI Factory. What is our output per hour? How much of our capacity sat idle last month? What does one unit cost us to make, and what is it worth? What does an hour of downtime cost us? Where are the single points of failure that would stop the line? When AI infrastructure is framed as a factory, the board conversation stops being about mysterious technology and starts being about operations, a language every leadership team already speaks.
What drives cost per token
The inputs are knowable and, more importantly, measurable. Hardware cost and hardware utilization. Power consumed, both by the compute and by the cooling that supports it. Facility costs. Network efficiency, because a GPU waiting on data is burning money while producing nothing. Software and people. Colocation facilities and data centers with modern Data Center Infrastructure Management (DCIM) tools can meter the power, cooling, and facility side of that list accurately.
The two nail problems from the top of this post are both in that list. The price of the nail is your hardware and your power. The wasted nail is utilization. Idle GPUs are one of the most expensive ways in the world to produce nothing. In factory terms, that is an assembly line standing still with the lights on. An idle GPU isn’t an asset. It is a financial liability.
When you can put numbers on those inputs, you can answer the questions that stall AI programs. Should this workload run in the public cloud, in a neocloud, in a colo, or in our own facility? What should we charge for an AI-powered product? Is this quarter’s GPU purchase defensible? Where are our inflection points? Without a cost per token, the answers to all of those questions are estimates at best. With one, they are arithmetic. In the field, I have noticed that most people are quick to ask how much something costs to buy. Almost nobody asks what it costs to operate at scale over time, and that is where AI initiatives live or die.
One more input belongs on the list, and it is the hardest to meter: the value of the output. An AI Factory can turn out a spectacular number of tokens at a great price, and if those tokens read as delusional nonsense once a human sees them, the factory has produced scrap. Not all output tokens hold the same value. In factory terms, this is yield. Understanding the unit cost without understanding the end value is how organizations make spectacular and expensive mistakes.
Picture a business decision maker comparing two proposals with their principal architect. Two partners have offered solutions for a fine-tuning environment. One is complete and purpose-built for fine-tuning. The other has the same number of servers and GPUs but leaves out the infrastructure around them.
Decision Maker: “One of these proposals is $3,000,000 cheaper than the other. Why are you trying to talk me into the $10,000,000 solution when this one costs $7,000,000 for exactly the same number and type of servers and GPUs?”
Architect: “I am not saying I want to spend $10,000,000, but I really don’t want to be responsible for wasting $7,000,000. The 10-million-dollar solution produces valuable output. The 7-million-dollar solution is missing the components our requirements call for.”
Decision Maker: “But it has the same number of GPUs!”
Architect: “It does. We just won’t be able to power them all on at the same time, and if we could, they wouldn’t be able to talk to each other at the speed the workload needs. We aren’t going to get the output in a reasonable amount of time. The assembly line will be slow.”
(The dollar figures in this exchange are round numbers for illustration only.)
For a factory to produce anything usable, every step of the assembly line has to be working, and the whole line can only move at the speed of its slowest component. In an AI Factory, the GPUs are fast. Everything else on the line has to meet the requirements of the use case. Design a system for inferencing when you need fine-tuning and you have made an expensive mistake, because the solution just won’t work. Design for training when you need inferencing and you have spent money you didn’t need to spend. The output would still be great, but your cost per token would be higher than necessary.
Establishing the vocabulary of tokenomics up front, and walking everyone involved through what different use cases actually demand of the infrastructure, puts the whole team on the same page early. That education, combined with open communication, helps to minimize or eliminate both the financial mistakes and the awkward conversations.
Terminology is cheap insurance
Before we get to what great looks like, let’s clear something up early in the series, because when you are designing complex systems, precise terminology is the cheapest insurance you can buy. Artificial intelligence is a broad field. Large language models (LLMs) are one class of AI system. ChatGPT and Claude are familiar, highly capable assistants powered by LLMs. But “AI” does not equal “LLM,” and “LLM” does not equal “ChatGPT.” Terms get misused constantly in this world, and misused terms have a way of finding their way into requirements documents.
AI is an umbrella term that covers a wide range of workloads: LLM inferencing (what most people mean when they say “AI”), enterprise RAG, fine-tuning, full model training, digital twins, computer vision, video analytics, speech recognition and synthesis, recommender systems, agentic AI, data analytics, rendering and visualization, and scientific workloads like genomics, to name a few.
Two more terms deserve special care, because they are not types of AI at all. Accelerated computing is a hardware approach for offloading work from general-purpose CPUs onto massively parallel processors like GPUs. High-performance computing, or HPC, is the decades-old discipline of large-scale scientific simulation. AI is a set of workloads that runs on accelerated computing. So is HPC. The GPU does not know whether it is folding a protein or finishing your email to your boss. Accelerated computing is the engine, while AI and HPC are two different vehicles built around it. Tell your architect you are buying “a vehicle” and you might get a muscle car when you needed a cement truck.
Why does this belong in a tokenomics post? Because terminology mistakes become design mistakes, and design mistakes at this scale are expensive. A genomics team that says “we are doing AI” when they mean HPC simulation can end up with a cluster tuned for token production that underperforms on the double-precision math their science requires. Even the unit economics differ. Tokenomics measures cost per token, but classic HPC has no tokens. HPC yardsticks are cost per simulation and time to solution. Same building, same GPUs, different product, different math. Articulate the requirement incorrectly, hand it to the technical team to design and build, and the project goes sideways. Mind the details and put in the work up front, because mediocre requirements produce very expensive implementations.
What great looks like
Successful projects don’t always look exactly alike, but in the field, they have a lot in common. Use cases are properly vetted and have measurable ROI. There is a documented process for getting them there, run by people with defined roles and open communication channels. Often, there is a Center of Excellence (CoE) for AI actively involved, and that team is cross-functional rather than a room full of technologists. That matters because the hardest questions in this post are business questions. Does the use case have ROI? Who owns the data? What is the output actually worth? Technologists can’t answer those alone. This is what the series introduction meant when it said AI-ready is a state of an organization, and not one you can buy your way into.
Not every build requires an AI Factory, but when one is needed, the organizations that get it right can tell you four numbers for any given cluster: tokens produced per second, kilowatts consumed to produce them, the all-in cost of a kilowatt-hour in their facility, and the hourly cost of the cluster itself, meaning the hardware amortized over its useful life plus the software and people that keep it running. From those four, cost per million tokens falls right out: the energy bill plus the ownership bill, divided by the tokens produced. That number becomes the yardstick for every modernization decision that follows. It is the factory’s unit cost, posted on the wall where everyone can see it. If you don’t know where to get those numbers, that is exactly why the ePlus AI Experience Center, hosted in Digital Realty’s Innovation Lab in Ashburn, Virginia, exists. Run a representative workload on real AI infrastructure and work out your own tokenomics. Someone else’s benchmark was built for someone else’s workload.
The rest of this series is about lowering cost per million tokens, one layer of the factory at a time. We will work from the intelligence stack down to the physical world: the compute, the network, the data, the security and operations that hold it all together, then the facility, the power, the cooling, and finally where the electrons come from in the first place.
If your team is working through this right now, it is exactly the kind of conversation we have every day. Get in touch and we can talk through where you are.
In two weeks: Compute and consumption models. Should you build your own AI Factory, or rent time in someone else’s?