One of the most interesting things happening in AI is how quickly the technology itself is getting cheaper.

Models are getting smaller and more efficient. Inference costs continue to fall. Open-source and open-weight models are improving. Some inference is moving onto PCs and even phones.

All of that is great news.

But I think it risks creating a misconception about the real cost of enterprise AI.

Having spent much of my career implementing large technology platforms inside financial institutions, I have learned something fairly simple:

The technology itself is often not the expensive part. Putting it safely into production is.

The model is only the beginning

Imagine a bank wants to deploy an AI agent to perform a relatively straightforward business process. The model works. The demo looks great.

Someone calculates the inference cost and concludes that the AI can perform the task for a fraction of what it costs today. Maybe it can.

But now let's put it into production.

Where does the data come from? What is the agent allowed to see? How is access controlled? What happens if the model makes a mistake? Who approves its actions? How do we monitor it? Can we reproduce what it did six months later for an auditor?

And then come cybersecurity, compliance, model governance, operational resilience, integration with existing systems and all the other things required to run technology inside a large regulated enterprise.

None of these are particularly exciting topics at an AI conference.

But they cost money.

And AI introduces some new problems

Traditional software is generally deterministic.

Give it the same inputs and you expect the same outputs.

AI doesn't necessarily work that way.

That creates a different set of questions for an enterprise.

How much autonomy do we give an agent? When does a human need to approve its work? How do we know when its behavior changes? Who owns the risk when it makes a decision that turns out to be wrong?

These aren't reasons not to use AI.

They are simply part of the cost of using it responsibly.

This is why I question some of the ROI assumptions

In my last article, I argued that enterprises ultimately won't care very much about tokens. They will care about the work AI performs and what that work costs.

I still believe that.

But the calculation can't simply be:

Human performs task for $50. AI inference costs $2. We saved $48.

The real calculation has to include the infrastructure surrounding that $2 of intelligence.

Data.

Integration.

Security.

Governance.

Controls.

Monitoring.

Resilience.

And, in many cases, humans who still need to supervise the process.

Suddenly the economics look somewhat different.

They may still be extremely compelling.

But we need to measure the right thing.

The irony

AI itself may continue getting dramatically cheaper.

I expect it will.

But as enterprises move from AI experiments to AI running important business processes, the cost surrounding the model may become increasingly important.

And that brings me back to the broader question I've been exploring in Inside Enterprise AI.

We are spending extraordinary amounts of capital building AI infrastructure because we expect enormous enterprise demand.

I think that demand will come.

But the ultimate economics won't be determined simply by how cheap a token becomes.

They will be determined by what it really costs to put AI to work inside an enterprise — and how much economic value comes out the other side.

AI may become cheap.

Enterprise AI won't necessarily be.