The Models Are Good Enough. Now Who Decides?
For at least the past year I’ve been making two arguments about enterprise AI. First, measuring AI productivity using tokens is a really bad idea. Second, the models are good enough. My friends over at Signal65 have just provided a lot more data to support both.
Start with tokens. Measuring AI productivity by how many tokens an enterprise consumes is like measuring a cab service by how much gasoline its cars burn. More gas burned doesn’t mean the company moved more passengers, and it certainly doesn’t mean it moved them efficiently. Tokens tell us something about compute consumption. They don’t tell us whether useful work got done.
That is what makes the work Signal65 is doing so interesting. Pinnacle scores on whether the job comes out right, how fast, and what it costs, and measures cost per correct task. That’s the same unit I arrived at from the other direction, over at Layer2C Labs.
My deskside testing has repeatedly shown that smaller models can perform a surprising amount of useful work. Depending on the workload, I’ve seen a deskside model deliver 80%, 90%, and sometimes effectively 100% of the useful capability I get from a frontier model. In my most recent lab, DeepSeek V4 Flash swept my 22-task repair pool twice on two clustered Sparks.
There are obvious qualifications. The local model may be much slower, it may require more deterministic code around it, and most importantly it eventually hits a capability ceiling.
“The models are good enough” has never meant that all models are equally capable. It means we’ve reached a point where there are large classes of enterprise work that several models can do well enough. Signal65 is giving us a much better view of those gradients.
A smaller model may be perfectly capable of one set of agentic tasks but struggle with another. A reasoning model may materially improve accuracy in one workload while providing little additional value somewhere else. A 70B model may compete surprisingly well with something dramatically larger until the task reaches a particular level of complexity.
That is much more useful than declaring one model the winner, because it helps answer the enterprise question I actually care about:
What is the least-capable model that can reliably perform this job?
Once we can answer that, we can build a very different AI architecture. You don’t necessarily start every task with the most expensive frontier model available.
You can start locally.
Let the deskside model perform the retrieval. Let it classify the data. Let it extract structured information. Let deterministic code perform the validation. Let the smaller model attempt the reasoning that it is capable of doing.
Then, when it reaches its ceiling, escalate.
In Lab 012 a free local model triaged the pool and I escalated only the five repairs it missed. That cost $1.12 against $13.79 for pushing all 22 through a frontier model, for output a validator scored identical.
That sounds simple.
It isn’t. Because now we’ve reached what I think is one of the unresolved problems in enterprise AI architecture.
Who decides when to escalate?
That’s a Decision Authority Placement Model question, and it lands on Layer 2C of my 4+1 AI Infrastructure Model, where reasoning and the agent lifecycle live. DAPM asks one thing: can you take this somewhere else?
Imagine that a Gemma-class local model can perform most of a workflow. At some point it encounters a problem that would benefit from a GPT-class model, Claude Opus, or whatever premium model happens to be appropriate for that workload.
Something has to decide that the first model is no longer good enough. Who gets that authority?
One answer is the platform provider. Perplexity’s architecture decides when a task should move from one model to another, and from the user’s perspective that’s incredibly convenient. But there’s an architectural decision hidden inside the convenience. You’ve ceded the authority to select the model. That may be exactly what you want. Enterprises should just recognize that they made the decision.
You might instead run your own model that evaluates requests and determines when escalation is necessary, fine-tuned around your workloads and risk tolerance. Now you control the policy, but the routing decision is still probabilistic. You’ve delegated it to a model you own.
Or you could retain the authority entirely and let deterministic compute decide when escalation happens. I call that Deterministic Code In The Loop, and it came out of Lab 003. Maybe a validator detects that required information is missing. Maybe the result fails a known test. Maybe policy assigns that class of problem to a premium model.
In that architecture, a model doesn’t decide that it needs a smarter model.
Your system does.
These are very different authority placements even though the result looks identical to the user. A local model starts the task. A more capable model finishes it.
There isn’t one universally correct answer, and that is the point.
Once the models are good enough, the architecture becomes less about finding the smartest model and more about deciding where authority belongs.
Signal65 is helping us understand the capability gradient. My Layer2C labs have been working the other side of it: how much useful work can we do locally before we need to borrow more intelligence? Those two lines intersect at escalation.
If a deskside model can perform 90% of a workflow, I don’t necessarily need to send the remaining 10% of that workflow’s context, data, credentials, tools, and authority to a frontier model. I may only need to escalate the unresolved problem.
That is an important distinction, and it’s why the goal shouldn’t simply be local-first AI.
It should be to escalate capability without unnecessarily escalating authority.
We are getting much better at measuring model capability. Work like Signal65’s Pinnacle effort helps turn “good enough” from a slogan into something we can actually characterize across models and workloads.
Now we need to do the same thing with authority.
- Who decides which model gets the work?
- Who decides when a task crosses an execution boundary?
- What context crosses with it?
- What tools does the next model inherit?
- And, ultimately, who decides when the job is done?
Those aren’t model benchmark questions. They’re architecture questions, and I think they’re the next ones enterprise AI has to answer.
Disclosure: Signal65 builds Pinnacle in partnership with Kamiwaza, who sponsored Layer2C Labs 015. I worked with the Signal65 team previously at The Futurum Group and have no formal economic relationship with them now. Neither Signal65 nor Kamiwaza reviewed this post.