All glossary terms
A Operating model Governance

AI Center of Excellence

A central team that succeeds at building capability eventually becomes the queue everyone routes around. The good ones plan their own obsolescence from the start, because the model that fits three use cases is the bottleneck at thirty.

Definition

An AI Center of Excellence is a central team that owns the platform, standards, and reusable assets for AI work, so individual teams do not each solve the same problems on their own. The hardest question it faces is not what it does, but how much it should centralize. And the right answer changes as the organization's AI footprint grows.

What is an AI Center of Excellence?

A central team that holds the shared parts of AI work so every business unit is not solving the same problems separately. That covers the platform, the standards, the reusable components and the evaluation harness, and the interesting question is how much authority it holds over the rest.

The term is used loosely enough that two organisations can both have one and mean entirely different things. What separates them is not the label but the answer to a single question: does the CoE build the AI, or does it make it possible for others to build it? Almost every disagreement about a CoE traces back to that.

A note on scope, since two questions hide behind the same phrase. This page covers what a CoE is, which operating model fits, and how to measure one. Designing, staffing and standing one up for your organisation is a services question, covered at establish AI centers of excellence.

What are the three operating models?

Centralised, federated and hybrid, and they differ on where the work happens rather than on where the org chart puts people. Most organisations start centralised because that is the only option when the skill is scarce, and most end up hybrid whether or not anybody decided to.

The distinction that matters is delivery, not reporting lines.

Three operating models compared. Centralised shows one central team building for every business unit, with a queue forming in front of it, marked as fastest at first and the first to become a bottleneck. Federated shows business units building for themselves while the central team supplies platform, standards and support, marked as scaling but requiring capability in the business units. Hybrid shows the central team building the first case in each domain and then handing over, marked as what most organisations converge on.
Three AI Center of Excellence operating models, what each optimises for, and where each fails
Model Who builds Where it fails
Centralised The CoE builds for everyone A queue forms, and the CoE never learns any domain deeply enough
Federated Business units build, the CoE supplies the platform and standards Requires capability in the business units that frequently is not there yet
Hybrid The CoE builds the first case in a domain, then hands over The handover never happens unless somebody is measured on it

The third row is where most organisations land, and the failure noted against it is the one to watch. Handover is the step that gets postponed indefinitely, because the CoE is busy and the receiving team is not ready, and both of those things are true every quarter.

How do you decide which AI model fits in for a requirement?

Ask which resource you are short of, the ability to build, or the knowledge of what to build. Engineering skill moves between teams easily, so a central group makes sense when that is your real constraint. But judgment, like understanding how underwriting works, does not move that way. No central group will ever build up enough of that kind of deep, specific knowledge on its own.

The CoE should own everything identical across use cases and nothing specific to one.

This is the decision rule, and it is more useful than any maturity model because it is answerable this afternoon.

  • Scarce skill, abundant context. Few people can build, and the requirements are clear once explained. Centralise. The CoE can learn enough of each domain to be useful.
  • Abundant skill, scarce context. Plenty of engineers, and what counts as a correct outcome lives in the heads of people who have done the job for fifteen years. Federate, because a central team will never know enough and will produce things that pass review and fail in practice.
  • Both scarce. Hybrid, deliberately. The CoE builds the first case in a domain alongside the people who own it, and the point of that first case is the handover rather than the delivery.
  • Both abundant. You may not need a CoE at all, only shared infrastructure and a standard. This is rarer than teams think and worth testing before funding a team.

The second row is the one organisations get wrong most expensively. A central team can be given the domain knowledge or it can be given the domain, and only the second actually works. Requirements documents are how context is lost rather than how it is transferred, which is why the evaluation criteria discussed at agent evaluation are so hard for a central team to write alone.

What should a CoE own, and never own?

It should own everything identical across use cases and nothing specific to one. The moment a CoE owns a use case it has become a delivery team with a waiting list, and the shared assets that justified its existence stop being maintained.

The split is cleaner than most operating model discussions suggest.

Two columns. What a CoE should own: the platform and control plane, standards and patterns, reusable components, the evaluation harness, procurement and vendor relationships, and skills development. What it should never own: individual use cases, domain logic and what counts as correct, the business outcome and its profit and loss, and the decision to keep an agent running. A note states that owning a use case turns the CoE into a delivery team with a waiting list.
  • Owns the platform and the control plane. Registry, identity scoping, audit, cost attribution, promotion gates. Identical for every team and expensive for each to build, which is the argument at build vs buy.
  • Owns the standards and the patterns. How an agent is registered, what must be traced, which approval gates apply. Enforced through the platform rather than through documents, per agent governance.
  • Never owns the use case. The business unit does, because it carries the consequence when the agent gets something wrong.
  • Never owns what counts as correct. That is domain knowledge, and a CoE writing evaluation criteria for a process it does not run is guessing politely.

Why do CoEs become bottlenecks?

Because success generates demand faster than a central team can absorb it. A CoE that delivers well becomes the place every good idea is sent, and the queue in front of it grows at the rate the organisation gets enthusiastic rather than at the rate the team can hire.

Four things happen next, and the last one is the expensive one.

  1. 01
    The queue becomes the constraintDelivery capacity stops being the limit and prioritisation does. What follows: the CoE spends more time deciding what not to do than building, which is a governance role nobody designed it for.
  2. 02
    Standards get skipped to clear the queuePressure to deliver erodes exactly the practices the CoE exists to hold. What follows: the central team becomes the largest source of exceptions to its own standard.
  3. 03
    The shared assets stop being maintainedPlatform work is never urgent against a named delivery deadline. What follows: the reusable components decay, which removes the reason the CoE existed.
  4. 04
    Teams route around itBusiness units start building without telling anybody, because the approved route is slower than the unapproved one. What follows: the estate the CoE was created to govern is now partly invisible to it. See shadow AI.

The pattern is the same one that breaks approval boards, arriving through a different door. A central function cannot scale linearly against demand that compounds, so the choice is between deliberately federating and being federated around. Only one of those leaves you with visibility.

How do you measure the success of an AI CoE?

Count the work it did not touch. Any team scored on use cases delivered is effectively being funded to remain in the critical path, and it will remain there, because that is what the scoring rewards. The figure worth reporting is how much AI reached production across the business with the central team nowhere near it.

Five measures, and the first one changes behaviour more than the other four together.

  1. Work delivered by others on your platform

    Agents built by business units, using the CoE's platform and standards, without the CoE building them. This is the enabling metric, and it rises only when the CoE invests in making itself unnecessary.

  2. Time from idea to first working version

    Measured for the requesting team, including the wait. A CoE that builds quickly but starts in eleven weeks is slower than it looks from inside.

  3. Reuse rate of shared components

    How often a new agent uses existing patterns rather than starting fresh. Falling reuse is the early signal that the shared assets have decayed.

  4. Share of the estate that is registered

    Agents in the registry against agents discovered. A widening gap means teams are routing around the approved path, which is a measurement of the CoE's usefulness rather than of anybody's compliance.

  5. Agents retired

    Almost never tracked, and the only metric that acknowledges an estate has a lifecycle. An organisation that has retired nothing has not yet had the ownership conversation.

Settle one structural question early, because it gets awkward if you leave it for later. When you create the AI Center of Excellence, write down what would have to be true for it to eventually be disbanded.

This is not a threat. It is a design choice. Here is why it matters. A team that knows what ‘finished’ looks like naturally advances toward handing things off to others. But a team with no such definition of ‘done’ will start optimizing to keep itself around. That is not a criticism of that team. It is just what any of us would do in the same position.

Frequently asked questions about an AI Center of Excellence

If your question is not here, our team will answer it directly.


Talk to a Specialist →
What is an AI Center of Excellence?

A central team holding the shared parts of AI work so every business unit is not solving the same problems separately: the platform, the standards, the reusable components and the evaluation harness. The term is used loosely enough that two organisations can both have one and mean different things, and what separates them is whether the CoE builds the AI or makes it possible for others to build it.

What are the three AI CoE operating models?

Centralised, federated and hybrid, differing on where the work happens rather than on reporting lines. Centralised means the CoE builds for everyone. Federated means business units build while the CoE supplies platform and standards. Hybrid means the CoE builds the first case in a domain then hands over. Most organisations start centralised and end up hybrid, whether or not anybody decided to.

Should an AI CoE be centralised or federated?

It depends whether your scarce resource is skill or context. If the hard part is knowing how to build an agent, centralise, because that skill is portable. If the hard part is knowing what good looks like in claims handling or underwriting, you cannot centralise, because that knowledge does not travel and a central team will produce things that pass review and fail in practice.

Why can't a central team just be given the domain knowledge?

Because requirements documents are how context is lost rather than how it is transferred. A central team can be given the domain knowledge or it can be given the domain, and only the second works. This shows up most sharply in evaluation criteria: deciding what counts as a correct outcome in a process you have never run is guesswork, however carefully the specification was written.

What should an AI CoE own?

Everything identical across use cases. The platform and control plane, meaning registry, identity scoping, audit, cost attribution and promotion gates. The standards and patterns for how an agent is registered and what must be traced, enforced through the platform rather than through documents. Plus reusable components, the evaluation harness, vendor relationships and skills development.

What should an AI CoE never own?

Anything specific to one use case. Not the use case itself, which belongs to the business unit carrying the consequence when the agent gets something wrong. Not what counts as correct, which is domain knowledge. Not the outcome or its profit and loss. The moment a CoE owns a use case it has become a delivery team with a waiting list, and the shared assets stop being maintained.

Who decides to retire an agent?

This is the question most CoE charters leave blank. The CoE will not, because it did not carry the outcome. The business unit will not, because retirement means finding another way to do the work. So nothing gets retired and the estate only grows. Naming that owner at the start costs nothing and is almost never done.

Why do AI Centers of Excellence become bottlenecks?

Because success generates demand faster than a central team can absorb it. A CoE that delivers well becomes where every good idea is sent, and the queue grows at the rate the organisation gets enthusiastic rather than at the rate the team can hire. Prioritisation then replaces delivery as the constraint, which is a governance role nobody designed the team for.

What happens when a CoE is under delivery pressure?

Standards get skipped to clear the queue, and the central team becomes the largest source of exceptions to its own standard. Shared assets stop being maintained, because platform work is never urgent against a named delivery deadline. And business units start building without telling anybody, since the approved route is slower than the unapproved one, so the estate becomes partly invisible to the team created to govern it.

How should you measure an AI CoE?

By the work it was not involved in. Score a team on use cases shipped and you have funded it to sit permanently in the critical path, which is exactly where it will stay. Report instead on how many agents reached production across the business, built on the shared platform, with the central team absent. That figure moves only when the team deliberately makes itself unnecessary.

What else is worth tracking?

Time from idea to first working version measured for the requesting team including the wait, since a CoE that builds quickly but starts in eleven weeks is slower than it looks from inside. Reuse rate of shared components, where a fall signals decay. Share of the estate actually registered, where a widening gap measures the CoE's usefulness rather than anybody's compliance. And agents retired.

Should an AI CoE be permanent?

Write down what would have to be true for it to be disbanded, at the point it is created. Not as a threat but as a design constraint: a team that knows what finished looks like builds toward handover, and a team with no such definition optimises for its own continuation, which is what any of us would do. The model fitting three use cases is the bottleneck at thirty.

Keep reading

Terms that come up in the same conversation

Browse the full glossary
A

Agent governance

What the CoE's standards have to become to hold: controls in the runtime, not documents.
B

Build vs buy for agent platforms

The platform decision a CoE owns, and the five cases where building genuinely wins.
A

Agent sprawl

What happens when nobody owns retirement, and why it is a rate problem.

Measure what happened without you
How much of your AI work runs on your platform without your team building it?

SERAA Cortex is the layer a CoE owns and everyone else builds on: one registry with auto-discovery and version lineage, prompts, keys and rules scoped per tenant, promotion through Dev, QA and Production, and every model and tool call recorded with the agent, the model, the user and the cost.

Book a demo Explore SERAA Cortex