All glossary terms
A Operating model Governance

AI operating model

Centralised against federated is the question every enterprise asks and almost nobody answers correctly, because it is not one decision. Different capabilities belong at different levels, and the organisations that get this right split them rather than choosing a side.

Definition

An AI operating model is how an organisation distributes AI capability, decision rights and accountability between a central function and its business units. It answers who builds, who decides and who carries the outcome. The common error is treating it as a single centralise-or-federate choice rather than a split made capability by capability.

What is an AI operating model?

It is the plan for who does what with AI inside a company. It decides which skills sit with a central team, which sit with each business unit, who has the authority to make decisions, and who is held responsible when something goes wrong. This is a question of how the organization is set up, not which technology to buy. It will shape how fast AI projects reach the market far more than the choice of platform will.

Three questions sit inside it, and they are routinely answered as though they were one.

The three questions inside an AI operating model and who typically answers each
Question What it settles
Who builds? Whether agents are produced centrally or by the business units that own the process
Who decides? Who approves what may go live, against which standard. Frequently centralised even where building is not
Who carries the outcome? Who answers when an agent gets something wrong. The question most often left unassigned

Those three can have different answers, and in working organisations they usually do. Build locally, decide centrally, and carry the outcome where the consequence lands is a coherent model that no single label describes, which is part of why the centralised-versus-federated framing causes so much confusion.

What are the three models?

Centralised, federated and hybrid. Each describes where delivery actually happens rather than where people report, and that distinction matters: an organisation can show a central AI team on its org chart while running a thoroughly federated delivery reality, or exactly the reverse.

Each is correct at a different stage, which is the part most comparisons leave out.

Three operating models compared across four attributes. Centralised: one team builds for all business units, fastest at the start, becomes a queue, and suits scarce skill. Federated: business units build while the centre supplies platform and standards, scales with the organisation, and requires capability in the units. Hybrid: the centre builds the first case in a domain alongside the unit and then hands over, which is where most organisations converge, with the handover as the deliverable rather than the agent.
  1. 01
    CentralisedOne team builds for everyone. Right when: the skill is genuinely scarce and you have few use cases. Fails when: demand outgrows hiring, and the queue rather than the capability becomes the constraint.
  2. 02
    FederatedBusiness units build; the centre supplies platform, standards and support. Right when: domain knowledge matters more than engineering skill. Fails when: the units lack the capability and nobody funded building it.
  3. 03
    HybridThe centre builds the first case in a domain alongside the people who own it, then hands over. Right when: both are scarce, which is most of the time. Fails when: the handover is nobody's objective, so it never happens.

The hybrid failure is worth dwelling on because it is the common one. In a hybrid model the deliverable is the handover, not the agent, and if the central team is measured on agents shipped it will rationally keep shipping them rather than transferring the capability that made it unnecessary.

Which capabilities belong where in the AI Operating Model?

Centralise whatever is identical everywhere, and push out whatever encodes one specific business. That single test settles most of the argument, because nearly every capability falls cleanly to one side of it the moment somebody asks the question out loud rather than debating it abstractly.

The split, with the reasoning rather than the assertion.

Capabilities sorted into two columns. Centralise: the platform and control plane, standards and policy, model and vendor contracts, the evaluation harness, security and identity, and skills development, because each is identical across units and expensive to duplicate. Distribute: use case selection, domain logic and prompts, what counts as a correct outcome, the decision to keep running an agent, and the profit and loss, because each encodes a specific business and does not transfer. A note reads that the second column cannot be centralised because the knowledge does not travel.
  • Centralise the platform and control plane. Registry, identity scoping, audit, cost attribution and promotion gates are identical for every unit and expensive for each to rebuild. See build vs buy for agent platforms.
  • Centralise standards, security and contracts. One negotiation, one policy set, one identity model. Duplicating these produces inconsistency rather than autonomy.
  • Distribute use case selection and domain logic. Which problems are worth solving, and what good looks like in claims handling or underwriting, live in the business and do not transfer through a requirements document.
  • Distribute the outcome. Whoever carries the consequence of a wrong action owns the decision to keep running the agent, which is the ownership gap described at agent sprawl.

The test that settles arguments quickly. For any capability, ask whether two business units doing it separately would produce meaningfully different answers. If they would, it belongs with them, because the difference is the value. If they would produce the same answer twice at twice the cost, centralise it. Most disagreements about operating models dissolve once this is applied item by item instead of debated in the abstract.

What changes in a regulated enterprise?

Decision rights pull inward harder while the building stays exactly where it was. What regulation actually constrains is who may approve, what has to be evidenced and how exceptions get handled, and not one of those three requires a central team to be writing the agents themselves.

Four things shift, and the fourth is the one that catches programmes out.

  • Approval cannot be federated. Somebody accountable must be able to state which systems are in scope and on what basis, which means one register and one standard regardless of who builds.
  • Evidence must be uniform. Per-unit audit formats make a regulator's question unanswerable at group level. The record has to look the same whoever produced it.
  • Human oversight must be assigned by name. Not to a function. Obligations of this kind attach to a person with the competence and authority to act.
  • Local exceptions must be visible centrally. A unit's justified deviation is fine; an invisible one is a finding. This is where federated models most often fail an audit, and it is a reporting problem rather than a governance philosophy problem.

The practical consequence is a specific shape. Regulated enterprises tend to land on federated delivery with centralised approval and uniform evidence, which looks like a hybrid from the outside and is more precisely a split along the decision axis rather than the building one. The regulatory detail itself sits at AI governance.

What does a mature AI Operating Model look like?

A mature model is not defined by being more centralized or more spread out. It is defined by its clarity. The company defines who builds each AI capability, who makes the decisions, and who is responsible for the results. This is done for each capability separately, not as one rule for everything. Someone is also assigned to watch for the moment when those answers need to change as the business grows.

Five markers, and none of them is a technology.

  1. The split is written down and specific

    Not "we are hybrid". A list of capabilities with a level against each, which is a document somebody can disagree with rather than a posture nobody can test.

  2. Business units can ship without asking the centre to build

    The platform and standards are self-service. The centre is in the path for approval, not for delivery, which is what stops the queue forming.

  3. Every agent has a named business owner

    Not a team, a person, and one who can decide to retire it. Estates that only grow are estates where this is missing.

  4. Evidence is produced automatically and looks the same everywhere

    Registry, traces and cost attribution come from the platform rather than from each unit's own practice, so a group-level question has a group-level answer.

  5. The model is reviewed on a schedule

    The right answer at five use cases is the wrong one at fifty. Maturity includes knowing the current model has an expiry date and having somebody responsible for calling it.

One marker deliberately absent from that list. Having a central AI team is not evidence of a mature operating model, and neither is not having one. The structure matters far less than whether the three questions have explicit, different, written answers.

How do you decide on an AI operating model, and when do you change it?

Look at what is hardest to get right now and change the model when that focus shifts. If building AI is the hard part, keep the work in a central team where the skills are concentrated. If knowing what to build is the hard part, give more control to the business units, because they understand the real problems. As a company matures, these challenges often swap places. That is why the operating model cannot stay fixed.

Four steps, and the last is the one almost nobody schedules.

  1. Establish which is scarce: skill or context

    Engineering capability moves between teams. Judgement about how underwriting actually works does not. Whichever is scarcer decides the default, and it is answerable this week.

  2. Sort capabilities with the two-units test

    Would two units doing this separately produce different answers? Different means distribute; identical means centralise. Work the list rather than debating the philosophy.

  3. Name the outcome owner for every use case

    Before anything ships. This is the cheapest step and the one that determines whether anything is ever retired.

  4. Put a review date on the model itself

    Six or twelve months, with a trigger: when the centre's queue exceeds some length, the model is already wrong. Without a scheduled review the change happens anyway, informally, as teams route around the constraint.

A closing observation about how this usually goes wrong. Most organisations do not choose a federated model, they arrive at one by having their central team become a bottleneck, at which point business units build quietly and the resulting estate is federated in delivery and invisible in governance. That is the worst of both, and it is the mechanism described at shadow AI. Federating deliberately is the same destination with the visibility intact.

Frequently asked questions about AI operating models

If your question is not here, our team will answer it directly.


Talk to a Specialist →
What is an AI operating model?

How AI capability, decision rights and accountability are distributed between a central function and the business units. It is an organisational design question rather than a technology one, and it determines delivery speed more than any platform choice. Three questions sit inside it: who builds, who decides, and who carries the outcome when an agent gets something wrong.

What operating model works best for enterprise AI?

There is no single best one, and the question is better asked capability by capability. Centralise what is identical across business units, meaning the platform, standards, security and contracts. Distribute what encodes a specific business, meaning use case selection, domain logic, what counts as a correct outcome, and the decision to keep an agent running. Most working models are a split rather than a side.

Should an AI team be centralised or federated?

It depends which resource is scarce. If the hard part is knowing how to build an agent, centralise, because engineering capability moves between teams easily. If the hard part is knowing what good looks like in claims handling or underwriting, distribute, because that judgement does not travel and a central team will produce things that pass review and fail in practice.

What are the three AI operating models?

Centralised, where one team builds for everyone and which suits scarce skill and few use cases. Federated, where business units build while the centre supplies platform and standards, which suits scarce domain context. Hybrid, where the centre builds the first case in a domain alongside the unit then hands over. They describe where delivery happens rather than where people report.

Why do hybrid operating models fail?

Because the handover never happens. In a hybrid model the deliverable is the handover, not the agent, and if the central team is measured on agents shipped it will rationally keep shipping them rather than transferring the capability that would make it unnecessary. The receiving team is never quite ready and the centre is always busy, and both of those remain true every quarter.

Which AI capabilities should be centralised?

Anything identical across business units and expensive to duplicate. The platform and control plane, meaning registry, identity scoping, audit, cost attribution and promotion gates. Standards and policy. Model and vendor contracts, since one negotiation beats five. The evaluation harness. Security and identity. Skills development. Duplicating any of these produces inconsistency rather than autonomy.

Which AI capabilities should business teams own?

Anything encoding a specific business. Which use cases are worth doing, the domain logic and prompts, what counts as a correct outcome, the decision to keep an agent running, and the profit and loss attached to it. None of this transfers through a requirements document, which is how context gets lost rather than how it gets handed over.

What is the quickest way to settle a centralise-versus-distribute argument?

Ask whether two business units doing it separately would produce meaningfully different answers. If they would, it belongs with them, because the difference is the value. If they would produce the same answer twice at twice the cost, centralise it. Most operating model disagreements dissolve once this is applied item by item rather than debated in the abstract.

How should a regulated enterprise choose between centralised and federated AI?

Split along the decision axis rather than the building one. Regulated enterprises tend to land on federated delivery with centralised approval and uniform evidence. Approval cannot be federated, because somebody accountable must state which systems are in scope and on what basis. Evidence must look the same whoever produced it. And human oversight has to be assigned to a named person rather than a function.

What does federated AI governance mean across business units?

Units build and run their own agents against one shared standard, with the centre holding the register, the policy and the evidence format. The failure mode is invisible local exceptions: a unit's justified deviation is fine, an undocumented one is an audit finding. That is a reporting problem rather than a governance philosophy problem, and it is what makes federated models fail reviews.

What does a mature AI operating model look like?

Not more centralised or more federated, but more explicit. The split is written down capability by capability rather than described as "we are hybrid". Business units can ship without asking the centre to build. Every agent has a named business owner who can decide to retire it. Evidence comes from the platform automatically. And the model itself is reviewed on a schedule.

When should you change your AI operating model?

When the constraint moves, which it does as an organisation develops. The right answer at five use cases is the wrong one at fifty. Put a review date on the model with a trigger, such as the length of the central team's queue. Without a scheduled review the change happens informally anyway, as business units route around the bottleneck and build quietly.

Keep reading

Terms that come up in the same conversation

Browse the full glossary
A

AI Center of Excellence

The central team itself: what it should own, what it must never own, and how to measure it.
A

Agent governance

What centralised decision rights have to become to actually hold: controls, not documents.
S

Shadow AI and shadow agents

What arriving at a federated model by accident produces, and why it is the worst of both.

Build locally. Decide centrally. Own the outcome where it lands.
Can your business units ship without asking the centre to build?

SERAA Cortex is the centralised half: one registry with auto-discovery whichever team or cloud an agent came from, prompts, keys and rules scoped per tenant, promotion through Dev, QA and Production, and every model and tool call recorded with the agent, the model, the user and the cost. Units build; the evidence looks the same everywhere.

Book a demo Explore SERAA Cortex