1.1 · Why data matters: the business case

Most data programmes are sold on tools. A new warehouse. A streaming engine. A catalog. A purchase order goes through, dashboards multiply, and a year later leadership asks the uncomfortable question: where is the value?

The honest answer is almost always that the buyer skipped what sits beneath the waterline. Tools are the visible tip of the iceberg. Strategy is the mass underneath: the problem worth solving, the value proposition, the people who will do the work, the systems that will support them, the cost it takes, and the outcomes that justify it.

A useful test before any procurement decision: can you describe the nine elements below the waterline for the use case in front of you? If you cannot, the tool is not the right next step.

Tools live above the waterline. Strategy lives beneath it. If you cannot name the problem, the people, and the outcome, the tool will not save you.

1.2 · Principles: what 'good' looks like

Principles are how a programme stays coherent when the people running it change. They are tool-agnostic. They give engineers, analysts, and leaders a shared answer to "why are we doing it this way?" The ten below are adapted from IBM's modern data architecture work and are the most useful we have seen in practice.

How to use these principles:

• Print them. Put them on the wall of the room where data decisions get made.

• Reference them by number in design reviews. "This proposal conflicts with #3 and #7" is a faster conversation than re-arguing first principles every time.

• Revisit them annually. Two or three will need refining as the programme matures; the rest will hold.

1.3 · Your data strategy on one page

Long strategy documents tend to die in shared drives. A one-page version survives, and more importantly, it can be reviewed. The seven steps below sequence the work in the order that compounds.

A few notes on the sequence:

• Steps 1 - 3 are diagnostic. Skipping them is the single most common reason data programmes lose momentum in year two.

• Step 4 is where the first real value lands. Pick a use case where the decision is obvious and the data is roughly available, not the hardest problem in the company.

• Step 5 (govern) earns its place after step 4. Governing data nobody uses yet is theatre.

• Steps 6 - 7 are how a one-off win becomes a programme.

Goal first, tools later. Start tiny. Share results in plain language. Repeat what works; drop what doesn't.

1.4 · The operating framework

Strategy says what to do. The operating framework says who decides, who does, and who is accountable when things go wrong. The picture below is the cleanest single view we have of how the layers fit together.

Reading the rings, from the centre outward:

• Data Foundation; the raw material. Source systems, master data, transactions, events. Most companies inherit this layer; few control it as well as they think.

• Data Management, what turns raw data into something usable. Quality, metadata, lineage, cataloging. This is where governance does its actual day-to-day work.

• Decision Authority, who has the right to define a metric, change a definition, or override a control. This is the layer most often missing and most often the cause of conflict.

• Analytics & AI; the visible output: dashboards, advanced analytics, AI models, RAG applications. None of it is reliable without the layers below.

The four corner boxes are enablers, not afterthoughts:

• Technical Enablement; the platforms and access models that make the rings physically possible.

• Business Outcomes, what the programme is being held to. Trusted metrics, faster decisions, safe AI adoption.

• Operating Model; the human structure: roles, responsibilities, governance cadence, stewardship.

• Risk & Control, regulatory, audit, ethics, bias. Increasingly non-optional as AI scales.

If ownership is unclear, nothing above it scales. Governance is decision clarity, not documentation.

One short idea before the machinery starts. It's the one the rest of the book quietly leans on, so it's worth the page; the interlude that follows, The Decision Test.

End of Part 1 sample

This is the first of eight parts. The remaining parts will cover: Governance & Accountability; Architecture & Engineering; Quality, Trust & Observability; Metadata, Privacy & Lifecycle; AI & Analytics Enablement; People, Culture & Adoption; and Measurement & Maturity. An appendix with reference grids, checklists, and source attributions will follow.

2.1 · An eight-step implementation playbook

Governance programmes fail in predictable ways. The most common failure is starting with policy instead of with the business case. The second most common is starting with the framework instead of with the sponsor. The eight steps below sequence the work so that each one earns the right to the next.

Three notes on how to run this sequence:

• Steps 1 and 2 are sales, first to the wider business, then to a single named executive. If you cannot complete step 2, do not proceed to step 3. Without a sponsor, the programme will be killed by the first conflict.

• Step 4, benchmarking maturity, is also a politically useful artefact. It gives leadership a tangible "where we are now" and "where we want to be in 18 months" without anyone losing face. We return to maturity in Part 8.

• Step 7 is where most programmes go wrong. Guardrails should empower decisions, not block them. If your governance documents are a list of things you can't do, you'll be ignored within a quarter. Write what you can do, and what the path is to do something new.

Govern at the level of decisions, not at the level of documents. If a policy doesn't change a decision, it doesn't exist.

2.2 · Three roles that keep getting confused

Most governance friction comes from a single confusion. The words governance, ownership, and stewardship are used as if they were synonyms, and the result is owners who don't have authority, stewards who are expected to decide, and governance that has no accountability. The fix is a shared definition of which is which.

The mental model that works in practice: a steward catches a quality issue this morning. They follow up with the affected consumers and document what happened. If the issue requires a policy change, a new threshold, a different SLA; they take it to the owner of that domain. If the change affects more than one domain, the owner takes it to the governance forum. Governance writes the new rule. The owner accepts it. The steward enforces it.

The signs that the roles are confused in your organisation:

• Stewards being held responsible for decisions they have no authority to make.

• Owners signing off on decisions but not knowing who actually executes.

• Governance meetings that last 90 minutes and produce no decision.

• Three different definitions of "employment outcome" across three teams in the same programme.

Governance = who decides. Ownership = who is accountable. Stewardship = who executes. Mix them up and you get endless meetings.

2.3 · Owners, custodians, and stewards in practice

Once governance, ownership, and stewardship are separate concepts, the next step is naming the practical roles inside ownership. The three that matter most are the data owner, the data custodian, and the data steward. All three are required. They have different decision rights, different backgrounds, and different failure modes.

How to use this in conversation:

• When a quality issue lands, ask: is this a steward problem (execution), a custodian problem (technical reliability), or an owner problem (decision-making)? The answer routes the work.

• When hiring or assigning, pay attention to the typical background row. Putting an IT background into a data owner role usually fails; the role needs business authority, not technical depth.

• The failure-impact row is also useful for risk reporting. It maps directly to the kind of business consequence each role's failure produces, which makes risk conversations more concrete than the usual "data quality is bad."

Owner decides. Custodian protects. Steward maintains. Name a specific human to each, for each critical dataset. Without a named human, the role is theoretical.

2.4 · The shape of the data team

Two analysts walk into the same meeting with different numbers for "sustained employment outcomes." Both are sure. Both have the query to prove it. The meeting burns forty minutes, not because either is wrong, but because there is no answer to the only question that matters: whose definition wins? That is not a data-quality problem. It is a question about the shape of the data team. At AKG, where the same participant can appear in PICS, AKG360, and a commissioner return on the same day, this question has a real cost.

There are four common shapes, sitting on a spectrum from one central team that does everything to every business unit running its own, shown below. None is the right answer. Each trades the same two things against each other: speed inside the domains, and consistency across them. A central team gives you one version of the truth and a queue to get anything done. Fully decentralised gives every team what it wants, fast, and three definitions of "sustained outcome" by Christmas. Hub-and-spoke and federated are the attempts to buy some of both.

But the shape is the second question. The first is Wernfeldt's, and it is the one to write on the wall: the model matters less than knowing who decides when people disagree. You can run any of these shapes well, and you can run any of them into the ground; the difference is whether, when two teams collide over a definition, someone can say whose call it is without looking it up. Centralised answers "the centre, always." Federated, done well, answers "the centre breaks the tie; the domains propose." Decentralised, too often, answers nothing at all, and nothing is the answer that quietly costs you a clean set of numbers.

Choosing a shape you can actually run:

• Start more central than feels comfortable. Hewing's rule holds, begin centralised, then loosen. It is far easier to hand autonomy out than to claw it back once five teams own five pipelines.

• Let the symptom tell you which way to move. Domains idling in a central queue means you have outgrown centralised. The same metric meaning three different things means you have drifted too far the other way.

• Name the tie-breaker before you need one. The cheap moment is now, in the calm; the expensive moment is mid-argument with two VPs watching. Write down who settles a cross-domain clash, a named person or a standing forum, not "we'll figure it out."

• Don't reorganise your way out of a quality problem. Moving the boxes on the org chart does not fix data nobody trusts; it just changes who gets blamed. Part 3 makes the same point about data mesh.

THE DECISION TEST Team shape is the Decision Test pointed at the org chart. A structure earns its place only if it makes one decision clearer, who settles a tie. If it cannot answer that, you have drawn boxes, not built a model.

There is no correct shape. There is only whether, when two teams disagree, someone can say whose call it is, quickly, and without anyone having to look it up.

2.5 · A 24-month governance roadmap

Eight steps tell you what to do in what order. A roadmap tells you how long it should take, and what "good" looks like at each milestone. The phases below are drawn from regulated industry practice, financial services in particular, but the shape applies more broadly. Two-year programmes are typical; faster is possible but rare; slower usually means the sponsor has lost interest.

What each phase optimises for:

• Phase 1 (0 - 6 months) optimises for clarity. Who owns what, what's risky, what the operating model looks like. The deliverables are mostly documents, but the substance is naming people.

• Phase 2 (6 - 12 months) optimises for embedded controls. The goal is moving governance from a folder of policies to lineage, contracts, and quality checks that live in the platform.

• Phase 3 (12 - 18 months) optimises for continuous assurance. The shift is from "we reviewed it last quarter" to "we know now whether it's working."

• Phase 4 (18 - 24 months) optimises for invisibility. Machine-readable policies, embedded guardrails, executive dashboards. At this point governance has become part of the architecture rather than a function bolted on top.

Governance maturity is a 24-month story. Programmes that promise it in 6 are selling you a document. Programmes that take 5 years have lost their sponsor.

End of Part 2

Next: Part 3, Architecture & Engineering. The platform that has to carry the governance model defined here, and how the choice of stack, mesh, fabric, or lakehouse follows from the bottleneck rather than the brand.

3.1 · The layered model of a modern data platform

Almost every modern data platform, regardless of vendor, cloud, or scale, has the same seven layers. The names vary; the function does not. Understanding the layers is useful because it gives a vocabulary for diagnosing problems ("the issue is in curation, not in storage") and because it makes clear which layers are optional and which are not.

Four things worth noticing in the model:

• The top layer (Experience & Consumption) is where the business sees value. Every layer below it exists to serve that one. If a programme cannot draw a line from a layer-7 investment to layer-1 outcomes, the investment is wrong-sized.

• Layer 7 (Source Systems) is usually inherited. You don't control it, but you inherit all its problems. A meaningful share of data quality issues are actually source system issues. Naming this honestly saves arguments.

• Layers 4 (Processing & Orchestration) is where most modernisation projects get stuck. The pipelines work in dev, the schedules drift, the dependencies are unclear, error handling becomes an inbox of alerts nobody acts on.

• The cross-cutting capabilities on the right, governance, metadata, DataOps, are not add-ons. Treating them as Phase Two work is the single most reliable way to produce an unusable platform.

Every modern data platform has these layers. The difference between a good one and a swamp is whether someone owns each layer, or just hopes it works.

3.2 · Stack vs platform

Two companies can buy exactly the same tools and end up in very different places. One has a reliable data platform people trust. The other has a stack, a chain of tools held together with glue and weekend work. The difference is not the tools. It is what was designed around them.

A modern data stack is a flexible, fast way to get started. Pick a tool for ingestion, one for storage, one for transformation, one for analytics. Each tool does one job well. New teams can be productive within weeks. This is genuinely valuable; most companies should start as a stack.

What changes is what the stack becomes after a year. Once there are 30 pipelines, 200 tables, and 50 consumers, the gaps that didn't matter at the start start to matter a lot. Who owns the customer table? Why is the dashboard wrong? How much is this costing us? A stack does not answer those questions. A platform does.

The transition from stack to platform is usually triggered by one of these:

• A senior leader stops trusting a dashboard because they spotted a discrepancy.

• Cloud spend grows faster than business value and someone in finance starts asking questions.

• A compliance event forces a question of who has access to what, and the answer is unclear.

• A new analyst joins and spends their first week asking which table holds the right participant record, what worked for a team of 5 does not scale to 15.

A stack is what you build with. A platform is what makes it reliable at scale. Don't just buy tools, design how they run.

3.3 · From source system to board deck

The seven-layer platform model is engineering-focused. The view that follows is the same system seen through a governance lens, six layers between the raw data leadership inherits and the numbers leadership puts in board decks. Each layer is a place where ownership can be clear or unclear. Most organisations stop being explicit about ownership at the third layer up. They then wonder why the numbers at the top don't reconcile.

The diagnostic question that comes out of this view is simple: can you trace any number at the top all the way back to its source? Not in theory, in practice, within the next thirty minutes, with documented links, named owners, and a quality check at each transition.

Reading the stack from the inside out:

• Source Systems exist whether you admit it or not. The ERP doesn't care about your data programme.

• Lineage & Transparency is where many organisations stop and think they're governed. They have a lineage tool, therefore they have lineage. The tool is necessary; on its own it is not sufficient.

• Data Quality is what makes change safe. Without it, every schema change in the source becomes a fire drill downstream. With it, broken contracts trigger alerts before broken dashboards.

• Definitions & Ownership is the governance layer. Business definitions, metric logic, named owners, escalation paths. It is an operating model, not a policy folder.

• Trust at Scale is what leadership thinks they're paying for when they buy a platform. They're not wrong to want it; they're wrong to think it can exist without the layers underneath.

• AI, Analytics & Automation is only possible when every layer below exists. Putting an LLM on top of layer-1 data is a known way to fail expensively.

Every company has this stack. The difference is whether someone owns each layer, or just hopes it works.

3.4 · Mesh, fabric, or lakehouse, pick by bottleneck

Three architectural paradigms get talked about as if they were competitors. They are not. Each answers a different question, and the right one for an organisation depends on which question is actually being asked. The framing that works: start with the bottleneck, not with the brand.

Each paradigm changes a different dimension. Data Mesh is an organisational shift; it changes who is accountable. Data Fabric is an architectural shift; it changes how you find and access data without moving it. Data Lakehouse is a technological shift; it changes the underlying storage model. They can coexist. A company can run a lakehouse, expose it through a fabric, and govern it as a mesh. What matters is which constraint you are actually trying to relieve.

How to decide which is the right next bet:

• If the central data team is the bottleneck, work piles up, domains can't move; the answer is organisational. Mesh.

• If integration and discovery are the bottleneck, data is everywhere but hard to find or join; the answer is architectural. Fabric.

• If storage cost, duplication, or the BI-vs-ML split is the bottleneck, multiple copies, slow analytics on lake data, expensive warehouse; the answer is technological. Lakehouse.

• If you cannot name the bottleneck specifically, the answer is not any of them. The answer is to fix governance and quality first. None of these paradigms will save a programme that hasn't done the work in Parts 1 and 2.

If your strategy starts with "which tool," it's already wrong. Start with which bottleneck. The paradigm follows.

THE DECISION TEST "Which paradigm?" is the same trap as "which tool?", a purchase hunting for a justification. Name the bottleneck a paradigm would actually relieve: the central queue, the discovery problem, the storage bill. If you can't name one, no architecture is the answer, and the real work is still back in Parts 1 and 2.

3.5 · The medallion architecture

A team stands up a warehouse, points every source at it, and lets analysts build straight on top of whatever lands. For a while it works. Then a source system renames a field, a board dashboard breaks, and nobody can say whether the number was ever right, because there was no layer between the raw feed and the report where anyone checked. The medallion architecture puts those layers back. It splits the journey from raw data to analytics into three deliberate stages, Bronze, Silver, and Gold, each with one job, so that quality is built in rather than hoped for.

The names are a convention, not a product you buy. Bronze is raw, Silver is refined, Gold is ready, and data moves one way through them, each layer improving on the one before. The whole discipline is in keeping the stages separate. The moment you clean data in Bronze, or let an analyst query raw rows from Gold, the layering stops protecting you and the old problem walks back in.

What each layer is for:

Bronze is the landing zone. It takes an exact, append-only copy of the source, schema quirks and all, and changes nothing. That is the point: because it is untouched, you can always reprocess from it and recover when something downstream breaks. The one rule here is the one most often broken. Do not clean or transform data in Bronze. The day you do, you lose the faithful copy you were keeping it for. This is the data engineer's floor.

Silver is where data becomes trustworthy. Records are deduplicated, formats standardised, invalid rows filtered out, and business logic and joins applied, so that a row in Silver means what it claims to mean. This is the layer programmes skip when they are in a hurry, building Gold straight on top of Bronze, and that skip is exactly what produces dashboards that are confidently wrong. Silver is where analytics engineers and data scientists do their work.

Gold is built for decision-making. It holds aggregated, business-level metrics in shapes optimised for querying, star schemas and the curated tables behind a dashboard or a model. It is the only layer most analysts and business users ever touch, and the only one where a number is ready to be acted on. Everything beneath it exists to make Gold worth trusting.

The takeaways worth keeping:

• Bronze is raw, Silver is clean, Gold is insight. Each name describes a job, not a tool.

• Every layer has one clear purpose, and the value is in keeping them separate.

• Transforming in Bronze, or skipping Silver, quietly reintroduces the problem the architecture removes.

• A layered design scales: new sources land in Bronze without disturbing the Gold tables a board depends on.

THE DECISION TEST "Build a pipeline" is not a decision; it is plumbing. The medallion architecture earns its keep at exactly one point, the Gold layer, where a number is finally ready to change something. Bronze and Silver are cost until Gold pays them back. If you cannot name the decision a Gold table changes, you have built three careful layers to move data that nobody acts on.

End of Part 3

Next: Part 4, Quality, Trust & Observability. The architecture has to carry trustworthy data. What that means in practice: the five pillars, the modern quality framework, the observability cycle, and the data contracts that hold producers and consumers to each other.

4.1 · The five pillars of data quality

Data quality is one of those phrases that means everything and therefore nothing. The five pillars below are the shared vocabulary that makes the phrase actionable. Every dataset the business depends on must hold up across all five, and when it doesn't, the pillar tells you what kind of problem you have and what kind of check would have caught it.

How to use the pillars in a conversation:

• When someone says "the data is bad," make them name a pillar. "The numbers don't match PICS" is an accuracy problem. "Half the rows are missing" is completeness. "The dashboard says 24 hours stale" is timeliness. Naming the pillar narrows the fix.

• Treat the five as a set, not a ranking. A table can be perfectly accurate and useless because it's two weeks stale. A real-time table is useless if it conforms to a schema that nobody understands.

• Each pillar has a typical technical check that catches its failure mode. Those checks are not optional; they are how a programme moves from manual eyeballing to automated trust.

All five pillars rest on the same foundation: trust. The day stakeholders stop trusting a dashboard, every layer underneath becomes wasted work.

4.2 · The modern data quality framework

The pillars tell you what to check. The framework that follows tells you how to organise the checking so it becomes a system. At the centre is data trust; the actual goal. Around it are four quadrants: what the business gets when trust is high, what is checked technically, what is measured as a programme, and what governance capabilities the system depends on.

Reading the four quadrants:

• Business value is the outcome you are paying for. High adoption, confident decisions, reduced risk, trusted AI. These are the things leadership signed up for.

• Technical audits are what your platform does automatically. Freshness checks, volume anomaly detection, schema drift, null rates. The cost of running them is trivial; the cost of not running them shows up in incidents.

• Quality KPIs are how you measure your own programme. Data SLAs, time-to-detect, time-to-resolve, issue volume. Without these, you can't tell whether you're getting better.

• Governance stack is what makes all the above possible: catalog, lineage, contracts, stewardship. Without these layers, technical audits are noise and KPIs are guesses.

The arrows in the framework all point inward because the relationship is causal, not incidental. Business value is produced by the other three working together. Removing any one collapses the model.

Data Pipelines × Governance → Reliability → Trust → Adoption → Value. Each arrow is earned, not assumed.

4.3 · The observability cycle

Every incident is data. Caught early, it is a learning event. Caught late, it is a credibility event. The observability cycle is the loop that turns incidents into permanent improvements instead of recurring fires.

The three steps and what "good" looks like at each:

• Detect; the goal is shorter time-to-detect. "We found out when the CEO emailed" is the failure mode. Real-time alerts on freshness, volume, schema, and quality thresholds are the floor, not the ceiling.

• Diagnose; the goal is shorter time-to-resolve (MTTR). Trace upstream through lineage, identify which job, source, or contract broke. Without lineage this step takes hours; with lineage it takes minutes.

• Prevent; the goal is non-recurrence. After every incident, ask: what test, contract, or guardrail would have caught this earlier? Add it. The same incident happening twice is a process failure, not a data failure.

The metrics that matter, time-to-detect, time-to-resolve, incident recurrence, should appear in your quality KPI set. They are leading indicators of trust. If they're moving in the right direction, trust will follow. If they're flat or worsening, no amount of communication will fix the perception, because the perception is correct.

The same incident twice is a process failure, not a data failure. Every fire should leave behind a test or a contract that prevents the next one.

THE DECISION TEST An alert no one acts on is a dashboard that changes no decision, wearing a different hat. Before you add a check, ask what you'd actually do the morning it fires. If the answer is "nothing," you haven't added safety; you've added noise, and noise is what trains people to ignore the alert that finally matters.

4.4 · Data contracts

A data contract is a written agreement between the team that produces a dataset and the teams that consume it. It looks like an API contract because it functions like one. The producer commits to a schema, an SLA, and a meaning. The consumer commits to depending only on what the contract guarantees. When either side wants to change the contract, that conversation happens before the change ships, not after a dashboard breaks.

What a working contract actually contains:

• Schema, field names, types, nullability. Strict enough to fail loudly when a producer tries to ship a breaking change.

• SLAs, freshness, latency, availability. "Updated by 6am, less than two hours behind source, 99.5% uptime."

• Semantics, what each field means in business terms. The field is called customer_id; the contract says which system of record it points to and what "customer" means here.

• Quality, allowed null rates, valid value ranges, referential rules. These are the technical audits from section 4.2 codified.

• Ownership; the producer team, the on-call rota, the escalation path. Named humans.

• Versioning, how breaking changes are signalled, how consumers are notified, how the deprecation window works.

Contracts work because they are enforced in CI/CD, not in PDF files. A schema drift breaks the build, not production. An SLA violation triggers an incident workflow, not a Slack argument. This is the difference between a contract that lives in the platform and a contract that lives in a folder.

A contract is what turns "who broke the dashboard?" into "the producer's CI caught it before deployment." Enforce in code, not in conversation.

End of Part 4

Next: Part 5, Metadata, Privacy & Lifecycle. The catalog, classification, lineage, and lifecycle controls that turn a trustworthy platform into a governable one. Where the data came from, who is allowed to see it, and what happens to it when nobody needs it any more.

5.1 · The data lifecycle

Every dataset moves through six stages, whether you manage them deliberately or not. Where programmes typically focus their attention is on Create and Use, getting data in, then getting value out. The expensive failures happen at the stages that get less attention: Share (who got what), Archive (where did it go), and Delete (is it actually gone).

Three things worth being explicit about:

• Classification happens at Create. Trying to classify retroactively is expensive and unreliable. Build classification into the ingestion process so every new dataset starts with a known tier.

• Share is where most privacy incidents originate. Data leaves the boundary of one team and ends up somewhere it shouldn't be. Contracts (Part 4) and audit trails are the controls that matter here.

• Delete is the stage where regulators ask hard questions. Right-to-erasure under GDPR is not a feature; it is a process that has to demonstrate what was deleted, when, and from how many systems. If you can't answer those questions in writing, you don't actually have a Delete stage.

Manage all six stages or hope the regulators don't ask about the three you ignored.

5.2 · The metadata stack

Metadata is often discussed as a single thing. It isn't. There are at least four kinds, each valuable for different reasons, and most data catalogs ship the foundational layer and call the job done. The value compounds upward, and the higher layers are what turn a catalog from a directory into a tool people actually use.

Reading the stack from the bottom up:

• Technical metadata is the foundation, schemas, types, file formats, lineage edges. This is what most catalogs auto-ingest. Without it, nothing above it is possible.

• Business metadata is what makes the data understandable to anyone outside the engineering team. Definitions, glossaries, metric logic, named owners. This is the layer that takes deliberate human effort and that delivers the most value when it's well-maintained.

• Operational metadata is the evidence that the platform is alive. Pipeline runs, refresh times, job status, cost per query. Most teams have this data somewhere, surfacing it in the catalog is what turns it into a trust signal.

• Social metadata is the most undervalued layer. Who endorsed this dataset? Who reported an issue last week? Who actually uses it? In practice this is the metadata consumers care most about, because it's the closest thing to a recommendation engine.

Most catalogs ship technical metadata and call it a day. The investment that matters is in the business and social layers, that's where adoption lives.

5.3 · Privacy classification

Privacy controls cannot be uniform. Treating every dataset like it contains PII is expensive and wasteful; treating no datasets like they contain PII is reckless. The fix is classification: every dataset belongs to exactly one tier, and the tier drives the controls.

How classification actually works in practice:

• Classify at point of creation. Retroactive classification of a large estate is a thankless multi-quarter project. New data gets tagged correctly from day one; legacy data gets cleaned up opportunistically.

• Unclassified data defaults to the most restrictive tier. This protects against the default failure mode of "we hadn't decided yet, so we let everyone see it." At AKG, where participant records carry protected characteristics and contractual obligations, that is not a defensible position.

• Classification is a property of the dataset, not the system. Restricted data in a public BI tool is still Restricted. Internal data in an external API is still Internal. The controls follow the tier across boundaries.

• Re-classification needs a process. Data that was Internal last year may be Confidential this year because the business has changed. Build a review cadence in, annual minimum for Confidential and above.

Classification is the input to every other privacy control. Get it wrong and every downstream control fires the wrong way.

5.4 · GDPR's seven principles

The European Union's General Data Protection Regulation has set the global default for how personal data should be handled. Even organisations outside the EU find themselves answering to it, through customers, partners, regulators, or successor laws modelled on it. The seven principles below are the foundation. They are not optional, and they are not bureaucratic; they are a coherent design philosophy for how a programme treats people whose data it holds.

Why each principle matters in practice:

• Lawfulness, fairness, transparency is the foundation. If a data subject would be surprised to learn you held this data, you have a problem.

• Purpose limitation prevents scope creep. Data collected to fulfil an order should not silently become marketing fuel.

• Data minimisation is the discipline of asking "do we actually need this field?" before adding it to a form. The cheapest field to protect is the one you didn't collect.

• Accuracy is shared with Part 4's quality pillars. Wrong data about a person can cause real-world harm, credit decisions, healthcare, employment.

• Storage limitation is the policy hook for the Delete stage in section 5.1. If you cannot delete, you cannot comply.

• Integrity and confidentiality is where information security meets data governance. Encryption, access control, breach response.

• Accountability is the meta-principle. Demonstrating compliance is itself a deliverable, documentation, audit trails, decisions on the record.

THE DECISION TEST Same question, one field smaller. Before you add a column to the form, ask what someone would do with it. If the answer is nothing, you've just spared yourself a thing to encrypt, retain, and one day explain to a regulator who asks why you were holding it.

Privacy is not about avoiding fines. It is about earning the right to keep operating with data that belongs, ultimately, to other people.

End of Part 5

Next: Part 6, AI & Analytics Enablement. The layer leadership is most excited about, and the one that depends most on everything in the first five parts being in place. The maturity ladder from chaos to AI-ready, the AI model lifecycle, BI and self-service, and the discipline of telling stories with data.

6.1 · The AI maturity ladder

Most organisations describe themselves as "AI-ready." Very few are. The maturity ladder below is a more honest diagnostic, five levels from firefighting chaos to genuine AI competitiveness, with a sharp threshold in the middle that very few cross.

What each level looks like in the wild:

• Level 1, Chaos & Firefighting. Excel-driven analytics. Different teams report different numbers for the same metric. A senior analyst leaves and three months of institutional knowledge leaves with them.

• Level 2, Defined Foundations. Standards are documented but enforcement is uneven. Owners are named on paper. Data quality fixes are ongoing, and ongoing, and ongoing.

• Level 3, Governed & Trusted. This is the tipping point. Trusted data products exist. Metadata and lineage are live. Stewards are active. The business has stopped treating data as a problem to be managed and started treating it as a capability to be invested in.

• Level 4, Scalable & Automated. Self-service analytics is the norm. Data contracts are enforced in CI. New use cases ship in weeks rather than quarters.

• Level 5, Strategic & AI-Ready. LLMs and RAG run reliably on production data. Autonomous agents act on trusted information. The company has a measurable advantage that competitors find hard to copy.

Don't build Level 5 models on Level 1 data. Most failed AI programmes are not AI failures; they are foundation failures. Fix the foundation first.

6.2 · The AI governance landscape

AI governance has stopped being optional. Three regimes, one regulatory, one voluntary framework, one certifiable management system, between them shape how serious AI work has to be governed today. They are not competitors. The EU AI Act says what is required by law. NIST AI RMF says how to organise the work. ISO 42001 provides a way to demonstrate it externally. Most programmes operating at any scale will need to satisfy all three.

Practical notes on each:

• The EU AI Act applies extraterritorially. If you serve EU customers, or your model does; the risk classification applies. Unacceptable-risk uses are prohibited outright. High-risk uses come with substantial documentation, conformity assessment, and ongoing monitoring obligations.

• NIST AI RMF is voluntary in the US, but its Govern - Map - Measure - Manage structure has become a de facto international vocabulary. Many organisations use it as their internal operating model regardless of jurisdiction.

• ISO/IEC 42001 is the management-system standard, Plan-Do-Check-Act for AI. It is certifiable, which matters for customers, regulators, and procurement teams who want assurance beyond a self-attestation.

• Other regimes worth knowing about: OECD AI Principles (high-level, influential), UNESCO AI Ethics Recommendation (rights-based), NIST AI 600-1 (generative AI specifically). Most of these reinforce rather than contradict the three above.

AI governance is not red tape. It is the operating model that lets a company deploy AI without exposing itself to consequences it has not thought through.

6.3 · The AI / model lifecycle

Models are not products you ship and forget. They are systems that drift, decay, and need active management. The lifecycle below applies as much to a fraud-detection model as to a RAG-powered customer service bot. The stages are not optional, what changes between use cases is the rigour applied at each, which is in turn determined by the risk classification set at the Design stage.

What can go wrong if a stage is skipped or rushed:

• Skipping Design and the risk classification is wrong from the start. You spend validation effort on the wrong things, and you discover at deployment that the use case was actually High Risk under the EU AI Act.

• Rushing Collect leads to bias problems that surface in production. "We trained on the data we had" is the most expensive sentence in machine learning.

• Cutting corners on Validate is how models with strong test-set accuracy fail in deployment. Fairness, robustness, and explainability tests are not optional; they are how you discover the problems your accuracy metric doesn't show.

• Skipping Monitor is the most common failure. A model performing well on day one is not the same as a model performing well on day 90. Drift is real, and unmonitored drift is the source of most quiet AI failures.

"Just put GPT on it" is not a lifecycle. Treat every model, including LLM-based applications, as a system that needs design, validation, deployment, and monitoring.

6.4 · The BI journey: raw to actionable

Not every analytics workload is AI. Most of the value most organisations get from their data comes from older, less glamorous disciplines, descriptive analytics, dashboards, reporting, and the slow work of helping people make better decisions. The journey below is the path data takes from arrival to action. Each stage is a discipline of its own, and the difference between a high-functioning BI capability and a low-functioning one is whether all six stages are taken seriously.

Where most BI programmes drop the ball:

• Stages 1 - 4 are the technical journey. Most BI teams handle these well. The data lands, gets sorted, gets arranged into a model, gets shown on a dashboard.

• Stage 5, Story, is where most programmes underinvest. A dashboard without a narrative is a wall of numbers. The analyst's job is not just to show the data but to explain what it means, what changed, and what action follows. Data storytelling is a real skill, and it is teachable.

• Stage 6, Actionable, is the measure of whether any of this matters. If the dashboard does not change a decision, the dashboard is decoration. The most useful question a BI team can ask its consumers: "what would you do differently if this number changed by 20%?" If the answer is "nothing," the report is on its way to being retired.

• Self-service does not skip these stages; it distributes them. The platform supplies stages 1 - 4 as infrastructure; the consumer supplies stages 5 - 6 themselves. This works only if the consumer has the literacy to use the platform well. That is the subject of Part 7.

If a dashboard does not change a decision, the dashboard is decoration. Measure your BI programme by decisions affected, not reports produced.

THE DECISION TEST "If this number moved by 20%, what would you do?" is the same question pointed at a dashboard instead of a data request. If the honest answer is "nothing," the dashboard was never for a decision, and someone is still paying to refresh it every morning. Turn it off.

End of Part 6

Next: Part 7, People, Culture & Adoption. None of what's been described in Parts 1 - 6 happens without people who can do the work, a culture that values evidence over intuition, and a deliberate effort to bring the organisation along. The literacy ladder, the change management model, and the communication patterns that turn a data programme into a way of working.

7.1 · The data literacy ladder

Data literacy is not a switch that flips. It is a ladder people climb, and the organisation's job is to help them climb it. The four rungs below describe a progression from simply knowing the data exists to being able to answer your own questions without help. Most self-service initiatives fail because they hand Level-2 people Level-4 tools and wonder why adoption stalls.

How to use the ladder:

• Assess where the bulk of your organisation actually sits. Be honest; most people in most companies are at Aware or Skilled, not Fluent.

• Match the intervention to the rung. Awareness campaigns move people from unaware to Aware. Tool training moves Aware to Skilled. Critical-thinking and interpretation coaching moves Skilled to Fluent. Only Fluent people genuinely self-serve.

• Don't confuse tool access with literacy. Giving everyone a BI licence does not make them literate any more than giving everyone a word processor makes them writers.

• The test questions on the right are deliberately concrete. "Can a programme manager answer a new question about their caseload without filing a ticket?" is a better measure of literacy than any survey.

Self-serve analytics works only when people are Fluent. Tools don't create literacy, training and culture do.

7.2 · The data culture scale

Every person relates to evidence in a particular way, and so does every organisation. The scale below runs from the gut-feeler who decides on instinct and uses data only to justify decisions already made, to the scientist who treats their own beliefs as hypotheses to be tested. Most people, and most companies, sit somewhere in the middle.

What the scale is good for:

• Diagnosis. Where does your leadership team actually sit? A data programme reporting to a gut-feeler CEO faces a different challenge than one reporting to a scientist.

• Realistic targets. The goal is not to turn everyone into a scientist, that's neither achievable nor desirable. The goal is to move the median one step to the right. A company of reporters who become analysts is transformed.

• Spotting the trap. The sceptic is not the enemy, healthy scepticism is part of good analysis. The trap is when scepticism becomes a permanent excuse to ignore inconvenient evidence.

• Self-awareness. The most useful application is personal. Where do you sit? When was the last time data changed your mind about something you believed?

The goal is not to make everyone a scientist. It is to move the median one step to the right.

7.3 · Two ways to build a data team

Give two leaders the same headcount, the same budget, and the same tools, and you can still get opposite outcomes. The difference is mindset. The comparison below contrasts the "dashboard factory builder", who optimises for output and treats governance as an obstacle, with the "modern data leader" who optimises for impact and treats governance as an enabler.

The pattern underneath the comparison:

• The dashboard factory measures itself by what it produces. The modern data leader measures by what changes. A team can ship a hundred dashboards and change zero decisions, and the factory builder will call that a good quarter.

• The factory builder treats governance as red tape that slows them down. The modern leader treats governance as the thing that lets users find data, trust it, and act on it without friction. Same word, opposite meaning.

• The most telling row is the last one. When friction appears, the factory builder blames the culture, the stakeholders, the lack of resources. The modern leader takes control, delivers value first, earns trust, and uses that trust to scale.

• This is not about talent. The factory builder is often more technically skilled. It is about where they point that skill.

Culture is not built by declaring it. It is built by delivering value before demanding literacy. Quick wins first, then the right to ask for more.

7.4 · Communication patterns by audience

The same finding needs four different deliveries depending on who is listening. This is the skill that most distinguishes analysts whose work gets used from analysts whose work gets filed. Most analysis dies in translation, not in calculation; the number was right, but the delivery didn't fit the listener.

The principle behind the four patterns:

• Executives want the decision. Lead with the recommendation, support it with the one number that matters, and leave the method in the appendix. A single slide beats a dashboard.

• Peers and analysts want the method. They will and should challenge the approach, so show the assumptions, make it reproducible, and state the confidence levels honestly.

• Business users want to know what to do next. Plain language, tied to their actual workflow, with the next action obvious. Jargon is where you lose them.

• The wider organisation wants to know why it matters. This is the domain of narrative, a story with a before and after, carried by visuals, repeated across channels until it sticks.

Notice that the analytical work is identical across all four. What changes is the framing, and the framing is the job. An analyst who can do this well multiplies the value of every other capability in this playbook.

The finding is the same. The framing is the job. Most analysis dies in translation, not in calculation.

End of Part 7

Next: Part 8, Measurement & Maturity, the final part. How to tell whether any of this is working: the maturity model that tracks the programme's progress, the KPIs that matter, and the discipline of proving return on investment. Followed by the appendix.

8.1 · The maturity pyramid

Maturity models are useful for one reason above all: they let you have an honest, non-personal conversation about where the programme actually is. "We're at Level 2" is a more productive sentence than "the data is bad." The five-level pyramid below runs from a programme that exists on paper to one that delivers measurable strategic value.

Using the model well:

• Assess honestly. Most programmes overestimate their level. Having owners named on paper (Level 1 - 2) is not the same as having active, accountable stewards (Level 3+).

• Don't skip levels. A programme at Level 2 cannot leap to Level 5 by buying an AI tool. The levels are cumulative; each depends on the one below being solid.

• Match metrics to level. This is the most practical use of the pyramid, and the subject of the next section. Measuring strategic-value metrics at a Level-1 programme produces nothing but demoralisation.

• Use it for the sponsor conversation. "We were at Level 2 a year ago, we're at Level 3 now, here's what Level 4 requires" is exactly the kind of narrative that keeps a programme funded.

Maturity is cumulative. You cannot buy your way to Level 5; you have to climb through the levels below it.

8.2 · Metrics by maturity level

The single most common measurement mistake is using metrics that don't fit your maturity. A Level-1 programme measuring "reduction in time-to-insight for analytics teams" will report noise, because there isn't yet a stable enough foundation to move that number. The matrix below matches three kinds of metric, activity, quality output, and business impact, to each maturity level.

How to read the matrix:

• Read across a row to get a coherent metric set for your current level. At Level 1, you measure whether assets have owners, count quality complaints, and track manual fixes. That's honest and it's achievable.

• Read up a column to see how the bar rises. The activity metric goes from "% of assets with an owner" (L1) to "data product reuse and adoption rate" (L5). Same dimension, vastly more sophisticated question.

• Notice the shift in emphasis. Lower levels are dominated by activity metrics, things you do. Upper levels are dominated by impact metrics, outcomes you produce. The transition happens around Level 3, the governance threshold from Part 6.

• Set targets one level up, not five. The goal of a Level-2 programme is to hit Level-3 metrics, not Level-5 ones. Ambition is good; measuring against an unreachable bar is not.

Measure what's real for your level. Activity metrics at the bottom, business impact at the top, and a deliberate climb between them.

THE DECISION TEST A metric nobody acts on is the same waste wearing a number. Keep the few that change what someone does; let the rest go before they quietly become wallpaper on a screen no one reads.

8.3 · The KPI hierarchy

Different audiences need different metrics. The board does not want to hear about pipeline success rates; the data team cannot meaningfully report on revenue influenced. The three-tier hierarchy below sorts metrics by the question they answer and the audience they serve.

The three tiers and who they're for:

• Operational metrics answer "is the machine running?", pipeline success, freshness, quality pass rates, incident volume. These are for the data team, reviewed daily or weekly.

• Tactical metrics answer "is the programme working?", SLA compliance, resolution time, adoption, certified products. These are for data leadership, reviewed monthly.

• Strategic metrics answer "why does this programme exist?", revenue influenced, risk reduced, cost avoided, decisions accelerated. These are for the board, reviewed quarterly.

• The cardinal sin is reporting the wrong tier to the wrong audience. A board deck full of pipeline success rates signals that the programme doesn't understand its own purpose. A data-team standup focused on revenue attribution wastes everyone's time.

Match the tier to the audience. Operational for the team, tactical for leadership, strategic for the board.

8.4 · Proving return on investment

The hardest question a data programme faces is also the most important: was it worth it? The value chain below traces the path from spend to outcome through four links. The uncomfortable truth is that most programmes can prove the first link; they spent the money, and struggle with everything after it.

Working the chain honestly:

• Investment is trivially measurable. You know what you spent on platform, people, governance, and tooling.

• Capability is measurable with some effort. Trusted data, faster access, working self-service; these show up in the tactical metrics from section 8.3.

• Behaviour is where attribution gets hard. Are more decisions actually being made with evidence? This requires instrumenting how decisions get made, which most organisations don't do. It is also where the value actually begins.

• Outcome is where the value lands and where attribution is hardest. Revenue went up, but was it the data programme, the new product, the market, or the sales team? The honest answer is usually "some combination," and the discipline is to claim a defensible share rather than the whole thing or nothing.

The practical move is to instrument behaviour change, not just spend. Track decisions that data influenced. Capture the before-and-after of specific use cases. Build a portfolio of documented wins; each one a small, defensible story, rather than reaching for a single grand ROI number that no one believes. A credible collection of "this programme manager intervened earlier because of this insight, and here is what happened to that participant" beats a spurious precise figure every time.

Attribution gets harder the closer you get to value. Instrument behaviour change, build a portfolio of documented wins, and claim a defensible share, not the whole thing, not nothing.

End of Part 8

That completes the eight parts of the playbook: Strategy, Governance, Architecture, Quality, Metadata & Privacy, AI & Analytics, People & Culture, and Measurement. What remains is the appendix, a consolidated reference of the frameworks, a glossary, and source attributions.