Unity Catalog Governance

This unlocks the rest of your study of data governance and, further on, AI governance. With Unity Catalog governance understood, you can move on to Regulatory Compliance Controls (how audit trails and retention rules are enforced technically), then to AI Governance and Regulation. You will also understand how model registries, prompt registries, and MLflow Tracing fit under one control plane, which you will meet again in the book 'Practical MLflow for Generative AI on Databricks'. Without this, terms like 'governed model registry' or 'lineage' will stay vague labels. With it, you can reason about who is allowed to see which table, which model, or which prompt, and know the machinery that enforces it.

Imagine an office building where every file cabinet, every whiteboard, every model prototype, and every data report is in rooms scattered across floors. Anyone with a badge can wander in and read whatever they want. There is no list of who looked at what, no record of where a number came from, and no way to know if the report on your desk was based on last month's data or a guess. Now imagine a security office on the ground floor. Every file, every whiteboard, every report is registered there with an owner. To open any drawer, you need a badge that is checked against that registration. Every time you pull out a document, the security office writes down a log line: who, what, when. If you are curious where a number in a report came from, you can walk back through the log to see the source documents, and those sources might themselves be logs, and so on until you reach raw data. Unity Catalog is that security office, but for a data lakehouse on Databricks. It centralises three things: access control (who can see what), lineage (what a number depended on), and discovery (what exists at all). Instead of each team bolting their own rules onto their own tools, one system governs everything: tables, files, models, functions, even prompts. The key insight is that a model registry is not just a folder of saved weights; with Unity Catalog it becomes a governed asset with permissions and an audit trail, just like a table. When someone asks 'can you prove this model was approved by legal?', the answer is a query into Unity Catalog, not a scramble through email.

Think of a large hospital. Every patient record, every lab result, every X-ray image, every prescription is a piece of information. Before a unified system, each department kept its own logs. A nurse in cardiology might write results in one book, the pharmacy in another, and the radiology images in a third. To treat a patient, you would have to walk from department to department, hope the right person is there, and trust that the notebook is up to date. Worse, nobody could easily answer: 'Why is this patient on this dose of this drug?' because the chain from the original lab value to the prescription was scattered. Now imagine the hospital adopts a single electronic health record system. Every test, every image, every prescription is entered into one central system, tagged with the patient's ID, the person who entered it, and the time. The system records who opened which record and when. A doctor can see the full history of a single lab value: where it came from (the machine, the technician, the calibration), and which prescriptions later used it. The chain is visible. The same idea applies to data: Unity Catalog is the electronic health record for a company's data assets. It tracks who can access what, and it records how each result was derived. But the analogy breaks in one important way. In a hospital, the goal is patient safety and legal compliance, and the system is used by a limited set of trained staff. In a data lakehouse, the users include data scientists, analysts, and automated pipelines, and the volume of accesses is millions per day. So Unity Catalog must be automated and designed for scale, not a manual logging system. Also, in a hospital, everyone is on the same network; a lakehouse often spans clouds and regions, so the catalog must work across those boundaries. The lessons from the hospital—centralised, auditable, with clear ownership—carry over; the technical machinery is different.

A governance layer over a lakehouse providing centralised access control, lineage, and discovery across data and models, so that one permission model spans tables, files, models, and functions, and every number can be traced back to its sources.

You have notes on Data Governance as a parent concept: 'policies, ownership, catalogues and controls that determine who may access which data for what purpose, and how that is documented and audited'. Unity Catalog is a concrete implementation of those ideas on Databricks. Your notes also mention Regulatory Compliance Controls under the same concept; Unity Catalog is the technical substrate that such controls rely on. In the book you studied ('Practical MLflow for Generative AI on Databricks'), you read about MLflow Tracing, Prompt Registry and Versioning, and Model Serving APIs. Unity Catalog governs all of those: the prompt registry is not a loose folder but a governed catalog entry, and the model serving endpoints check permissions against the catalog. Your notes on Data Modeling are the prerequisite for understanding what a 'table' is in this context, and later, AI Governance and Regulation will build on the audit and lineage capabilities you learn here.