I recommend taking metadata seriously because teams cannot govern or trust data they cannot find. Lineage, ownership, quality signals, and definitions are operational tools, not catalog decoration.
The data about trust
Metadata is the difference between data that exists and data that people can safely use.
How metadata makes data usable
Metadata is the difference between data that exists and data that people can safely use.
Storage patterns I would expect
When I talk about "Storage patterns I would expect", I am checking whether Metadata Architecture can be traced back, trusted, and used by someone making a decision.
DataHub, Amundsen, Apache Atlas, Glue Data Catalog, Microsoft Purview, Dataplex, graph databases, Elasticsearch or OpenSearch.
Useful cloud services
| Provider | Typical services |
|---|---|
| AWS | Glue Data Catalog, DataZone, Lake Formation, CloudTrail lineage signals. |
| Azure | Microsoft Purview, Data Factory lineage, Fabric OneLake catalog. |
| Google Cloud | Dataplex Universal Catalog, BigQuery lineage, and governance metadata. |
Public example worth knowing
I use "Public example worth knowing" to keep Metadata Architecture grounded in a real system, because abstract patterns are too easy to agree with and too hard to operate.
LinkedIn DataHub is the public industry example. It models datasets, ownership, lineage, schema, and governance metadata so large organizations can discover and trust data across teams.
A business example
I use "A business example" to keep Metadata Architecture grounded in a real system, because abstract patterns are too easy to agree with and too hard to operate.
A data platform scans tables nightly, captures dbt lineage, records owner and SLA, and warns users when a dashboard depends on a stale upstream table.
Operational code example
When I show "Operational code example", I want the code in Metadata Architecture to reveal the production decision, not just the syntax.
select urn,
owner,
last_successful_refresh,
freshness_sla_minutes,
case
when last_successful_refresh < current_timestamp - freshness_sla_minutes * interval '1 minute'
then 'PAUSE_PUBLISH'
else 'OK'
end as action
from metadata.datasets
where used_in_customer_reporting = true;The query turns freshness metadata into an action. Customer-facing datasets that miss their freshness SLA get paused instead of silently publishing stale numbers.
What I would check first
- Make ownership mandatory.
- Capture lineage automatically from pipelines.
- Surface freshness and quality in the same place as search.