WALMART DATA PLATFORMWALMART DATA PLATFORM

Raw retail data — customers, stores, products, orders, reviews — lands in PostgreSQL and S3, then flows through a medallion architecture into analytics-ready gold tables. Bronze holds untouched source data via Unity Catalog external locations; silver deduplicates and conforms it with incremental dbt models; gold delivers dimension tables with full SCD Type 2 history alongside an order-item-grain fact table.

The whole pipeline runs as a single containerized Airflow DAG — CeleryExecutor, Redis broker, Postgres metadata store — that triggers ingestion, dbt runs, snapshots, and data-quality tests in sequence with freshness checks along the way. dbt-databricks handles the transformation layer against a Databricks Lakehouse, so every model is tested, documented, and lineage-tracked.

Built with the same rigor as a production platform: a full documentation suite (architecture rationale, column-level data dictionaries, ERDs, operational runbooks), a changelog, and a defined release process — the kind of knowledge-transfer artifacts a real data team would expect, not just working code.