Data Platforms & Quality

What you will be able to do

• You will be able to design and load a cloud data warehouse such as Snowflake or BigQuery, using separated storage and compute, virtual warehouses or slots, and the right table and partitioning choices for a workload.
• You will be able to model analytical data with dimensional and wide-table patterns, and tune queries for both performance and cost.
• You will be able to architect a data lake on object storage with columnar Parquet and open table formats (Apache Iceberg, Delta Lake, or Apache Hudi) for ACID transactions and time travel.
• You will be able to explain the lakehouse architecture and decide, with real trade-offs, when to use a warehouse, a lake, or a lakehouse.
• You will be able to build streaming ingestion with Apache Kafka, reasoning about topics, partitions, consumer groups, and delivery guarantees including exactly-once processing.
• You will be able to capture change from source databases with CDC and combine batch and streaming into one coherent pipeline.
• You will be able to enforce data quality with tests, data contracts, and freshness, volume, and schema checks, and stand up data observability to catch pipeline breakage early.
• You will be able to govern a platform with a data catalog, end-to-end lineage, access controls, and classification of sensitive data to meet privacy and compliance needs.

What is inside

4 modules, 8 lessons. Each module ends in a graded quiz and most carry an assignment.

1. 01

   Cloud Data Warehouses at Scale

   Modern analytics runs on cloud data warehouses that separate storage from compute and scale each on demand. This module opens up how Snowflake and BigQuery actually execute a query: the storage and compute split, massively parallel processing, virtual warehouses and slots, and the micro-partitions and columnar formats that let an engine skip data instead of reading it. It then turns to modeling and tuning, weighing dimensional star schemas against wide denormalized tables, showing how clustering and partitioning drive pruning, and connecting every choice to what you actually pay. You finish able to load, model, and cost-tune a warehouse, not just write SQL against one.

   2 lessons · 5 quiz questions

2. 02

   Data Lakes and the Lakehouse

   Not all data belongs in a warehouse. This module builds a data lake on cheap object storage with columnar Parquet, then adds the open table formats (Apache Iceberg, Delta Lake, and Apache Hudi) that bring ACID transactions, schema evolution, and time travel to the lake. You learn the lakehouse pattern that serves BI and machine learning from one governed copy, and how to choose deliberately between warehouse, lake, and lakehouse for a given workload.

   2 lessons · 5 quiz questions · assignment

3. 03

   Streaming and Ingestion with Kafka

   Batch alone cannot keep a data platform fresh, so real-time ingestion is core data engineering work. This module teaches Apache Kafka from the storage layer up: topics, partitions, offsets, consumer groups, and the delivery guarantees that decide whether events are lost, duplicated, or processed exactly once. You then wire streaming into the platform with change data capture and stream processing, land events in the warehouse and lake, and learn where streaming genuinely beats batch and where it does not.

   2 lessons · 5 quiz questions

4. 04

   Data Quality, Testing, and Governance

   A platform nobody trusts is worse than no platform, so this module makes the data provably correct and well governed. You build quality in with tests, data contracts, and freshness, volume, and schema checks, then add data observability so breakage surfaces before stakeholders notice. It closes with governance the way auditors and regulators expect it: a data catalog, end-to-end lineage, access control, and classification of sensitive data.

   2 lessons · 5 quiz questions · assignment