Data Lineage
Wolt Snack Store — Berlin 2023

This diagram shows how raw Wolt snack store files are transformed through dbt into the dimensional model that powers dashboards, exports, and the AI assistant.

3 sources 13 dbt models 7 mart tables Documented pipeline
End-to-end pipeline

How raw Wolt snack store files flow into staging, intermediate logic, and analytics-ready marts.

flowchart LR subgraph RAW["Raw sources"] IL[item_logs] PR[promos] PL[purchase_logs] end subgraph STG["Staging"] S1[stg_item_logs] S2[stg_promos] S3[stg_purchase_logs] S4[stg_purchase_items] end subgraph INT["Intermediate"] I1[int_items_scd] I2[int_purchase_items_enriched] end subgraph MARTS["Marts"] D1[dim_items] D2[dim_items_history] D3[dim_customers] D4[dim_promotions] D5[dim_dates] F1[fct_purchases] F2[fct_purchase_items] end IL --> S1 PR --> S2 PL --> S3 PL --> S4 S1 --> I1 S1 --> D1 S1 --> D2 S2 --> D4 S3 --> D3 S3 --> F1 S3 --> D5 S4 --> I2 I1 --> I2 S2 --> I2 I2 --> F2 F1 --> F2
Staging layer

Parse JSON payloads, deduplicate keys, explode baskets, and standardise column names.

flowchart TB IL[item_logs CSV] --> S1[stg_item_logs<br/>JSON parse · dedup · price imputation] PR[promos CSV] --> S2[stg_promos<br/>date windows · discount %] PL[purchase_logs CSV] --> S3[stg_purchase_logs<br/>order-level cleaning] PL --> S4[stg_purchase_items<br/>basket JSON explosion] S3 --> S4
Intermediate layer

Business rules that must happen before facts and dimensions are published.

flowchart TB S1[stg_item_logs] --> I1[int_items_scd<br/>SCD Type 2 validity windows] S4[stg_purchase_items] --> I2[int_purchase_items_enriched] S2[stg_promos] --> I2 I1 --> I2 I2 --> OUT[Point-in-time price · promo flags · net value]
Star schema marts

Final tables consumed by dashboards, exports, and the AI assistant context.

flowchart TB D1[dim_items] D2[dim_items_history] D3[dim_customers] D4[dim_promotions] D5[dim_dates] F1[fct_purchases<br/>order grain] F2[fct_purchase_items<br/>order line grain] D3 --> F1 D5 --> F1 D1 --> F2 D2 --> F2 D4 --> F2 D3 --> F2 D5 --> F2 F1 --> F2
Platform consumption

How marts and Task 2 assets power the self-service site.

flowchart LR MARTS[(main_marts<br/>dbt marts in DuckDB)] ASSETS[(main_assets<br/>Task 2 CSVs)] DASH[Dashboards] DATASET[Dataset Explorer] AI[AI Assistant] SUB[Submission browser] MARTS --> DASH MARTS --> DATASET MARTS --> AI MARTS --> SUB ASSETS --> DATASET ASSETS --> SUB
How to read this
  • Arrow A → B means model or table A feeds model or table B.
  • Staging cleans and parses raw CSV inputs (JSON extraction, deduplication, basket explosion).
  • Intermediate applies SCD Type 2 windows and point-in-time promo/price joins.
  • Marts publish the star schema: dimensions for products, customers, promos, dates; facts at order and order-line grain.
  • The platform reads main_marts for live dashboards and main_assets for Task 2 deliverables.