Rethinking Data Architecture: The Promise of Data Mesh
It’s not every day that you stumble upon a book that completely shifts your perspective on your day-to-day work. That’s exactly what happened when...
Partitioning Tables in BigQuery
If you are analyzing relatively small amounts of data, then your queries won’t cost very much, but if you regularly analyze huge datasets, then...
Generating Dates Between Two Dates in BigQuery
Generating a list of dates between two dates can be very helpful in many situations. Many source systems typically provide you with a start...
Medallion Architecture in Modern Data Platforms
If you’ve explored or worked with modern data management strategies, you’ve likely encountered terms like data lake, data warehouse, and the increasingly popular concept...
Incremental Load of Delta Records Using Dataform in BigQuery
When working with large datasets in data pipelines, processing every record repeatedly can be inefficient. A common approach to optimize this process is to...