TWIL: July 3, 2022

This week I continued looking into Data Mesh, as well as Data Warehouse, Data Lake and Data Lakehouse architectures. I also recommend a few episodes from the usual podcasts, namely an interesting conversation between Scott Hanselman and the Director of Engineering at Stack Overflow, and the 1800th episode of .Net Rocks!


Podcasts

.NET Rocks

Episode 1800: Episode 1800 with Heather Downing Live from NDC London!
Episode 1800! While at NDC London, Carl and Richard were on stage for a live show with Heather Downing, discussing the modern developer career. The pandemic created considerable changes in work, and developers were also affected. Do you have the job you want? How do you change it? With some questions from the online viewers, Heather talks about taking control of your career and turning it into the life you want – and celebrating 1800 episodes of .NET Rocks!

The Azure Podcast

Episode 429: Azure Native Independent Software Vendor (ISV) Solutions
Cale and Cynthia chat with Manju Ramanathpura about the Azure Native ISV program, the customer experience, the background of the initiative and the investments that are made to streamline various products that developers are using onto Azure.

Hanselminutes

Episode 847: Engineering Stack Overflow with Roberta Arcoverde
Interesting talk with Roberta Arcoverde, the Director of Engineering at Stack Overflow. They discuss the underlying architecture of Stack Overflow, and how it supports the 2 billion pages views per month. I was suprised by the pragmatic approach of not trying to solve a problem that isn’t there, and that is why it’s a monolythic application hosted on-premises in 9 physical web servers.


Data Mesh

A financial institution scenario for data mesh
This reference architecture is written for customers that want to use cloud-scale analytics for scalability and data mesh architectures. It demonstrates a more complex scenario, with multiple landing zones, data integrations, and data products.

A Data Mesh Approach to Data Warehousing
Data mesh is a provocative new paradigm that offers a data strategy that is similar to software engineering best practices like microservice architecture. But, is this right for data? This article will discuss the pros and cons of data mesh as well as steps that organizations should take to improve their strategies. My opinion is that this approach should have gained traction long ago, but we must proceed with caution and offer some possible modifications to the original concept.

Data Mesh Explained
In the following article, I start by giving a brief presentation of Zhamak Dehghani. I follow up explaining what data mesh is and why it’s important. I give a brief summary of how it works. Finally, I share a few questions of my own. At the end of this article you will find a glossary of data mesh technical terms and a list of resources used for this article or that may interest you.

A practical guide for Data Mesh implementation
In this article, I will share an overview of Data Mesh, a few key principles, and a couple of implementation approaches.


Data Platform

Databricks vs Snowflake: The Definitive Guide
There is a lot of discussion surrounding Snowflake and Databricks in determining which modern cloud solution is better for analytics. However, both solutions were purpose-built to handle different tasks, so neither should be compared from an “apples to apples” perspective.

Data Lakehouse Architecture — Azure Synapse Serverless SQL Pools
Companies are looking beyond the limitations of the traditional data warehouses architecture to enable advanced analytics, data science, and machine learning on all of this data. Data Lakehouse is one such architecture that addresses many of the traditional data warehouses architecture limitations.


Cool Stuff

How to learn things fast without going crazy
As people who work in the tech industry, we often need to learn new technologies for our work. The solution that I am about to propose is around one core idea: Learn the general idea fast, and then gradually deep dive in.


Have a great week!

Photo by JJ Ying on Unsplash