Big Tech Digest #19 💥
Featuring articles from Wix, Doordash, Etsy, Flipkart, and many more!
Happy Thursday 👋!
Welcome to another Big Tech Digest issue.
I’m thrilled to announce that we’ve reached 2000 subscribers 🎉
Big thanks to all of you 🙏🙏🙏!
I’m doing my best to make the newsletter valuable while keeping it free. If you have any feedback, please reply to this message or leave a comment.
This time around, we have many fresh great articles since the last 2 weeks and I had a really hard time building the featured list.
There’s just one thing you could do to help me grow Big Tech Digest: go ahead and mention it to your friends and/or teammates. Thank you! 🙏
Without further ado, let’s get started!
// 🏆 Must reads
1. "Don’t Get Caught by Surprise: Understanding the READ AFTER WRITE Problem"
by Dean Bachar ⸱ Wix ⸱ 8 min read ⸱ 11 Aug
Describes the problem of Read After Write issues in microservices architectures
Explores how a hypothetical case of an e-commerce platform can lead to flaky bugs
Discusses the challenges of distributed databases and the potential for data inconsistency
Shares the options for handling data changes in a distributed database system
Presents potential solutions for addressing the issues of data inconsistency in distributed databases
2. "DoorDash Empowers Engineers with Kafka Self-Serve"
by Multiple authors ⸱ DoorDash ⸱ 13 min read ⸱ 13 Aug
Describes the development journey of DoorDash's Kafka Self-Serve platform
Discusses the pain points faced in managing Kafka resources with Terraform
Explains the design and implementation of the Kafka Self-Serve platform
Shares the impact of the platform, including customer accolades and velocity and reliability improvements
// 📬 Optional reads
a.k.a. The Best of the Rest!
"Mitigating Deadlocks in high concurrency environments"
by Elson Oliveira ⸱ Shopify ⸱ 3 min read ⸱ 14 Aug
Explores the problem of deadlocks in MySQL when updating data
Describes how gap locks are required for updating existing records
Presents the use of composite primary keys as a solution to mitigate deadlocks
Shares the benefits of using composite primary keys to reduce overlapping gaps
Explains how the solution improved the overall throughput of import jobs in the DB layer
"Machine Learning in Content Moderation at Etsy"
by David Azcona ⸱ Etsy ⸱ 1 min read ⸱ 06 Aug
"dbt Generic Tests in Sessions Validation at Yelp"
Yelp ⸱ 6 min read ⸱ 14 Aug
Describes how dbt is used as a test and validation tool for enhancing the Sessions Data Mart at Yelp
Covers the challenges encountered before adopting dbt as the data validation tool
Explains the development cycle for the Sessions Data Mart and how dbt generic tests are utilized for data validation
Shares an example of data validation using dbt generic tests for the sessions_table model
Goes through the process of storing test failures for further debugging
"Iceberg vs Hudi — Benchmarking TableFormats"
by Mudit Sharma ⸱ Flipkart ⸱ 1 min read ⸱ 13 Aug
"Platform as a Runtime (PaaR) — Evolving Platform Engineering"
by Aviran Mordo ⸱ Wix ⸱ 12 min read ⸱ 12 Aug
Describes the hindrance large, complex systems pose to development speed and the benefits of "Platform as a Runtime" to simplify the environment and increase security.
Discusses the impact of complex software systems on developers and the need for standardization to mitigate scaling challenges.
Explains the concept of "Platform as a Runtime" (PaaR) and how it focuses on optimizing service footprint and the developer experience.
Shares Wix's experience with developing their own PaaR solution, "Nile," and the significant improvements in developer velocity and product quality.
Advises organizations facing significant scaling challenges to prioritize building the platform first and emphasizes the need for a value-driven platform team and consistent communication with developers.
"Unveiling the Future of RAG: The papers you don’t want to miss"
by Dvir Cohen ⸱ Wix ⸱ 6 min read ⸱ 04 Aug
Thanks for reading Big Tech Digest. If you enjoyed this issue, 🔗 share it with your friends or teammates.
See you in two weeks 👋!