Big Tech Digest #19 π₯
Featuring articles from Wix, Doordash, Etsy, Flipkart, and many more!
Happy Thursday π!
Welcome to another Big Tech Digest issue.
Iβm thrilled to announce that weβve reached 2000 subscribers π
Big thanks to all of you πππ!
Iβm doing my best to make the newsletter valuable while keeping it free. If you have any feedback, please reply to this message or leave a comment.
This time around, we have many fresh great articles since the last 2 weeks and I had a really hard time building the featured list.
Thereβs just one thing you could do to help me grow Big Tech Digest: go ahead and mention it to your friends and/or teammates. Thank you! π
Without further ado, letβs get started!
// π Must reads
1. "Donβt Get Caught by Surprise: Understanding the READ AFTER WRITE Problem"
by Dean Bachar βΈ± Wix βΈ± 8 min read βΈ± 11 Aug
Describes the problem of Read After Write issues in microservices architectures
Explores how a hypothetical case of an e-commerce platform can lead to flaky bugs
Discusses the challenges of distributed databases and the potential for data inconsistency
Shares the options for handling data changes in a distributed database system
Presents potential solutions for addressing the issues of data inconsistency in distributed databases
2. "DoorDash Empowers Engineers with Kafka Self-Serve"
by Multiple authors βΈ± DoorDash βΈ± 13 min read βΈ± 13 Aug
Describes the development journey of DoorDash's Kafka Self-Serve platform
Discusses the pain points faced in managing Kafka resources with Terraform
Explains the design and implementation of the Kafka Self-Serve platform
Shares the impact of the platform, including customer accolades and velocity and reliability improvements
// π¬ Optional reads
a.k.a. The Best of the Rest!
"Mitigating Deadlocks in high concurrency environments"
by Elson Oliveira βΈ± Shopify βΈ± 3 min read βΈ± 14 Aug
Explores the problem of deadlocks in MySQL when updating data
Describes how gap locks are required for updating existing records
Presents the use of composite primary keys as a solution to mitigate deadlocks
Shares the benefits of using composite primary keys to reduce overlapping gaps
Explains how the solution improved the overall throughput of import jobs in the DB layer
"Machine Learning in Content Moderation at Etsy"
by David Azcona βΈ± Etsy βΈ± 1 min read βΈ± 06 Aug
"dbt Generic Tests in Sessions Validation at Yelp"
Yelp βΈ± 6 min read βΈ± 14 Aug
Describes how dbt is used as a test and validation tool for enhancing the Sessions Data Mart at Yelp
Covers the challenges encountered before adopting dbt as the data validation tool
Explains the development cycle for the Sessions Data Mart and how dbt generic tests are utilized for data validation
Shares an example of data validation using dbt generic tests for the sessions_table model
Goes through the process of storing test failures for further debugging
"Iceberg vs HudiβββBenchmarking TableFormats"
by Mudit Sharma βΈ± Flipkart βΈ± 1 min read βΈ± 13 Aug
"Platform as a Runtime (PaaR)βββEvolving Platform Engineering"
by Aviran Mordo βΈ± Wix βΈ± 12 min read βΈ± 12 Aug
Describes the hindrance large, complex systems pose to development speed and the benefits of "Platform as a Runtime" to simplify the environment and increase security.
Discusses the impact of complex software systems on developers and the need for standardization to mitigate scaling challenges.
Explains the concept of "Platform as a Runtime" (PaaR) and how it focuses on optimizing service footprint and the developer experience.
Shares Wix's experience with developing their own PaaR solution, "Nile," and the significant improvements in developer velocity and product quality.
Advises organizations facing significant scaling challenges to prioritize building the platform first and emphasizes the need for a value-driven platform team and consistent communication with developers.
"Unveiling the Future of RAG: The papers you donβt want to miss"
by Dvir Cohen βΈ± Wix βΈ± 6 min read βΈ± 04 Aug
Thanks for reading Big Tech Digest. If you enjoyed this issue, π share it with your friends or teammates.
See you in two weeks π!