Big Tech Digest #16 💥
Featuring articles from DoorDash, Netflix, Slack, Lyft, and many more!
Happy Saturday 👋!
Big Tech Digest is back!
After a short break, we’re back with fresh must-read articles from engineering blogs. This issue includes articles from DoorDash, Netflix, Slack, Lyft, and many more!
There’s just one thing you could do to help me grow Big Tech Digest: go ahead and mention it to your friends and/or teammates. Thank you! 🙏
💥 Tech Talks Weekly
I’d like to share an excellent newsletter, called Tech Talks Weekly, which delivers all the recently uploaded tech conference videos from over 100 engineering conferences (see the full list) like Devoxx, GOTO, NDC, QCon, LeadDev, and many more (see a recent issue). I highly encourage to subscribe if you’re into watching tech talks.
Without further ado, let’s get started!
// 🏆 Must reads
1. "The Great Rewrite - How Wix is Preparing to Rewrite 100s of Systems - Part 2"
Wix ⸱ 8 min read ⸱ 24 Jun
Discusses the process of rewriting systems at Wix
Explains how backwards compatibility helped with migration
Shares the process of rolling out to new and existing tenants
2. "A Mission to Cost-Effectiveness: Reducing the cost of a single Google Cloud Dataflow Pipeline by Over 60%"
Allegro ⸱ 9 min read ⸱ 20 Jun
Presents real-life scenario of cost optimization in stages, focusing on physical resources optimization and data processing engine configuration
Explores hypothesis testing for underutilized physical resources and price-to-performance ratio of virtual machine types
Shares findings on optimizing memory utilization and changing virtual machine types for cost-effectiveness
Gives an overview of turning off Shuffle Service to achieve significant cost reduction and provides a final test on a full dataset with estimated annual savings of $78,740.
// 📬 Optional reads
a.k.a. The Best of the Rest!
"Enhancing Netflix Reliability with Service-Level Prioritized Load Shedding"
Netflix ⸱ 11 min read ⸱ 25 Jun
Discusses how Netflix has extended prioritized load shedding to the individual service level
Describes the advantages of applying prioritization logic at the service layer
Presents a solution for prioritizing user-initiated requests over prefetch requests without sharding
Shares testing results to validate the effectiveness of load-shedding
Introduces generic CPU and IO-based load-shedding techniques for services not autoscaling on CPU utilization
"ETA (Estimated Time of Arrival) Reliability at Lyft"
by Rachita Naik ⸱ Lyft ⸱ 7 min read ⸱ 20 Jun
Discusses the challenges of estimating ETA reliability at Lyft
Explores the complexities of reliability before and after a ride is requested
Presents the use of machine learning for reliability prediction
Describes the tree-based classification model used for reliability estimation
Covers the innovative training approach and model performance evaluation
"Proactive Measures Against Password Breaches and Cookie Hijacking"
by Nathan Lehotsky ⸱ Slack ⸱ 5 min read ⸱ 28 Jun
Discusses proactive measures for protecting user data and credentials
Covers the process of identifying and resetting compromised passwords
Shares the strategy for invalidating hijacked cookies and maintaining user experience
"Secure Code Reviewer — Copilot"
by Ashwath Kumar ⸱ Razorpay ⸱ 8 min read ⸱ 21 Jun
Explains the integration of Large Language Models (LLMs) into the code review workflow
Describes the tiered risk management approach for different application priorities
Shares the experiment of using LLMs to identify security flaws in known vulnerable applications
Presents cost considerations and optimization strategies for using LLMs in security workflows
"Catching Compromised Cookies"
by Ryan Slama ⸱ Slack ⸱ 1 min read ⸱ 24 Jun
"FAQ: Common Questions from Candidates During Lyft Data Science Interviews"
by Kelly Haberl ⸱ Lyft ⸱ 9 min read ⸱ 25 Jun
Describes the different Data Science roles at Lyft, focusing on Decision Scientists and Algorithm Scientists
Covers the interview process for Data Science candidates, including recruiter screen, technical phone screen, and virtual onsite interviews
Explains the structure of the technical interviews, including coding and experience interviews
Gives an overview of the work and team matching process for Data Scientists at Lyft
"Failing to Auto Scale Elasticsearch in Kubernetes"
by Juho Vuori ⸱ Zalando ⸱ 6 min read ⸱ 21 Jun
Describes an incident where an Elasticsearch cluster in Kubernetes failed to scale down at night, leading to potential issues in the morning
Explores the root cause analysis, revealing a bug in the es-operator that caused the scaling issue
Shares the first fix implemented, which did not work, leading to another bug being uncovered in the es-operator
Explains the second fix, involving manual removal of a "zombie" node from the exclusion list
"DoorDash Opens a New Engineering Hub in São Paulo"
by Demetrius Nunes ⸱ DoorDash ⸱ 1 min read ⸱ 28 Jun
"Journey to Contract Testing through Pact"
by Paulo Zenida ⸱ OLX ⸱ 18 min read ⸱ 24 Jun
Discusses OLX's use of Pact for contract testing in microservices
Describes how Pact streamlines the process, offers quick feedback, and enhances API stability
Explains the benefits of contract testing, such as improved API reliability and faster feedback loops
Introduces Pact as a mature contract testing framework supporting multiple languages
Shares OLX's experience with Pact, including highlights and next steps
"Bypassing Boundaries: 4 Basic Steps for Indirect Prompt Injection in LLMs"
by Zeev Kalyuzhner ⸱ Wix ⸱ 2 min read ⸱ 01 Jul
Discusses the risks associated with indirect prompt injection in LLMs
Describes the vulnerability of insecure output handling in LLMs
Explores the two forms of prompt injection: direct and indirect
Presents a practical scenario to illustrate the potency of indirect prompt injection
Thanks for reading Big Tech Digest. If you enjoyed this issue, 🔗 share it with your friends or teammates.
See you in two weeks 👋!