Big Tech Digest #10

2024-01-11: Rebuilding Netflix video processing pipeline with microservices, Cloudflare's DDoS threat report, Making Expedia flights search faster, Measuring Booking mobile app performance and more!

Happy Thursday 👋 and happy new year!

I’m excited to kick off the 2024 chapter of Big Tech Digest. The growth in 2023 was incredible as we’ve hit the +1k subscribers mark. Big thanks to all of you 🙏🙏🙏!

I’m trying to increase my activity on X and share the articles ahead of time. Feel free to follow Big Tech Digest there, too!

One thing you could do to help me grow Big Tech Digest is to tell your connections about it. Any share is much appreciated 🙏!

Without further ado, let’s get started!

// 📫 Engineering blogs digest

Most notable articles posted in the tech companies’ engineering blogs over the past two weeks.

1. "Rebuilding Netflix Video Processing Pipeline with Microservices"

by many authors ⸱ Netflix ⸱ 12 min read ⸱ 10 Jan

• Explains the transition from centralized linear encoding to distributed chunk-based encoding

• Introduces the move from dedicated instances to utilizing autoscaling microservices for improved computation elasticity

• Shares the benefits of the new platform, including ease of feature delivery and support for new business needs like Ad processing

2. "Building Pinterest’s new wide column database using RocksDB"

by Rajath Prasad ⸱ Pinterest ⸱ 13 min read ⸱ 04 Jan

• Covers the features supported, such as versioned values, TTL, and pagination of responses

• Presents the motivation behind the decision to consolidate key-value systems

• Explores the details of the data model, APIs, and key features of the database

• Shares information about how the database supports a key platform in the product

3. "10 Microservices Design Patterns for Developers"

Capital One ⸱ 13 min read ⸱ 09 Jan

• Describes the Database per service pattern for microservices architecture

• Explores the Saga pattern for maintaining data consistency during distributed transactions

• Covers API gateways for insulating clients from service partitioning

• Shares the Command Query Responsibility Segregation (CQRS) for reducing complexity and improving app performance

4. "DDoS threat report for 2023 Q4"

by Omer Yoachimik and Jorge Pacheco ⸱ Cloudflare ⸱ 20 min read ⸱ 09 Jan

• Discusses the increase in network-layer DDoS attacks, particularly targeting retail, shipment, and public relations websites during Black Friday and the holiday season

• Describes the surge in DDoS attack traffic targeting Environmental Services websites coinciding with COP 28

• Presents an overview of attack vectors and attributes, including emerging attack vectors such as ACK-RST Floods, CLDAP floods, and SPSS floods

5. "Search Speed: Making Expedia Flights Faster"

by Heena Gupta ⸱ Expedia ⸱ 11 min read ⸱ 03 Jan

• Gives an overview of the performance metrics used to measure search speed

• Explores experimental initiatives to improve performance, such as prefetching and caching

• Shares the impact of these initiatives, with a 52% improvement in page usable time and a 40% improvement in non-supply overhead

• Introduces the use of micro frontend architecture to improve build size and flexibility

6. "Command injection in Python: examples and prevention"

by Rubaiat Hossain ⸱ Snyk ⸱ 13 min read ⸱ 21 Dec

• Describes common scenarios that lead to command injection vulnerabilities

• Explores the risks associated with dynamic command construction and insecure use of eval()

• Shares guidelines for mitigating command injection vulnerabilities in Python applications

• Presents secure coding conventions and best practices for secure Python development

7. "An overview of Cloudflare's logging pipeline"

by Colin Douch ⸱ Cloudflare ⸱ 8 min read ⸱ 08 Jan

• Describes the use of various logging libraries like Go’s zerolog, C++’s KJ_LOG, or Rusts log at Cloudflare

• Explores the use of syslog-ng to collect logs from every machine and provide redundancy with log-x operation

• Covers the buffering of logs in a Kafka

• Shares upcoming projects to improve the logging pipeline at Cloudflare, including multi-tenancy capabilities and migrating towards Open Telemetry

8. "How Prezi Serves Customer Traffic"

by Alex ⸱ Prezi ⸱ 8 min read ⸱ 09 Jan

• Describes how AWS Global Accelerator reduces latency for customer requests

• Covers the protection provided by AWS WAF and Shield against DDoS attacks

• Shares the importance of using CloudFront for content delivery to decrease load times and costs

9. "Introducing Cloudflare’s 2024 API security and management report"

by John Cosgrove and Sabina Zejnilovic ⸱ Cloudflare ⸱ 14 min read ⸱ 09 Jan

• Explores the use of rate limits for minimizing attack potential

• Shares insights on protecting APIs from web application attacks

• Presents a look at the human-driven nature of API traffic and its implications for security

10. "Solving The Async Context Challenge In Node.Js"

by Israel Zablianov ⸱ Wix ⸱ 9 min read ⸱ 09 Jan

• Explores the limitations of using a global object as a context and the challenges of implementing a multi-threaded approach in Node.Js

• Presents the solution of using AsyncLocalStorage API to propagate context through the call chain

• Shares a performance analysis of AsyncLocalStorage and its impact on application performance

• Explains the potential context loss and performance impact of AsyncLocalStorage

11. "Mastering Python virtual environments: A complete guide to venv, Docker, and securing your code"

by Liran Tal ⸱ Snyk ⸱ 9 min read ⸱ 10 Jan

• Goes through how to create and activate a virtual environment using venv

• Introduces the use of Docker for creating a local Python development environment

• Shares how to keep Python Docker applications secure

12. "Measuring mobile apps performance in production"

by Gleb Tarasov ⸱ Booking.com ⸱ 12 min read ⸱ 11 Jan

• Shares why existing third-party tools for monitoring app performance did not meet the requirements

• Covers the primary user concerns to monitor such as:

  • app startup time

  • screen time to interactive

  • frame rendering performance

• Introduces open-sourced performance tracking libraries for iOS and Android available on GitHub

13. "Experimenting with Handwriting Recognition for The New York Times Crossword"

by Shafik Quoraishee ⸱ The New York Times ⸱ 10 min read ⸱ 08 Jan

• Describes the process of implementing On Device ML onto the Android Crosswords app

• Explores the process of data preparation, conditioning, and normalization for handwriting recognition

• Shares the challenges and solutions faced in building a Deep Convolutional Network for letter recognition

14. "Embed multicolor icons using a single DOM element"

by Paweł Lesiecki and Maciej Suszko ⸱ Allegro ⸱ 6 min read ⸱ 10 Jan

• Starts with the challenges of embedding icons in modern UIs

• Introduces the SVG+CSS technique for controlling icon colors with a single DOM element and one external SVG covering the step-by-step implementation

• Shares the benefits of the technique, including caching and customizability

I hope you enjoyed this issue. Let me know in the comments what have you learned ⭐️!

Thanks for reading Big Tech Digest. Please don’t forget to 🗣️ spread the word and see you in two weeks!

---

Delivered bi-weekly to your inbox, Big Tech Digest brings you a collection of links to the latest engineering blog posts from +300 Big Tech companies and startups like Airbnb, Uber, Netflix or Meta. Aimed at Software Engineers and AI/ML folks at any level, Big Tech Digest is focused on interesting solutions to engineering problems that tech companies come across. No marketing or non-tech stuff.

Subscribe now to receive a new issue directly to your inbox every two weeks!