2022

Best Individual Data Streaming System

Walmart
Best Individual Data Streaming System

Walmart has continuously redefined what can be achieved in the data streaming space for many years now across its entire business. In the 2022 Data Streaming Awards, they have been recognized for their move from Batch inventory and Batch replenishment to Real-time Inventory and real-time Replenishment at scale by leveraging multifaceted Streaming Architecture. Walmart doesn’t just have physical stores, it also has hundreds of distribution centers (DCs) of varying types, in a multi-echelon ecosystem that makes accurate, reliable, real-time replenishment a very challenging proposition. Apache Kafka® is a big part of the cutting-edge IT architecture that makes it all happen.

Technologies Used:

  • Apache Kafka®
  • Apache Spark®
  • Kafka Stream
  • Spring Kafka
  • Kafka Connect
  • Elastic Search
  • GCS/Big Query

What problem was the nominee organization looking to solve with Data Streaming Technology?

Historically like most Retail companies, Walmart leveraged Batch Platforms and Data ingest technologies to bring in the data from various Physical nodes (Stores, DCs, etc) and calculate the Order points and the Order plans for the future. That lag in time between when the input data is created and when it is ingested and used causes inefficient and inaccurate Order plans in most cases. Walmart’s team onboarded the whole Platform into streaming architecture using Kafka to make more real-time, accurate decisions.

How did they solve the problem?

On any given day, Walmart’s real-time replenishment system processes more than tens of billions of messages from close to 100 million SKUs in less than three hours. They leverage an array of processors to generate an order plan for their entire network of Walmart stores with great accuracy and at high throughputs of 85GB messages/min. While doing so, it also ensures there is no data loss through event tracking and necessary replays and retries.

Positive outcome

The work that Walmart’s architects did changed the quality of their Order plans and in turn allowed the wider business and Merchants to make better decisions. It also helped reduce the frequency of products going out of stock, which of course drastically helped improve Walmart’s Customer experience.

Links:

Your 2022 Winners!

Streaming Company of the Year

NASA + JPL

View Winner
The Data Streaming Award for Innovation

Ademen

View Winner
The Best Company-wide data streaming Implementation

Pinterest

View Winner
Startup of the Year

Control

View Winner
Best Individual Data Streaming System

Walmart

View Winner
The Legend Award

LinkedIn

View Winner
Nominations are closed for the 2022 awards
NOMINATE
Nominations close on the 30th June 2023