Apache Flink Use Cases – Real life case studies of Apache Flink
In this Apache Flink Use Cases tutorial, we will discuss top 7 use case of Apache Flink deployed in Fortune 500 companies. Apache Flink also known as 4G of Big Data, understand its real life applications, here we will discuss real world case studies of Apache Flink. We deploy it in production at leading organizations like Alibaba, Bouygues, Zalando, etc. we will see these game-changing use cases of Apache Flink.
Refer this guide to learn installation of Apache Flink on ubuntu.
2. Top 7 Apache Flink Use Cases
Let’s discuss top 7 real life case studies of Apache Flink-
a. Bouygues Telecom – Third largest mobile provider in France
The Bouygues Group ranks in Fortune’s “Global 500.” Bouygues uses Flink for real-time event processing and analytics for billions of messages per day in a system that is running 24/7.
Bouygues chose Apache Flink because it supports true streaming at the API and at the runtime level, thus providing low latency that company was looking for. In addition, their systems were up and running in shortest duration with Flink as compared to other solutions (follow this Flink vs Spark vs Hadoop comparison guide for more details), which helped them in expanding the business logic in the system.
Bouygues wanted to get real-time insights about customer experience, what is happening globally on the network, and what is happening in terms of network evolutions and operations. For this, its team built a system to analyze network equipment logs to identify indicators of the quality of user experience. The system handles 2 billion events per day (500,000 events per second) with a required end-to-end latency of fewer than 200 milliseconds (including message publication by the transport layer and data processing in Flink). This was achieved on a small cluster reported to be only 10 nodes with 1 gigabyte of memory each.
Planning was to use Flink’s stream processing for transforming and enriching data and pushing back the derived stream data to the message transport system for analytics by multiple consumers.
Flink’s stream processing capability allowed the Bouygues team to complete the data processing and movement pipeline. While meeting the latency requirement and with high reliability, high availability, and ease of use. The Apache Flink framework, for instance, is ideal for debugging, and it can switch to local execution. Flink also supports program visualization to help understand how programs are running. Furthermore, the Flink APIs are attractive to both developers and data scientists.
b. King – The creator of Candy Crush Saga
King – the leading online entertainment company has more than 200 games in more than 200 countries and regions.
Any stream analytics use case becomes a real technical challenge when more than 300 million monthly users generate more than 30 billion events every day from the different games and systems. To handle these massive data streams using data analytics while keeping maximal flexibility was a great challenge that has been overcome by Apache Flink.
Flink allows data scientists at King to get access to these massive data streams in real time. Even with such a complex game application, Flink is able to provide out of the box solution.
c. Zalando – Leading E-commerce Company in Europe
Zalando has more than 16 million customers worldwide and uses Apache Flink for real-time process monitoring. A stream-based architecture nicely supports a microservices approach being used by Zalando, and Flink provides stream processing for business process monitoring and continuous Extract, Transform and Load (ETL)
d. Otto Group – World’s second largest online retailer
Otto Group BI Department was planning to develop its own streaming engine for processing their huge data. As none of the open source options were fitting its requirements. After testing Flink, the department found it fit for crowdsourcing user-agent identification and identifying a search session via stream processing.
e. ResearchGate – Largest academic social network
ResearchGate is using Flink since 2014 as one of its primary tools in the data infrastructure for both batch and stream processing. It uses Flink for its network analysis and near duplicate detection to enable flawless experience to its members.
f. Alibaba Group – World’s largest retailer
Alibaba works with buyers and suppliers through its web portal. Flink’s variation (called Blink) is being used by the company for online recommendations. Apache Flink provides it the feature to take into consideration the purchases. That are being made during the day while recommending products to users. This plays a key role on special days (holidays) when the activity is unusually high. This is an example where efficient stream processing plays over batch processing.
g. Capital One – Fortune 500 financial services company
Being a leading consumer and commercial banking institution, the company had the challenge to monitor customer activity data in real time. They wanted this to detect and resolve customer issues immediately and enable flawless digital enterprise experience. Current legacy systems were quite expensive and offered limited capabilities to handle this. Apache Flink provided a real time event processing system. It was cost effective and future proof to handle growing customer activity data.
Flink provided advanced analytics on data streams like advanced windowing, event correlation, event clustering, anomaly detection etc.