Apache Kafka Use cases
Apache Kafka can be used for various purposes in an organization, such as
1. Messaging service
Millions of messages can be sent and received in real-time. In comparison to most messaging systems Kafka has better throughput, built-in partitioning, replication, and fault-tolerance which makes it a good solution for large scale message processing applications.
2. Real-time stream processing
Kafka can be used to process a continuous stream of information in real-time and pass data to stream processing systems such as Storm.
3. Log aggregation
Kafka can be used to collect physical log files from multiple systems and store it in a central location such as HDFS.
4. Commit log service
Kafka can be used as an external commit log for distributed systems. The log helps replicate data between nodes and acts as a re-syncing mechanism for failed nodes to restore their data. The log compaction feature in Kafka, helps in supporting this use-case.
5. Event sourcing
A time ordered sequence of events can be maintained through Kafka.
6. Kafka Metrics
For operational monitoring data, Kafka is often used. This involves aggregating statistics from distributed applications to produce centralized feeds of operational data.
7. Website Activity Tracking
Website activity (page views, searches, or other actions users may take) is published to central topics with one topic per activity type. These feeds are available for subscription for a range of use cases including real-time processing, real-time monitoring, and loading into Hadoop or offline data warehousing systems for offline processing and reporting.
Apache Kafka Applications
Kafka supports many of today’s best industrial applications.
1. Twitter
Twitter is one of the best Kafka Applications. A famous online social networking service or a platform Twitter uses Kafka. Basically, it provides a way to send and receive users tweets. Through this platform, registered users can read and post tweets, but unregistered users can only read tweets. However, it uses Storm-Kafka as a part of their stream processing infrastructure.
2. LinkedIn
Another Kafka Application is LinkedIn. For activity stream data and operational metrics, LinkedIn uses Apache Kafka. There are several products like LinkedIn Newsfeed, LinkedIn Today, for online message consumption and in addition to offline analytics systems like Hadoop, Kafka messaging system helps LinkedIn. Moreover, we can say the strong durability of Kafka is also one of the key factors in connection with LinkedIn.
3. Netflix
An American multinational provider of on-demand internet streaming media, Netflix, also uses Kafka. Basically, for the purpose of real-time monitoring and event processing, it uses Kafka.
4. Mozilla
In 1998, members of Netscape created a free-software community, Mozilla. In order to collect performance and usage data from the end-users browser for projects like Telemetry, Test Pilot, etc. Kafka will soon be replacing a part of Mozilla current production system.
5. Oracle
Oracle provides native connectivity to Kafka from its Enterprise Service Bus product called OSB (Oracle Service Bus) which allows developers to leverage OSB built-in mediation capabilities to implement staged data pipelines.