Recent Posts

Wednesday, 21 November 2018

Apache Kafka - Pros and Cons

Advantages of Kafka  
     Apache Kafka is selected for it's strengths in the space of messaging. The following are some of the advantages
1. High-throughput
     Kafka is capable of handling high-velocity and high-volume data using not so large hardware. It is capable of supporting message throughput of thousands of messages per second.

2. Distributed
     The distributed architecture of Kafka makes it scalable using capabilities like replication and partitioning.

3. Low Latency
     It is capable of handling these messages with the very low latency of the range of milliseconds, demanded by most of the new use cases.

4.  Scalability
     Without incurring any downtime on the fly by adding additional nodes, Kafka can be scaled-out. Moreover, inside the Kafka cluster, the message handling is fully transparent and these are seamless.

5.  Fault-Tolerant
     One of the best advantages is Fault Tolerance. There is an inherent capability in Kafka, to be resistant to node/machine failure within a cluster.

6. Durability
     Here, durability refers to the persistence of data/messages on disk. Kafka uses Distributed commit log which means messages persists on disk as fast as possible, hence it is durable.

7. High Concurrency
     Apache Kafka is able to handle thousands of messages per second and that too in low latency conditions with high throughput. In addition, it permits the reading and writing of messages into it at high concurrency.

8. Consumer Friendly
     It is possible to integrate with the variety of consumers using Kafka. The best part of Kafka is, it can behave or act differently according to the consumer, that it integrates with because each customer has a different ability to handle these messages, coming out of Kafka. Moreover, Kafka can integrate well with a variety of consumers written in a variety of languages.

9. Real-Time Handling
     Kafka can handle real-time data pipeline. Since we need to find a technology piece to handle real-time messages from applications, it is one of the core reasons for Kafka as our choice.

Disadvantages of Kafka
     The following are some of the disadvantage associated with Kafka.
1. No Complete Set of Monitoring Tools
      Kafka don’t have complete set of management and monitoring tools, so Startup companies fear to use Kafka for long run.

2. Not support wildcard topic selection
      There is an issue that Kafka only matches the exact topic name, that means it does not support wildcard topic selection. Because that makes it incapable of addressing certain use cases.

3. Reduces Performance
      In general, there are no issues with the individual message size. However, the brokers and consumers start compressing these messages as the size increases. Due to this, when decompressed, the node memory gets slowly used. Also, compress happens when the data flow in the pipeline. It affects throughput and also performance. 

4. Behaves Clumsy
      Sometimes, it starts behaving a bit clumsy and slow, when the number of queues in a Kafka cluster increases. 

Next Tutorial : Apache Kafka - Use cases and Applications 

Previous Tutorial : Apache Kafka – Terminology 


  1. I am writing a Business Justification Document to sunset the use of a database and utilise Kafka as the data we have is ingested by multiple DB's, Apps etc. I was hoping to find out whether there is a white paper available to assist in this decision making and justifying the business benefits etc.

  2. In your Job paper body, you should put your exchange subsequent to expressing every contention.Best Essay Writing Service UK