I’m happy to announce that Karafka 2.1 has just been released. For those who wonder what Karafka is, Karafka is a Ruby and Rails multi-threaded efficient Kafka processing framework. The Karafka 2.1 release builds upon the foundation set by its predecessor, 2.0, making it a seamless continuation rather than a major rewrite. This means that …
Kafka topics are a fundamental concept in Apache Kafka. Topics are logical names or labels representing a stream of messages that Kafka clients can produce and consume. What makes them interesting is the variety of settings that can be applied to them. These settings, amongst others include: Partition count: The number of partitions that a …
Kafka topics as code – declarative Kafka topics management in Ruby Read More »
I’m thrilled to announce the new and shiny addition to the Karafka ecosystem: Karafka Web. For those who wonder what Karafka is, Karafka is a Ruby and Rails multi-threaded efficient Kafka processing framework. Karafka has always been a convenient framework, and I’ve abstracted or hidden many complexities related to working with Apache Kafka. However, the …
Karafka Web UI – Your Ruby and Rails out-of-the-box Kafka UI Read More »
I’m thrilled to announce the new and shiny Karafka 2.0. It is an effect of my work of almost four years. For those who wonder what Karafka is, Karafka is a Ruby and Rails multi-threaded efficient Kafka processing framework. Karafka 2.0 is a major rewrite that brings many new things to the table but removes …
When doing multi-threaded work in Ruby, there are a couple of ways to control the execution flow within a given thread. In this article, I will be looking at Thread#pass and Queue#pop and how understanding each of them can help you drastically optimize your applications. Thread#pass – what it is and how does it work …
Reduce your method calls by 99.9% by replacing Thread#pass with Queue#pop Read More »
Bug hunting Recently I’ve been working with one small application that would gradually become slower and slower. While there were many reasons for it to happen, I found one of them interesting. To give you a bit of context: the application was a simple single topic legacy Kafka consumer. I rewrote it to Karafka, and …
This release mostly solves problems related to message deserialization and normalizes some of the naming conventions to ease during the upgrade to the upcoming 2.0 version. Note: This release is the last release with ruby-kafka under the hood. We’ve already started the process of moving to rdkafka-ruby. Note: If you are using Sidekiq-Backend plugin, please …
Karafka framework 1.4.0 Release Notes (Ruby + Kafka) Read More »
Note: These release notes cover only the major changes. To learn about various bug fixes and changes, please refer to the change logs or check out the list of commits in the main Karafka repository on GitHub. TL;DR If you would prefer to see the changes in the code, here’s the upgrade PR from the …
Karafka framework 1.3.0 Release Notes (Ruby + Kafka) Read More »
