Karafka

Karafka framework 2.0 announcement 1

Karafka framework 2.0 announcement

I’m thrilled to announce the new and shiny Karafka 2.0. It is an effect of my work of almost four years. For those who wonder what Karafka is, Karafka is a Ruby and Rails multi-threaded efficient Kafka processing framework. Karafka 2.0 is a major rewrite that brings many new things to the table but removes …

Karafka framework 2.0 announcement Read More »

Reduce your method calls by 99.9% by replacing Thread#pass with Queue#pop 8

Reduce your method calls by 99.9% by replacing Thread#pass with Queue#pop

When doing multi-threaded work in Ruby, there are a couple of ways to control the execution flow within a given thread. In this article, I will be looking at Thread#pass and Queue#pop and how understanding each of them can help you drastically optimize your applications. Thread#pass – what it is and how does it work …

Reduce your method calls by 99.9% by replacing Thread#pass with Queue#pop Read More »

The hidden cost of a Ruby threads leakage 19

The hidden cost of a Ruby threads leakage

Bug hunting Recently I’ve been working with one small application that would gradually become slower and slower. While there were many reasons for it to happen, I found one of them interesting. To give you a bit of context: the application was a simple single topic legacy Kafka consumer. I rewrote it to Karafka, and …

The hidden cost of a Ruby threads leakage Read More »

Karafka framework 1.4.0 Release Notes (Ruby + Kafka)

This release mostly solves problems related to message deserialization and normalizes some of the naming conventions to ease during the upgrade to the upcoming 2.0 version. Note: This release is the last release with ruby-kafka under the hood. We’ve already started the process of moving to rdkafka-ruby. Note: If you are using Sidekiq-Backend plugin, please …

Karafka framework 1.4.0 Release Notes (Ruby + Kafka) Read More »

Karafka framework 1.3.0 Release Notes (Ruby + Kafka)

Note: These release notes cover only the major changes. To learn about various bug fixes and changes, please refer to the change logs or check out the list of commits in the main Karafka repository on GitHub. TL;DR If you would prefer to see the changes in the code, here’s the upgrade PR from the …

Karafka framework 1.3.0 Release Notes (Ruby + Kafka) Read More »