kafka

Monitoring Karafka Jobs Progress Using Web UI

Introduction Karafka is a Ruby and Rails framework designed to simplify processing messages consumed from Apache Kafka. One of Karafka’s components is the https://karafka.io/docs/Web-UI-Getting-Started/. It provides a convenient way for developers to monitor and manage their Karafka-based applications without using the command line or third-party software. The interface, amongst others, includes: historical metrics, real-time aggregated […]

Monitoring Karafka Jobs Progress Using Web UI Read More »

Delaying Kafka Messages Processing with Karafka: A Deep Dive into Delayed Topics

Kafka is a popular distributed streaming platform that is commonly used for building real-time data pipelines and streaming applications. One of the core features of Kafka is its ability to handle high-volume, real-time data streams and reliably process and distribute them to multiple consumers. However, in some cases, it may be necessary to postpone the

Delaying Kafka Messages Processing with Karafka: A Deep Dive into Delayed Topics Read More »

Kafka topics as code – declarative Kafka topics management in Ruby

Kafka topics are a fundamental concept in Apache Kafka. Topics are logical names or labels representing a stream of messages that Kafka clients can produce and consume. What makes them interesting is the variety of settings that can be applied to them. These settings, amongst others include: Partition count: The number of partitions that a

Kafka topics as code – declarative Kafka topics management in Ruby Read More »

Karafka Web UI – Your Ruby and Rails out-of-the-box Kafka UI

I’m thrilled to announce the new and shiny addition to the Karafka ecosystem: Karafka Web. For those who wonder what Karafka is, Karafka is a Ruby and Rails multi-threaded efficient Kafka processing framework. Karafka has always been a convenient framework, and I’ve abstracted or hidden many complexities related to working with Apache Kafka. However, the

Karafka Web UI – Your Ruby and Rails out-of-the-box Kafka UI Read More »

Karafka framework 2.0 announcement

I’m thrilled to announce the new and shiny Karafka 2.0. It is an effect of my work of almost four years. For those who wonder what Karafka is, Karafka is a Ruby and Rails multi-threaded efficient Kafka processing framework. Karafka 2.0 is a major rewrite that brings many new things to the table but removes

Karafka framework 2.0 announcement Read More »

Karafka framework 1.4.0 Release Notes (Ruby + Kafka)

This release mostly solves problems related to message deserialization and normalizes some of the naming conventions to ease during the upgrade to the upcoming 2.0 version. Note: This release is the last release with ruby-kafka under the hood. We’ve already started the process of moving to rdkafka-ruby. Note: If you are using Sidekiq-Backend plugin, please

Karafka framework 1.4.0 Release Notes (Ruby + Kafka) Read More »

Karafka framework 1.3.0 Release Notes (Ruby + Kafka)

Note: These release notes cover only the major changes. To learn about various bug fixes and changes, please refer to the change logs or check out the list of commits in the main Karafka repository on GitHub. TL;DR If you would prefer to see the changes in the code, here’s the upgrade PR from the

Karafka framework 1.3.0 Release Notes (Ruby + Kafka) Read More »