Introduction Now and then, I like to go on a performance improvement hunt because life isn’t just about adding new features. Recently, I have been focusing on enhancing efficiency, particularly regarding CPU and memory usage in Karafka. Three of my recent pull requests (PR 117, PR 118, PR 123), have made some minor improvements, and […]
Under the Hood: Enhancing Karafka’s CPU and Memory Efficiency Read More »
Opening Note We all make mistakes, and fundamentally, the havoc caused by this incident was due to a flaw in the design of rdkafka-ruby. While the disappearance of librdkafka from GitHub was unexpected, this article aims to clarify and explain how rdkafka-ruby should have prevented it and what was poorly designed. By examining this incident,
The librdkafka Supply Chain Breakdown: rdkafka-ruby’s Darkest Hour Read More »
As an open-source developer, I constantly seek performance gains in the code I maintain. Since I took over rdkafka from AppSignal in November 2023, I promised not only to maintain the gem but to provide a stream of feature improvements and performance enhancements. One key area where performance can often be improved is how synchronization
From Sleep to Speed: Making Rdkafka Sync Operations 16 Times Faster Read More »
Introduction I am thrilled to announce the release of Karafka 2.4, a significant milestone in the evolution of my Ruby and Rails multi-threaded efficient Kafka processing framework. This release builds upon the solid foundation laid by its predecessor, Karafka 2.3. It introduces many new features, enhancements, and improvements across the entire Karafka ecosystem, including Karafka,
Karafka 2.4 Release Announcement: Advancing Kafka Processing for Ruby and Rails Read More »
Introduction In the world of Kafka and its applications, utilizing regular expressions for topic subscriptions is a common strategy. This approach is particularly beneficial for dynamically managing data. For example it can be used to handle information from various zones without necessitating application redeployment for each new topic. For instance, businesses operating across multiple zones
From Oniguruma to POSIX: The Regex Rift Between Ruby and C Read More »
Introduction The journey towards efficient parallelization in library development has often been based on using threads. As Karafka celebrates its eighth anniversary, it’s become clear that while threads have served us well for many tasks, there’s room to explore further. That’s why I’ve decided to introduce forking capabilities into Karafka, aiming to offer another dimension
The Art of Forking: Unlocking Scalability in Ruby Read More »
Introduction Leaving Cracow’s familiar scenes behind, I headed to Warsaw with anticipation for the Ruby Warsaw Community Conference. The compact yet promising event marked a day dedicated to Ruby enthusiasts like myself. Below, you can find my after-thoughts about this peculiar event. Speaker’s Dinner and Before Party The speakers’ dinner and pre-party set the tone
Ruby Warsaw Community Conference 2024: A Compact Adventure in the Heart of Poland Read More »
Introduction I’m happy to announce that Karafka 2.3 and its Web UI 0.8 have just been released. For those who wonder what Karafka is, Karafka is a Ruby and Rails multi-threaded efficient Kafka processing framework. The Karafka 2.3 release builds upon the foundation set by its predecessor, 2.2, making it a seamless continuation rather than
Karafka Framework 2.3 + Web UI 0.8 Release Announcement Read More »
While officially End-of-Life (EOL), Ruby 2.7 remains critical in many ongoing projects. Despite its EOL status, a significant user base continues to rely on this version for various reasons, ranging from legacy system compatibility to specific feature dependencies. As a developer of Karafka, an open-source software (OSS), I recognize the importance of supporting Ruby 2.7,
The Ruby 2.7 Challenge: Adapting to Bundler’s Latest 2.5+ Update Read More »
Introduction Karafka is a Ruby and Rails framework designed to simplify processing messages consumed from Apache Kafka. One of Karafka’s components is the https://karafka.io/docs/Web-UI-Getting-Started/. It provides a convenient way for developers to monitor and manage their Karafka-based applications without using the command line or third-party software. The interface, amongst others, includes: historical metrics, real-time aggregated
Monitoring Karafka Jobs Progress Using Web UI Read More »
