Protobuf vs Avro for Kafka, what to choose?

Kateryna — Wed, 01 Mar 2023 13:46:32 +0000

Hey there!👋 Recently I had a few quite insightful evenings comparing possible options for the Kafka messaging format, so I gathered some useful articles for you.

Data Serialization: Avro vs Protobuf
Awesome comparison of Protobuf and Avro. Well structured, and detailed where it's needed. This article summarizes traits, pros and cons, and leaves you with a basic understanding of when and where to use these formats.
What you can do with Protobuf and can't with Avro? and vice versa
Fabulous article with a detailed overview of how serialization works with Protobuf, Avro and JSON Schema. With real-life code samples and connectivity with schema registry samples.
Arvo and Protobuf performance compared
Quite a good example if you are a fan of "example with real numbers". This article shows the serialization of both types in action. Definitely worth reading.
Kafka Schema evolution
If your schema changes often and you wanna a format that will simplify this process as much as possible, this article will save you time and energy.

P.s my personal preference -- Protobuf (100%), however, the reading was fun

Kafka & Docker Golden Finds

Kateryna — Mon, 27 Feb 2023 13:44:18 +0000

Hey there👋! For your attention: list of awesome Kafka & Docker-compose articles/tutorials. Hopefully, I will save someone's evening or two from the struggle🤗

Docker-compose cookbook
One of the most fabulous cheat sheets I ever saw! In this repo, you will find almost every possible set-up of docker-compose for popular services, including Kafka😉
Kafka Security set-ups in Kubernetes
Great set of examples of Confluent Kafka configurations for different types of authorization
Grokking Kafka Listeners with Confluent
Old but good. great explanation of Kafka Listeners with real-life examples and pretty drawings (if you're a visual person -- this article will close a lot of gaps for you)
Troubleshooting connection for all possible set-ups of your Kafka

How to connect to the broker running in a container from the local host? From the same container? From a different machine? Anything U wish😉

P.S. Seems as if Kafka wouldn't let me go easily, so get yourself comfortable, this series has the tag "to be continued"😁 Cheers!

Forem: Kateryna

Protobuf vs Avro for Kafka, what to choose?

Kafka & Docker Golden Finds