frogo-cli: My attempt at making Kafka (a bit) more hackable
Hey all! I’ve worked with Kafka here and there over the past few years and I’ve been really frustrated with the tooling.
I took a stab at a tool which simplifies reads and writes to topics down to:
frogo get <topic> —from <offset-like> —to <offset-like>
frogo put <topic> —file <file-w-one-msg-per-line>
GitHub link: frogo-cli
As a brief overview:
For ‘frogo get’ - the main idea is that an ‘offset-like’ supports literal offsets, timestamps, dates, and aliases (START, END, FUTURE).
For ‘frogo put’ - the main idea is you have a file which has one message per line. Multiple formats are supported (e.g. base64 for binary data)
Some other features I’ve added:
- mockserver: thin wrapper exposing a franz-go mock server (frogo mockserver)
- configuration profiles (use —profile or FROGO_PROFILE)
- multiple input / output formats (—format)
- fixture topics with example data (frogo topic demo)
This tool is by no means comprehensive, and I haven’t added support for things like:
- consumer groups
- schemaregistry
- certain authN / authZ configs
But… I hope this could serve as a helpful development / ops tool for those not needing all the bells and whistles.
Any feedback / criticism would be much appreciated. Feel free to create an issue on the GitHub for any feature requests!