Use case: I am writing a Kafka consumer for my node-red project. Libraries used: node-red-contrib-kafkajs, node-red-contrib-kafka-manager Error: "Snappy compression not implemented" / "Snappy codec is not available"
Question
I have tried most of the node-red Kafka libraries & but all libraries throw error or shutdowns application, saying snappy compression is not supported.
There are no configuration available to support kafka message compression.
Can anyone help me on this? Is anyone able to write a kafka consumer, which can consume messages generated from a non node-red producer?
Do you have the snappy codec installed? It's not explicitly required by us in setup.py, but if you want to use that feature you'll need python-snappy installed
That would need to be raised as an issue or feature request on the node in question. You can find the repository by visiting the NPM page for each of the kafka nodes where you can raise issues.
Yes, I have snappy installed.
I don't require python-snappy, since my application is not python based, hoewever I have snappy installed. Still getting the same error
To resolve this, I had to write my own custom - library in node-red using kafkaJS.
Kafka JS does not support snappy compression by default hence all libraries using kafkaJS throw this error.
We need to use KafkaJS & add Snappy dependency manually & write a custom Consumer node to resolve this, no built-in support exist in any library/depedency as of now.
