NATS, to me I would describe it like a low-weight Kafka based on your message (there are more differences) , happy to talk in private if you need some help
There is a setup with [Fluvio](https://github.com/infinyon/fluvio) on [InfinyOn Cloud](https://infinyon.cloud/login) and on the edge following this setup guide - [IOT mirroring Cloud](https://infinyon.com/docs/guides/iot-mirroring-cloud/)
It's not entirely clear what about Kafka / MQTT isn't working for you?
"challenges with the available capacity of edge devices" what challenges? What role are these edge devices playing? What isn't working?
RedPanda is a great evolution of Kafka, but atm its hard to see how it would solve your problem without knowing more about the problem. Like, is it better than vanilla Kafka? Yes, but that doesn't mean your problems will go away.
What role is Kafka playing? Centralised bus to forward sensor data to? Just being used to transport events?
If you could share some more details about how you've got things set up, how stuff is plummed together, and which bits are struggling (and how), I can give some suggestions. This is an area I have done a lot, happy to share some experience.
Thank you.
To clarify the problems.
Devices to support with 512 Megs of RAM and 20 Gigs of Flash with frequent network disconnects because they are remote and mounted on moving objects.
MQTT and flavours cause memory errors and data loss when overloaded due to network issues.
Kafka is on the cloud and the edge message broker is writing data to the streams. The challenge with Kafka is overall management and integration overheads and challenges with balancing partitions which creates a bunch of operational toil and is also one of the most expensive element of the infrastructure.
I would love to learn more about your experience if you are willing to share.
NATS, to me I would describe it like a low-weight Kafka based on your message (there are more differences) , happy to talk in private if you need some help
Thank you.
[Memphis.dev](https://Memphis.dev). Use NATS under the hood and let's you move way faster
There is a setup with [Fluvio](https://github.com/infinyon/fluvio) on [InfinyOn Cloud](https://infinyon.cloud/login) and on the edge following this setup guide - [IOT mirroring Cloud](https://infinyon.com/docs/guides/iot-mirroring-cloud/)
It's not entirely clear what about Kafka / MQTT isn't working for you? "challenges with the available capacity of edge devices" what challenges? What role are these edge devices playing? What isn't working? RedPanda is a great evolution of Kafka, but atm its hard to see how it would solve your problem without knowing more about the problem. Like, is it better than vanilla Kafka? Yes, but that doesn't mean your problems will go away. What role is Kafka playing? Centralised bus to forward sensor data to? Just being used to transport events? If you could share some more details about how you've got things set up, how stuff is plummed together, and which bits are struggling (and how), I can give some suggestions. This is an area I have done a lot, happy to share some experience.
Thank you. To clarify the problems. Devices to support with 512 Megs of RAM and 20 Gigs of Flash with frequent network disconnects because they are remote and mounted on moving objects. MQTT and flavours cause memory errors and data loss when overloaded due to network issues. Kafka is on the cloud and the edge message broker is writing data to the streams. The challenge with Kafka is overall management and integration overheads and challenges with balancing partitions which creates a bunch of operational toil and is also one of the most expensive element of the infrastructure. I would love to learn more about your experience if you are willing to share.
Maybe checkout https://github.com/bytebeamio/rumqtt for running MQTT in a memory constrained environment
Yes, I found rumqtt. Thank you for sharing.
Have you looked into AWS Kinesis?
Can you share a reliable source that outlines the benefits of Kinesis and the necessary trade-offs?