Sometimes you might want to access Apache Kafka that’s running on your local machine from another device not on the same network. I’m not sure I can think of a production use-case, but there are a dozen examples for sandbox, demo, and playground environments.
In this post we’ll see how you can use ngrok to, in their words, Put localhost on the internet. And specifically, your local Kafka broker on the internet.
Here’s a collection of Kafka-related talks, just for you.
Each one has 🍿🎥 a recording, 📔 slides, and 👾 code to go and try out.
Data comes into Kafka in many shapes and sizes. Sometimes it’s from CDC tools, and may be nested like this:
(This was cross-posted on the Confluent.io blog)
This question comes up on StackOverflow and such places a lot, so here’s something to try and help.
tl;dr : You need to set advertised.listeners (or KAFKA_ADVERTISED_LISTENERS if you’re using Docker images) to the external address (host/IP) so that clients can correctly connect to it. Otherwise they’ll try to connect to the internal host address–and if that’s not reachable then problems ensue.
Put another way, courtesy of Spencer Ruport:
LISTENERSare what interfaces Kafka binds to.ADVERTISED_LISTENERSare how clients can connect.
tl;dr Use curl to pull data from the Mockaroo REST endpoint, and pipe it into kafkacat, thus:
curl -s "https://api.mockaroo.com/api/d5a195e0?count=2&key=ff7856d0"| \
kafkacat -b localhost:9092 -t purchases -P
