Extend Kafka Topic Modeling with Diffusion
November 24, 2020 | Matias Prados
Diffusion Real-Time Event Stream
Introduction to Diffusion Real-Time Event Stream through a simple application using Diffusion Cloud and Apache Kafka.
A set of simple projects, illustrating production and consumption of foreign exchange (fx) data streams to and from Kafka clusters and Diffusion Cloud instance via the use of our Kafka Adapter.
Lesson 1: Publish event streams to Kafka cluster
Lesson 2: Fine-grained distribution of Kafka event firehose with Topic Views
Lesson 2 code example introduces the concept of Topic Views, a dynamic mechanism to map part of a server’s Topic Tree (data structure) to another. This enables real-time data transformation before sending it to subscribed clients as well as to create dynamic data models based on on-the-fly data (eg: Kafka firehose data).
Lesson 3: Security with Topic-level access control
Lesson 3 code example introduces the concept of Security and topic path permissions for fine-grained security management of your data structure. It also shows how to Subscribe to fx data using Diffusion Topic Views in order to consume what you need, not all the Kafka stream.
- Download our code examples or clone them to your local environment:
- A Diffusion service (Cloud or On-Premise), version 6.5.0 or greater. Create a service here.
- Follow our Quick Start Guide and get your service up in a minute!
public/kafka2diffusion.html‘ and ‘
public/js/producerApp.js‘ and ‘
public/js/subscriberApp.js‘ to the hostname of your Diffusion Cloud service, which you can find in your service dashboard. You can also leave the default values and connect to our sandbox service:
- host: host (“kafkagateway.us.diffusion.cloud” by default)
- user: ‘user’
- password: ‘password’
Really easy, just open the ‘
index.html‘ file locally and off you go!
The code in action
(Click to watch the video)
Get familiar with our SDK, our user friendly dashboard and all the power of our platform.
Watch our Webinar
Tuned our webinar introducing the basic concept of your Diffusion dashboard, console and Data Wrangling.