Presentation - Efficient Data Streaming with Event Hubs for Kafka

Recently I did a technical session in Perth Azure User Group at Microsoft Perth. The event was themed as Azure Open Source Night.

My session was on "Efficient Data Streaming with Event Hubs for Kafka"

Following is the presentation I did

Following is the code I demonstrated. In this sample An Azure Function written in .NET 6 pushes events to an event hub, those are consumed by Apache Kafka consumer written in Java. 

Following are the configuration changes you need to perform.

1. In Producer, change the EventHubConnectionAppSetting in local.settings.json file. You can get the connection string by navigating to SAS policies and in the policy you created for sending.

{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsStorage": "",
    "FUNCTIONS_WORKER_RUNTIME": "dotnet",
    "EventHubConnectionAppSetting":"CONNECTION"
  }
}

2. In Consumer, change the Event Hubs namespace and connection string in Consumer.config file.You can get the connection string by navigating to SAS policies and in the policy you created for receiving.

bootstrap.servers=EVENTHUBNAMESPACE.servicebus.windows.net:9093
group.id=$Default
request.timeout.ms=60000
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="CONNECTION";

You can get further information from this repository

I loved the event. Following are few snaps.