Menu
Article Azure

The Missing kcat Event Hub OAuth configuration

Michael D

Michael D

• 2 min read

I've been enamored with kcat. Doing simple operations or checks of Kafka messages always felt tedious because I felt the need to maintain a utility that performs basic broker operations.

Something that threw me off getting started with kcat is that, because it depends on librdkafka, which is written in c, it cannot use Java keystores or JAAS configurations for Kerberos authentication.

Following the configuration guide for rdkafka one can build a working Kerberos configuration with little effort.

Event Hub throws a brick in this window

Azure Event Hub is really great to work with thanks to it's Kafka compatible api. Event Hub doesn't support Kerberos authentication and by default Microsoft instructs users to use a JAAS configuration for SAS and OAuth authentication methods.

The next thing you may do is look up "kcat with Event Hub" which will eventually lead you to Microsoft's example repo. Here you will helpfully find both a kcat, and a go example (which also uses librdkafka). These examples will quickly get kcat working SAS token authentication.

metadata.broker.list=<eventhub-name>.servicebus.windows.net:9092
security.protocol=sasl_ssl
sasl.mechanism=PLAIN
sasl.username=$ConnectionString
sasl.password=Endpoint=sb://<eventhub-name>.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=<primary-key>

Using SAS token's are fine and dandy if your Azure governance permits the use of SAS tokens. So are we dead in the water if we need OAuth and cannot use JAAS because we aren't using Java?

Technically no, here is an example that can help you derive how to derive Event Hub's OAuth configuration for kcat.

metadata.broker.list=<eventhub-name>.servicebus.windows.net:9093
broker.version.fallback=1.0
security.protocol=sasl_ssl
sasl.machanisms=oauthbearer
sasl.oauthbearer.method=oidc
sasl.oauthbearer.client.id=<service_principal_clientid>
sasl.oauthbearer.client.secret=<service_principal_client_secret>
sasl.oauthbearer.token.endpoint.url=https://login.microsoftonline.com/<service_principal_tenantid>/oauth2/v2.0/token
sasl.oauthbearer.scope=https://<eventhub-name>.servicebus.windows.net/.default

The key components here are the service principal that grants roles for accessing Service Bus, and the Service Bus's OIDC Scope endpoint.

If you are having trouble, you can visit the openid configuration endpoint for your service principal via:
https://login.microsoftonline.com/<service_principal_tenantid>/.well-known/openid-configuration

With the configuration in place, you can test by listing the broker metadata with kcat pointed to your new configuration file: kcat -L -F kcat.conf.

Cheers 🥂

Share