Kafka Pattern Authorizer

A lightweight, stateless, zero-management Authorizer for Apache Kafka that maps client identities directly to resource access using patterns.

Motivation

In dynamic microservice environments, managing Kafka access via traditional ACLs is an operational bottleneck. Standard ACLs require stateful management, manual intervention, and often lead to "permission drift" where services retain access they no longer need.

When choosing an authorization strategy for Apache Kafka, there are generally two extremes:

• Complex Solutions: Using standard Kafka ACLs, OPA (Open Policy Agent), or RBAC. These require centralized state management, complex automation pipelines, sidecars, or heavy administrative burden.
• Not Caring: Skipping authorization altogether, which avoids operational overhead but introduces significant security risks.

The Pattern Authorizer solves it in a simple way. It takes a decentralized, stateless approach by encoding permissions directly into the client's identity (e.g., mTLS certificate). This provides robust access control without the operational bottlenecks of traditional centralized solutions. The identity is the permission. When a service is decommissioned or its certificate expires, its access is automatically revoked with zero manual cleanup.

Installation

To enable the Pattern Authorizer, download the compiled JAR (or build it from source), place it into the Kafka libs/ directory, and set the following properties in server.properties:

# Enable Pattern Authorizer
authorizer.class.name=io.github.rmrustem.kafka.authorizer.PatternAuthorizer

# Map incoming TLS certificates to principals
ssl.principal.mapping.rules=RULE:^CN=(.*?),.*$//$1/L,DEFAULT

# (Optional) Separate admin user with OU
# super.users=User:ADMIN
# ssl.principal.mapping.rules=\
#       RULE:^.*OU=ADMIN.*$/ADMIN/U, \
#       RULE:^.*CN=([^, ]*).*$/$1/L, \
#       DEFAULT

Permission Rules

The Authorizer evaluates the Principal string presented by the client to determine its permissions. The string should follow this key logic:

• Multiple permission patterns can be joined with pipe | characters.
• TopicName@GroupName grants Consumer (Read) access to that topic using the specified consumer group.
• TopicName grants Producer (Write) access to that specific topic.
• * acts as a wildcard for both topic names and group names. For example, * alone grants Producer access to all topics, while *@* grants Consumer access to all topics across all consumer groups.
• *@GroupName grants permission to consume any topic, but restricted to the specified consumer group.

Examples

• orders grants producer access to the orders topic
• orders|payments grants producer access to bot the orders and the payments topic

• orders|payments|*@shipping grants producer access to the orders and payments topics, and consumer access to all topics under the shipping consumer group.

Kubernetes Integration

In Kubernetes environments, this pattern becomes highly automated using cert-manager (often backed by Vault or another PKI). Developers simply request a certificate, declaring their required Kafka access directly in the commonName of their manifest.

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: my-app-kafka-tls
  namespace: my-app
spec:
  secretName: my-app-kafka-tls-secret
  issuerRef:
    name: vault-issuer
    kind: ClusterIssuer
  # The CN defines the exact topics and consumer groups this app can access
  commonName: "orders|payments@shipping"
  duration: 24h
  renewBefore: 8h

When the app is deployed, cert-manager mints a short-lived mTLS certificate with those exact permissions. When the app is removed or the certificate expires, access is automatically revoked—zero ACL management required.