Kafka

Flowgen produces messages to Apache Kafka topics and consumes topics into flows.

• Produce — sends the incoming event to a topic and emits the delivery result downstream.
• Subscribe — consumes every partition of a topic and emits each record as an event.

Credentials

credentials_path is optional and points to a JSON file with authentication details. Omitting it connects to the brokers without authentication.

Both sasl and ssl are optional, and which blocks are present decides the security protocol:

SASL_SSL is what managed Kafka (Confluent Cloud, Amazon MSK) expects. Set security_protocol to override the derived value; a credentials file with no sasl and no ssl is rejected rather than silently connecting in plaintext.

{
  "sasl": {
    "username": "user",
    "password": "pass",
    "mechanism": "SCRAM-SHA-256"
  },
  "ssl": {
    "ca_location": "/etc/kafka/ca.pem",
    "certificate_location": "/etc/kafka/client.pem",
    "key_location": "/etc/kafka/client.key",
    "key_password": "secret"
  }
}