The Kafka Connector is quite simple. It will send every record of a fluvio topic to a kafka cluster for a given kafka topic.
# kafka-sink-connector.yml
name: my-kafka-sink
type: kafka-sink
topic: fluvio-output-topic
parameters:
kafka-url: "localhost:9092"
kafka-topic: kafka-input-topic
kafka-option:
linger.ms: 10
topic.metadata.refresh.fast.interval.ms: 10
topic.metadata.refresh.fast.cnt: 10
kakfa-url
is requiredkafka-topic
is optional and will default to the top level topic
which is
the topic used on the fluvio side of things.kafka-partition
is option and will default to 0
unless specified.kafka-option
is an option that is specified as a dictonary. The Configuration
properties are from the rdkafka
options.This configuration file is used together with the fluvio connector create
command, like so:
$ fluvio connector create --config=./kafka-sink-connector.yml
Events are sent to fluvio as raw bytes. The record are sent along to fluvio as well.