We are hiring! Apply here
Outbound Kafka Connector

The Kafka Connector is quite simple. It will send every record of a fluvio topic to a kafka cluster for a given kafka topic.

 

Common config values

type: kafka-sink
version: 0.3.1
 

Parameters

 

kakfa-url

required

The url of the Kafka instance to connect to.

 

kafka-topic

Default: Same name as the configured fluvio topic

The name of the Kafka topic name to connect to.

 

kafka-partition

Default: 0

The Kafka partition to connect to.

 

kafka-option

optional

option that is specified as a dictonary.

See: The Configuration properties are from the rdkafka options.

 

security

 

security-protocol

optional

The kafka security protocol.

Currently only supports SSL.

 

Secrets

 

FLUVIO_KAFKA_CLIENT_KEY

optional

The SSL key pem text.

 

FLUVIO_KAFKA_CLIENT_CERT

optional

The SSL cert pem text

 

FLUVIO_KAFKA_CLIENT_CA

optional

The SSL CA pem text

 

Example connector config

name: my-outbound-kafka
type: kafka-sink
topic: fluvio-output-topic
version: 0.3.1
parameters:
 kafka-url: kafka-ssl.host.tld:9092
 security-protocol: 'SSL'
secrets:
 FLUVIO_KAFKA_CLIENT_CA: |
   -----BEGIN CERTIFICATE-----
   MIIEQTCCAqmgAwIBAgIULrWm1Uq3+NpQsgmG71auMAv0GcgwDQYJKoZIhvcNAQEM
   BQAwOjE4MDYGA1UEAwwvZTE1OWM1YTktODNhZC00ZGIxLTlmZTItNWU5OTYyMTRh
   NGFjIFByb2plY3QgQ0EwHhcNMjIxMTA4MTgxOTQ4WhcNMzIxMTA1MTgxOTQ4WjA6
   MTgwNgYDVQQDDC9lMTU5YzVhOS04M2FkLTRkYjEtOWZlMi01ZTk5NjIxNGE0YWMg
   UHJvamVjdCBDQTCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAL7I5SPc
   eHsBAtuH07iZ2aLln0NrNSimFd5DuSPMu0kVOQk9UVHHnKU1AHHv0GjsoF+JcMPw
   D3UzK2ljxK1nYPagC0blMxBs80l+ml7MYNSncweE2p1p1NlZ6pWuZ3ZT0ZpK2K6W
   S0THmAVfeyhYUYC7sS42dfe+7y3ZbCegnYwx8dovaYoRthNTHatTA9XK8ZOBmNz8
   T6LKqaM6nydLNhb+qjkm2VdBPUjmiRcfxP8DKE1aJnEUMwsTnXIAq5nxQl7z7NBH
   mxxnjidNjWtXIOkoI7OZSD3K4Qqwn4/Uuvn3qSjAvLTo18e8RcUmMrarBGbb+r0r
   VrZqsIzNQweQtV7SqRemHC0v9Sa8tojK3xHbXdFPEmqt4c1rCdrrZI7D/EvdKatI
   12yT1xXBChNvkTLEqt5C3f0JH7TtmqdU49ZNGV/cJka752IqeabOMaHFeIARMZEv
   cb+HZR+ODEe9fQ2Vp0uk6emNNA4nCv9HZIkt/Ut/LPNCJsAepLPgWQCPFwIDAQAB
   oz8wPTAdBgNVHQ4EFgQUYBtj2SI4WMkIouvfseCuCYdOSr8wDwYDVR0TBAgwBgEB
   /wIBADALBgNVHQ8EBAMCAQYwDQYJKoZIhvcNAQEMBQADggGBAEf0D2FDA8IO6RyN
   EOnZNwSy/6kFZSDg/EGJDbTB+MS287AZOlaREGcdNKjSe6vJRQT3F9xC43XbwEcQ
   zzjeukjEQUov6Nb98KO+tChcqE7Lhl0Lr2TEfv2kefXvkbZbGX7M6zhZrSdgrtZ9
   EB2aGPF4m+Yc7z4LM4WEcQsyawPMkZNIJg7CGn0s9premcu9m5OUz04G2nDsVWs9
   jE7TFeIqn/kxLzkSK7EanpO58/L8yfqCUKdTYldaT0xrIi4GhA+vIoUAFQsM8nJQ
   4+92VjOHGVbxyalMQLALpEItXrQZv7g2naFPWX50A8RWZdZ6JwQVyNoPu2co/Fa1
   8lQIi8BqZK/zS0PzpPegFOgalofpXxGduENFpeLCpWT+/pGLhNI+56cLi8Arz2tE
   UMpPbY6UPs6IfKD+D/k2TsaFDqqwb8iD1QDzc+Kt/1Kw2wV9w+0oeo1CPgt+9fW3
   peTDDglPT0sjQU77PhRxzbaCNdKyHnZr2NM2Adz7GXrbO54giA==
   -----END CERTIFICATE-----   
 FLUVIO_KAFKA_CLIENT_CERT: |
   -----BEGIN CERTIFICATE-----
   MIIEQDCCAqigAwIBAgIUI8E080fbqB5tu0jRIBWSJn8ZIFcwDQYJKoZIhvcNAQEM
   BQAwOjE4MDYGA1UEAwwvZTE1OWM1YTktODNhZC00ZGIxLTlmZTItNWU5OTYyMTRh
   NGFjIFByb2plY3QgQ0EwHhcNMjIxMTA4MTgyMjE4WhcNMjUwMjA1MTgyMjE4WjA/
   MRcwFQYDVQQKDA5rYWZrYS1zc2wtdGVzdDERMA8GA1UECwwIdTFpMGozdzAxETAP
   BgNVBAMMCGF2bmFkbWluMIIBojANBgkqhkiG9w0BAQEFAAOCAY8AMIIBigKCAYEA
   pX7Hpdu7j7qq/JlgMLwFywTGTra7OYUifrH/Tq7ad++TRqs8RqZWC+c+snpFje94
   P+dXGQJwTsUITffg4bPCvhvPLHO0g/vdldOvgOIf52tLHceWaj4Hh396q8z8zAPZ
   SNJ5l0GVt9/f9KcnG8rvKaSUWj1jsW1cfGgUZEhii7KlFO3Z3VLXt7SXOGfLX2yZ
   QxYBvpbeHSmz742Ii5jBGkOmwBJvYIx5YHbryy8AKr1VIJft/yzf+1W2endB0INA
   0G7CoJsxZo3BL95lzkrn1Nynq2cvGcYskQM7kNa17aKuE2GNzL0IjwIEuqyCZHY/
   docnYjErTX5pcJHdpIdBjfK/w/a7ulQT7obqXp/h2krMIT4JaNrqIc6Er86t1bGY
   iUAmrA3yoaGUkfoEirCT3zpscENw9SX7leB/2UU2VLoKQ0br8GvbnmH9rZeK/Gzg
   5Q45z+Kf2vzLc3ecelsKfkt9eGdDZf7BaWgjBjdMzSQydZOIW/rdsxiD9xeO0XAt
   AgMBAAGjOTA3MB0GA1UdDgQWBBQVByHTjDGMAkDxSl4ZJfsd0+9ytjAJBgNVHRME
   AjAAMAsGA1UdDwQEAwIFoDANBgkqhkiG9w0BAQwFAAOCAYEAeKpjksILVFudG17G
   Lt7Ha7gA58OwXawMz5BOYk+RlV/mBTBg1CbFRNk6ezyPxjZLwHWVe+iv+y45K5Ob
   dRrYVog792VGTDUruH8GCBhR5rlwMlQN+mJBKIn3mT3/cuL7Pfb5rPZpRjnycplY
   3Qtw+1iygfDVUU4fYMN0zH8uGjqjh1tr6nk5wYaNmHRDy345y2N3AOKIoTFjYgvQ
   6nBSYpOq/MTvwSPshk+FXhVAxRBTbZh398Fzo0HnFViUBNFC/kF83SNQeMjGLrhE
   4cp8WT0CY+3Mc0L9xNLugwZBJPF5KU3U+iMW4IK2mtkQIKjF+oZXUsPxoL2jjCzw
   GWl0jNfL9c0ftbvYRvGJ0TB9P8z1jdWbbAZ4jnW65bH9v/dzlJ99XZ/RPwxzzlx8
   pl0WrQxUVOX5B0f064DMrzMEhRvIrR+maRMqlPtKESRvTmdcOOftEm4cECtY3Wh4
   JXmarWObSnpGRVMfFF7/XryjIrKc32+iYOT4y+vIxctTpeiP
   -----END CERTIFICATE-----   
 FLUVIO_KAFKA_CLIENT_KEY: |
   -----BEGIN PRIVATE KEY-----
   MIIG/wIBADANBgkqhkiG9w0BAQEFAASCBukwggblAgEAAoIBgQClfsel27uPuqr8
   mWAwvAXLBMZOtrs5hSJ+sf9Ortp375NGqzxGplYL5z6yekWN73g/51cZAnBOxQhN
   9+Dhs8K+G88sc7SD+92V06+A4h/na0sdx5ZqPgeHf3qrzPzMA9lI0nmXQZW339/0
   pycbyu8ppJRaPWOxbVx8aBRkSGKLsqUU7dndUte3tJc4Z8tfbJlDFgG+lt4dKbPv
   jYiLmMEaQ6bAEm9gjHlgduvLLwAqvVUgl+3/LN/7VbZ6d0HQg0DQbsKgmzFmjcEv
   3mXOSufU3KerZy8ZxiyRAzuQ1rXtoq4TYY3MvQiPAgS6rIJkdj92hydiMStNfmlw
   kd2kh0GN8r/D9ru6VBPuhupen+HaSswhPglo2uohzoSvzq3VsZiJQCasDfKhoZSR
   +gSKsJPfOmxwQ3D1JfuV4H/ZRTZUugpDRuvwa9ueYf2tl4r8bODlDjnP4p/a/Mtz
   d5x6Wwp+S314Z0Nl/sFpaCMGN0zNJDJ1k4hb+t2zGIP3F47RcC0CAwEAAQKCAYAX
   h60PMey7BMlm3M/GXhVcxjMD4gtApGNBLEGmiEnbn56K/7G0QdYeHyTU73mQMqog
   O94wWh5ptK33l/TLYmmf6ghMw664zUi2EsDzTDNJDgJCG6yRMnyTkVZ5ss2GnE16
   gyJylQqaeCQ5fsKauwHCI/pL0kqoYWoy7gFoOLp6Lk42aZL+Wtb88ZpGlMtfp+sq
   lKhu1riB4PoKZ08wuelKuK+uw3A+g/qF8qKX4vjdGspNtgxKqbC25XrqcqZS5TZJ
   YDweek6nfKSvgZt+xusddDV9t0it5dubxsxVlbL7tKa5aq5EldhNu8HrJXJU8Kfw
   6rRm3vycmOujA9HpS9lVyahcnTP4UUGRF2xw47C2v8uMCvZDx1pcyrnTIvWCzl1J
   j6kBk/gH3hMS+9HLIm9fwsbO6iMnwBbdZg7Mil8opubIRLIqmW7wTbwqctir312Y
   eFgYPOB1GjlGMXZQefb4D6gS0UfxkHpER3x9eXcTriQyrqmVsSq3jLZnS+yTP+kC
   gcEA0sCJvUBT8jKT2AdVdirXbq/Q9iIGXvYKxEqrv8tKSt8rg0ypX+zmGlinbp+8
   xLZFwOXhXH5XKcOBczhNNiRbaz8DnSHswfJaH7/BWaZ+G33Z56S6BPvsa5uZytCm
   k7CiTCrvZj21jLsYjcddKQtZhYF4ZZQy06zbq1xkklJcyQI201Jx+xZ384UbmlIU
   BZ1A2dGhHE6vjM6MYlZwY1GepokIfLLDrc7yyPUQa1n8JFj1ydIYCRrgE0pd07Mh
   gfj7AoHBAMkGz7L9UD7NPkNIHjxBa4vxTgBAiFxX+cDJQ0Op+6tYdexMuj7ftxmc
   rl/nQdl49YuT1IM3Ge8ohRYJQrRza7kCzNklyPBvhVLKFdWNC1Kma3+Wf9VZFJDc
   FW11y8GYVuSlfgLORXAqQ4BavnzysJeIv69yI6YxIGy3T+Z714ofeOgjqg6Crb5n
   1Qw11DCrbe+1r2D3GYQ67cnJoIZ2ZbAgFWqpYOz6yZsSSgU+cmuorazUt8zyjTyy
   IIiKnnTC9wKBwQDJ2qOSYfSFOzAjbxLJtUrVtwubUwdrGJlDXImXj2w+vfV9MoXf
   XJUfkt36tIAmoLW6hfMNleyCEcEAzGIXg0cUNyyM6ZtMP4VWd2EHLtIGHsoAdeO9
   NlVA062/dYy2VC1+ankVIfGrMt/Xm6V81GQP3OulzwXHe/6B6GJjSwOXeslsKcEF
   zoVg9pJzgj2vCui8N23qIOs/H+2RmlzWo6wisYkxdj65UEd1dlMdnvhDD/mKLfHb
   dqQ7lbS8OQvU4tUCgcEApvahS2hZXc/aBse/VQyA5ppkzpDKsOfg595Bn9Fu1cGD
   L+AKUG4/BWa87aqFZrqwNl1yBAmf4KMm5kaKwJWcwu0wdqqLWidF9J0jxVqALQVM
   A4xcYDa8fhRNN7y+nuyIPx+0qV/pDGAN8G1uCRkI0tMhYfbK5R2ocKNQGX6CZpX9
   NdFpTOP8qkhTCvEqa8H2bUHZiA+zgb/v/8kXKpbrnn0HTK+hcTJagBGQqQpUvlY6
   6XFDt1U9OUw4+S8orJHBAoHBAKn9yiFwgelj8OGDo6ekG+Szup+DtGFjmfTldP/Q
   gpXvDTDaQ6ScMP4wBWBjPqegz4plX5UMSIh4O8iZbF/ffTeVdlSZtQK8MFIZ2PrM
   8xxLSxQgFfcQBFjS5W6oiLF5ewCaRbDEquXWcYKNKgCVgjtwR7CG7riMX+epMsn2
   KgfeMUXfypqI8nFtTrACmBhpN/d6dASwCrcAEfTacRRwsbjDeidgKFrLJ1Vj3bYE
   xT82f1hAD+c5UzPz9WN91Duu5A==
   -----END PRIVATE KEY-----   
 

Data Events

Events are sent to Fluvio as raw bytes. The event records are sent along to Fluvio as well.