Smart Connectors make the process of importing or exporting data simple.
You can import data with an
Inbound connector and export data with an
Inbound and outbound connectors fundamentally work in the same way. The only difference is the direction your data is streaming with respect to a Fluvio topic.
There are 4 steps to the connector:
The Protocol and Extract stages are built directly into the connector. They offer basic access to your data through the various protocols your data sources use.
In the Extract stage, your data is structured from whatever protocol it is sourced from.
Additionally, You can apply custom pre-processing or post-processing to data, before it arrives to or while it streams from a Fluvio topic. The Filter and Shape stages are provided through SmartModules.
Powered by WebAssembly (also called wasm), SmartModules are pre-packaged or user-provided operations such as filters, maps, or aggregators that can be applied to records at various points in the streaming pipeline. Supporting access to your data while it is in transit provides you the ability to clean, transform and enrich your data before it is stored in a topic, or it exits the Fluvio cluster.
Use Connectors either as:
You can customize how your connectors run through a configuration file.
For more info about connectors or configuration, check out our supported Inbound and Outbound connector docs.