Fluvio SmartModules Fluvio SmartModules
Programmable platform for data in motion
An open source-data streaming platform with in-line computation capabilities. Apply your custom programs to aggregate, correlate, and transform data records in real-time as they move over the network.
Download
Contribute
The Stack
The Stack
SmartModules
Programmable stream processing to clean, transform, correlate, and derive insights from data in real-time.
Data Streams
Distribured stream processing with redundancy and failover to prevent data loss and minimize downtime.
Immutable Store
Long-lived immutable storage layer that persists data without compromising latency.
Clients & Connectors
Native API support to most popular programming languages & connector catalog (in preview).
The Technology
Powered by Rust
Powered by Rust
Blazing fast and memory-efficient, no runtime or garbage collector.
Async End-to-End
Async End-to-End
Scales to many concurrent clients without blocking threads.
Cloud Native
Cloud Native
Native integration with Kubernetes, to start small and scale on demand.
Universal Client API
Universal Client API
Programming languages are bound natively, for optimal performance.
WebAssembly
WebAssembly
Apply powerful custom operations to real-time data streaming.
Declarative Management
Declarative Management
Declare desired state and the system applies the difference.
Connect Anywhere
Connect Anywhere
Small footprint binary that runs anywhere from IOT device to multi-core servers.
Horizontal Scale
Horizontal Scale
Scalable streaming processing units (SPU) with built-in replicas, partitions and failover.
Low Latency
Low Latency
Single digit millisecond latency at high throughput and consistent variance up to p99.
Hybrid SPU
Hybrid SPU
Deploy your SPUs anywhere: cloud, data center, VMs, desktop, etc.
Immutable Store
Immutable Store
All records are written in the order received and cannot be altered.
Self Service
Self Service
Powerful CLI with built-in cluster management for hands free opertions.
The Platform
Unified Cluster Unified Cluster
A unified cluster for streaming and stateful computation minimizes delay, reduces operational complexity, and boosts security. When streaming and stateful computation are combined, it sets the foundation for a new class of real-time streaming use cases unique to Fluvio. For example:
  • Normalize: remove null values, map invalid fields, filter out records, and more.
  • Protect: strip out personally identifiable information (PII), and encrypt fields.
  • Refine: compute aggregates, derive substreams, and more.
    Extensible Extensible
    Fluvio uses WebAssembly(WASM) to extend real-time stateful computation to a countless number of use cases. Custom modules are loaded dynamically and applied to any number of data streams. For example:
    • Filter patients with high blood sugar in real-time.
    • Count all users that have a declined credit card transactions.
    • Transcode video file when played to a mobile device with insufficient network bandwidth.
      Efficient Efficient
      Fluvio is highly optimized for machine code, and it does not require a virtual machine or garbage collection. It can scale from IOT devices such as Raspberry Pi to multi-core servers.
      • Low latency: single-digit milliseconds response at high throughput and consistent variance.
      • Low footprint: small executable and low memory.
      • Asynchronous architecture: non-blocking calls to reduce latency and scale to a large number of concurrent streams.
        Self Healing Self Healing
        Fluvio operates the cluster automatically with minimum human intervention. The platform implements self-healing by combining a variety of technologies:
        • Declarative Management: a technology pattern where users declare intent and the system provisions resources as they become available.
        • Reconciliation: the cluster continuously checks system components and brings them to a stable state.
        • Replication: all data streams can have multiple copies to reduce the possibility of data loss during outages.
          Developer Friendly Developer Friendly
          Fluvio was built with the development community in mind. It offers a powerful CLI for operational efficiency and native language bindings for most common programming languages. For example:
          • One-Click Deploy: create a cluster locally fluvio cluster start or login to cloud fluvio cloud login with one simple command.
          • Simple CLI: provision streams, apply stateful computations, produce, consume and more.
          • Native APIs: support for Rust, Node.js, Python, and Java with other languages coming soon.