SPU Groups

SPU Groups create a set of SPU replicas that are provisioned by Fluvio Streaming Controller (SC) and managed by Kubernetes. A group can have one or more replicas that are mapped to Kubernetes stateful-sets.

SPU-Groups module defines the following CLI operations:

fluvio spu-group <SUBCOMMAND>

SUBCOMMANDS:
    create    Create managed SPU group
    delete    Delete managed SPU group
    list      List managed SPU groups

Create SPU-Group 

Create SPU-Group operation adds a set of SPU replicas to a Fluvio deployment.

fluvio spu-group create [OPTIONS] --name <string> --replicas <replicas>

OPTIONS:
    -n, --name <string>          Managed SPU group name
    -l, --replicas <integer>     SPU replicas
    -i, --min-id <integer>       Minimum SPU id (default: 1)
    -r, --rack <string>          Rack name
    -s, --size <string>          storage size
    -c, --sc <host:port>         Address of Streaming Controller
    -P, --profile <profile>      Profile name

The options are defined as follows:

  • --name <string>: is the name of the SPU group to be created. See List SPUs to view the SPUs generated by the Create SPU-Group. The names of the SPUs generated by a group are prepended with the group name. SPU-Group is a mandatory option.

  • --replicas <integer>: is the number of SPUs to be generated by the SPU-Group. SPU id is computed by combining replica index with --min-id option documented below. Replica is a required option and must be a number equal or greater than 1.

  • --min-id <integer>: is the id of the first replica in the SPU-Group. SPU id is computed by combining replica index with --min-id option. Min-id is optional and it defaults to 0.

  • --rack <string>: is the rack label inherited by all SPUs generated by the SPU-Group. Racks names have an impact on the replica-assignment when new topics are provisioned. The rack is an optional field.

  • --size <string>: is the amount of storage to be reserved by all SPUs generated by the SPU-Group. The size is an optional field and it defaults to 1Gi.

  • --sc <host:port>: is the public interface of the Streaming Controller. The SC is an optional field used in combination with CLI Profiles to compute a target service.

  • --profile <profile>: is the custom-defined profile file. The profile is an optional field used to compute a target service. For additional information, see Target Service section.

Create SPU-Group Example 

Create an SPU-Group with 2 replicas and starting with SPU id 101:

$ fluvio spu-group create --name my-group --replicas 2 --min-id 101 --sc `SC`:9003
spu-group 'my-group' created successfully

Delete SPU-Group 

Delete SPU-Group operation removes the SPU group and all associated SPUs from a Fluvio deployment.

Delete SPU-Group has the following syntax:

fluvio spu-group delete [OPTIONS] --name <string>

OPTIONS:
    -n, --name <string>        Managed SPU group name
    -c, --sc <host:port>       Address of Streaming Controller
    -P, --profile <profile>    Profile name

The options are defined as follows:

  • --name <string>: is the name of the SPU-Group to be deleted. The name is a mandatory option.

  • --sc <host:port>: See Create SPU-Group

  • --profile <profile>: See Create SPU-Group

Delete SPU-Group Example 

Delete SPU-Group:

$ fluvio spu-group delete --name my-group --sc `SC`:9003
spu-group 'my-group' deleted successfully

Ensure the SPU engines have also been deleted:

$ fluvio spu list --sc `SC`:9003 | grep my-group
<empty>

List SPU-Groups 

List SPU-Groups operation lists all SPU groups in a Fluvio deployment.

fluvio spu-group list [OPTIONS]

OPTIONS:
    -c, --sc <host:port>       Address of Streaming Controller
    -P, --profile <profile>    Profile name
    -O, --output <type>        Output [possible values: table, yaml, json]

The options are defined as follows:

  • --sc <host:port>: See Create SPU-Group

  • --profile <profile>: See Create SPU-Group

  • --output <type>: is the format to be used to display the SPU-Groups. The output is an optional field and it defaults to table format. Alternative formats are: yaml and json.

List SPU-Groups Example 

List all SPU-Groups:

$ fluvio spu-group list --sc `SC`:9003
 NAME      REPLICAS  MIN ID  RACK  SIZE  STATUS 
 group3       3           0         1Gi  Reserved 
 my-group     2         101         1Gi  Reserved 

To list the SPUs generated by SPU-Group “my-group”, pipe the output through grep:

$ fluvio spu list --sc `SC`:9003 | grep my-group
 101  my-group-0  online  managed   -    10.111.137.254:9005  flv-spg-my-group-0.flv-spg-my-group:9006 
 102  my-group-1  online  managed   -    10.102.120.188:9005  flv-spg-my-group-1.flv-spg-my-group:9006 
Related Topics