HStreamDB Configuration
HStreamDB configuration file is located at path /etc/hstream/config.yaml in the docker image from v0.6.3. or you can download the config file
Configuration Table
hserver
| Name | Default Value | Description |
|---|---|---|
| id | The identifier of a single HServer node, the value must be given and can be overwritten by cli option `--server-id | |
| bind-address | "0.0.0.0" | The IP address or name of the host to which the HServer protocol handler is bound. The value can be overwritten by cli option --bind-address |
| advertised-address | "127.0.0.1" | Server listener address value, the value must be given and shouldn't be "0.0.0.0", if you intend to start a cluster or trying to connect to the server from a different network. This value can be overwritten by cli option --address |
| gossip-address | The address used for server internal communication, if not specified, it uses the value of advertised-address. The value can be overwritten by cli option "--gossip-address" | |
| port | 6570 | Server port value, the value must be given and can be overwritten by cli option --port |
| internal-port | 6571 | Server port value for internal communications between server nodes, the value must be given and can be overwritten by cli option --internal-port |
| metastore-uri | The server nodes in the same cluster shares an HMeta uniy, this is used for metadata storage and is essential for a server to start. Specify the HMeta protocal such as zk:// or rq://, following with Comma separated host:port pairs, each corresponding to a hmeta server. e.g. zk://127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183. The value must be given and can be overwritten by cli option --metastore-uri | |
| onnector-meta-store | The metadata store for connectors (hstream io), the value must be given. | |
| log-with-color | true | optional, The options used to control whether print logs with color by the server node, can be overwritten by cli option --log-with-color |
| log-level | info | optional, the setting control lof print level by the server node the default value can be overwritten by cli option --log-level |
| max-record-size | 1024*1024 (1MB) | The largest size of a record batch allowed by HStreamDB |
| enable-tls | false | TLS options: Enable tls, which requires tls-key-path and tls-cert-path options |
| tls-key-path | TLS options: Key file path for tls, can be generated by openssl | |
| tls-cert-path | The signed certificate by CA for the key(tls-key-path) | |
| advertise-listeners | The advertised listeners for the server |
hstore
The configuration for hstore is optional. When the values are not provided, hstreamdb will use the default values.
| Name | Default Value | Description |
|---|---|---|
| log-level | info | optional |
Store admin section specifies the client config when connecting to the storage admin server
| Name | Default Value | Description |
|---|---|---|
| host | "127.0.0.1" | optional |
| port | 6440 | optional |
| protocol-id | binaryProtocolId | optional |
| conn-timeout | 5000 | optional |
| send-timeout | 5000 | optional |
| recv-timeout | 5000 | optional |
hstream-io
| Name | Description |
|---|---|
| tasks-path | the io tasks work directory |
| tasks-network | io tasks run as docker containers, so the tasks-network should be the network that can connect to HStreamDB and external systems |
| source-images | key-value map specify the images used by the source connectors |
| sink-images | key-value map specify the images used by the sink connectors |
Resource Attributes
Stream
| Name | Description |
|---|---|
| name | The name of the stream |
| shard count | The number of shards in the stream |
| replication factor | The number of the replicas |
| backlog retention | The retention time of the records in the stream in seconds |
Subscription
| Name | Description |
|---|---|
| id | The id of the subscription |
| stream name | The name of the stream to subscribe |
| ackTimeoutSeconds | Maximum time in the server will wait for an acknowledgement |
| maxUnackedRecords | The maximum amount of unacknowledged records allowed |
Command-Line Options
For ease of use, we allow users to pass some options to override the configuration in the configuration file when starting the server with hstream-server :
| Option | Meta var | Description |
|---|---|---|
| config-path | PATH | hstream config path |
| bind-address | HOST | server host value |
| advertised-address | HOST | server listener address value |
| gossip-address | HOST | server gossip address value |
| port | INT | server port value |
| internal-port | INT | server channel port value for internal communication |
| server-id | UINT32 | ID of the hstream server node |
| store-admin-port | INT | store admin port value |
| metastore-uri | STR | Specify the HMeta protocal such as zk:// or rq://, following with Comma separated host:port pairs, each corresponding to a hmeta server. e.g. zk://127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183. |
| log-level | <critical | fatal | warning | info | debug> | Server log level |
| log-with-color | FLAG | Server log with color |