Skip to content
Open in Gitpod

HStreamDB Configuration

HStreamDB configuration file is located at path /etc/hstream/config.yaml in the docker image from v0.6.3. or you can download the config file

Configuration Table

hserver

NameDefault ValueDescription
idThe identifier of a single HServer node, the value must be given and can be overwritten by cli option `--server-id
bind-address"0.0.0.0"The IP address or name of the host to which the HServer protocol handler is bound. The value can be overwritten by cli option --bind-address
advertised-address"127.0.0.1"Server listener address value, the value must be given and shouldn't be "0.0.0.0", if you intend to start a cluster or trying to connect to the server from a different network. This value can be overwritten by cli option --address
gossip-addressThe address used for server internal communication, if not specified, it uses the value of advertised-address. The value can be overwritten by cli option "--gossip-address"
port6570Server port value, the value must be given and can be overwritten by cli option --port
internal-port6571Server port value for internal communications between server nodes, the value must be given and can be overwritten by cli option --internal-port
metastore-uriThe server nodes in the same cluster shares an HMeta uniy, this is used for metadata storage and is essential for a server to start. Specify the HMeta protocal such as zk:// or rq://, following with Comma separated host:port pairs, each corresponding to a hmeta server. e.g. zk://127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183. The value must be given and can be overwritten by cli option --metastore-uri
onnector-meta-storeThe metadata store for connectors (hstream io), the value must be given.
log-with-colortrueoptional, The options used to control whether print logs with color by the server node, can be overwritten by cli option --log-with-color
log-levelinfooptional, the setting control lof print level by the server node the default value can be overwritten by cli option --log-level
max-record-size1024*1024 (1MB)The largest size of a record batch allowed by HStreamDB
enable-tlsfalseTLS options: Enable tls, which requires tls-key-path and tls-cert-path options
tls-key-pathTLS options: Key file path for tls, can be generated by openssl
tls-cert-pathThe signed certificate by CA for the key(tls-key-path)
advertise-listenersThe advertised listeners for the server

hstore

The configuration for hstore is optional. When the values are not provided, hstreamdb will use the default values.

NameDefault ValueDescription
log-levelinfooptional

Store admin section specifies the client config when connecting to the storage admin server

NameDefault ValueDescription
host"127.0.0.1"optional
port6440optional
protocol-idbinaryProtocolIdoptional
conn-timeout5000optional
send-timeout5000optional
recv-timeout5000optional

hstream-io

NameDescription
tasks-paththe io tasks work directory
tasks-networkio tasks run as docker containers, so the tasks-network should be the network that can connect to HStreamDB and external systems
source-imageskey-value map specify the images used by the source connectors
sink-imageskey-value map specify the images used by the sink connectors

Resource Attributes

Stream

NameDescription
nameThe name of the stream
shard countThe number of shards in the stream
replication factorThe number of the replicas
backlog retentionThe retention time of the records in the stream in seconds

Subscription

NameDescription
idThe id of the subscription
stream nameThe name of the stream to subscribe
ackTimeoutSecondsMaximum time in the server will wait for an acknowledgement
maxUnackedRecordsThe maximum amount of unacknowledged records allowed

Command-Line Options

For ease of use, we allow users to pass some options to override the configuration in the configuration file when starting the server with hstream-server :

OptionMeta varDescription
config-pathPATHhstream config path
bind-addressHOSTserver host value
advertised-addressHOSTserver listener address value
gossip-addressHOSTserver gossip address value
portINTserver port value
internal-portINTserver channel port value for internal communication
server-idUINT32ID of the hstream server node
store-admin-portINTstore admin port value
metastore-uriSTRSpecify the HMeta protocal such as zk:// or rq://, following with Comma separated host:port pairs, each corresponding to a hmeta server. e.g. zk://127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183.
log-level<critical | fatal | warning | info | debug>Server log level
log-with-colorFLAGServer log with color