An LambdaStack cluster has a number of components which log, collect and retain data. To make sure that these do not exceed the usable storage of the machines they running on, the following configurations are available.
There are two types of retention policies that can be configured at the broker or topic levels: based on time or size. LambdaStack defines the same default value for broker size retention policy as Kafka, -1, which means that no size limit is applied.
To define new log retention values following configuration can be used:
kind: configuration/kafka title: "Kafka" name: default specification: kafka_var: partitions: 8 log_retention_hours: 168 log_retention_bytes: -1
Sets num.partitions parameter
Sets log.retention.hours parameter
Sets log.retention.bytes parameter
Since this limit is enforced at the partition level, multiply it by the number of partitions to compute the topic retention in bytes.
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.