public abstract class ConfluentMetricsSamplerBase extends java.lang.Object implements MetricSampler
MetricSampler.Samples, MetricSampler.SamplingMode
Modifier and Type | Field and Description |
---|---|
protected org.slf4j.Logger |
LOG |
static java.lang.String |
METRIC_SAMPLER_BOOTSTRAP_SERVERS |
static java.lang.String |
METRIC_SAMPLER_GROUP_ID |
protected org.apache.kafka.clients.consumer.Consumer<byte[],byte[]> |
metricConsumer |
static java.lang.String |
TELEMETRY_REPORTER_TOPIC_PATTERN |
EMPTY_SAMPLES
Constructor and Description |
---|
ConfluentMetricsSamplerBase() |
Modifier and Type | Method and Description |
---|---|
protected static boolean |
checkIfMetricReporterTopicExist(java.lang.String metricReporterTopic,
org.apache.kafka.clients.consumer.Consumer<byte[],byte[]> metricConsumer) |
static void |
checkStartupCondition(KafkaCruiseControlConfig config,
java.util.concurrent.Semaphore abortStartupCheck)
Make sure any condition needed to start this
CruiseControlComponent is satisfied. |
void |
close() |
void |
configure(java.util.Map<java.lang.String,?> configs)
Configure this class with the given key-value pairs
|
protected abstract java.util.List<CruiseControlMetric> |
convertMetricRecord(org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]> record) |
protected static org.apache.kafka.clients.consumer.Consumer<byte[],byte[]> |
createConsumerForMetricTopic(java.util.Properties consumerProps,
java.lang.String metricReporterTopic) |
protected abstract java.lang.String |
defaultMetricSamplerGroupId() |
protected static java.util.Properties |
getMetricConsumerProperties(java.util.Map<java.lang.String,?> configs) |
protected static java.lang.String |
getMetricReporterTopic(java.util.Map<java.lang.String,?> configs) |
MetricSampler.Samples |
getSamples(org.apache.kafka.common.Cluster cluster,
java.util.Set<org.apache.kafka.common.TopicPartition> assignedPartitions,
long startTimeMs,
long endTimeMs,
MetricSampler.SamplingMode mode,
MetricDef metricDef,
long timeout)
Get the metric sample of the given topic partition and replica from the Kafka cluster.
|
protected final org.slf4j.Logger LOG
public static final java.lang.String METRIC_SAMPLER_BOOTSTRAP_SERVERS
public static final java.lang.String TELEMETRY_REPORTER_TOPIC_PATTERN
public static final java.lang.String METRIC_SAMPLER_GROUP_ID
protected org.apache.kafka.clients.consumer.Consumer<byte[],byte[]> metricConsumer
protected static java.lang.String getMetricReporterTopic(java.util.Map<java.lang.String,?> configs)
protected static java.util.Properties getMetricConsumerProperties(java.util.Map<java.lang.String,?> configs)
protected static boolean checkIfMetricReporterTopicExist(java.lang.String metricReporterTopic, org.apache.kafka.clients.consumer.Consumer<byte[],byte[]> metricConsumer)
protected static org.apache.kafka.clients.consumer.Consumer<byte[],byte[]> createConsumerForMetricTopic(java.util.Properties consumerProps, java.lang.String metricReporterTopic)
public MetricSampler.Samples getSamples(org.apache.kafka.common.Cluster cluster, java.util.Set<org.apache.kafka.common.TopicPartition> assignedPartitions, long startTimeMs, long endTimeMs, MetricSampler.SamplingMode mode, MetricDef metricDef, long timeout) throws MetricSamplingException
The samples include PartitionMetricSamples and BrokerMetricSamples.
Due to the lack of direct metrics at partition level, Kafka Cruise Control needs to estimate the CPU utilization for each partition by using the following formula:
BROKER_CPU_UTIL = a * ALL_TOPIC_BYTES_IN_RATE + b * ALL_TOPIC_BYTES_OUT_RATE + c * ALL_FOLLOWER_BYTES_IN_RATE
LEADER_PARTITION_CPU_UTIL = a * LEADER_PARTITION_BYTES_IN + b * LEADER_PARTITION_BYTES_OUT
FOLLOWER_PARTITION_CPU_UTIL = c * LEADER_PARTITION_BYTES_IN
Kafka Cruise Control needs to know the parameters of a, b and c for cost evaluation of leader and partition movement.
getSamples
in interface MetricSampler
cluster
- the metadata of the cluster.assignedPartitions
- the topic partitionstartTimeMs
- the start time of the sampling period.endTimeMs
- the end time of the sampling period.mode
- The sampling mode.metricDef
- the metric definitions.timeout
- The sampling timeout to stop sampling even if there is more data to get.MetricSamplingException
protected abstract java.util.List<CruiseControlMetric> convertMetricRecord(org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]> record)
protected abstract java.lang.String defaultMetricSamplerGroupId()
public void configure(java.util.Map<java.lang.String,?> configs)
CruiseControlConfigurable
configure
in interface CruiseControlConfigurable
public static void checkStartupCondition(KafkaCruiseControlConfig config, java.util.concurrent.Semaphore abortStartupCheck)
CruiseControlComponent
is satisfied.public void close()
close
in interface java.lang.AutoCloseable