public class KafkaPartitionMetricSampleAggregator extends MetricSampleAggregator<java.lang.String,PartitionEntity>
The partition metric sample aggregator performs the sanity check on the samples and aggregate the samples into the corresponding window. we assume the sample we get are only from the leaders, and we are going to derive the follower metrics based on the leader metrics.
MetricSampleAggregator
MetricSampleAggregator.SampleType
_identityEntityMap, _metricDef, _minSamplesPerWindow, _monitoringPeriodMs, _numWindows, _numWindowsToKeep, _sampleType, _windowMs
_generation
Constructor and Description |
---|
KafkaPartitionMetricSampleAggregator(KafkaCruiseControlConfig config,
MetadataClient metadataClient)
Construct the metric sample aggregator.
|
Modifier and Type | Method and Description |
---|---|
boolean |
addSample(PartitionMetricSample sample)
Add a sample to the metric aggregator.
|
boolean |
addSample(PartitionMetricSample sample,
boolean leaderValidation)
Add a sample to the metric aggregator.
|
MetricSampleAggregationResult<java.lang.String,PartitionEntity> |
aggregate(MetadataClient.ClusterAndGeneration clusterAndGeneration,
long from,
long to,
ModelCompletenessRequirements requirements,
OperationProgress operationProgress)
Collect the aggregated metrics for all the topic partitions for a time window.
|
MetricSampleAggregationResult<java.lang.String,PartitionEntity> |
aggregate(MetadataClient.ClusterAndGeneration clusterAndGeneration,
long now,
OperationProgress operationProgress)
Collect the aggregated metrics for all the topic partitions.
|
MetricSampleCompleteness<java.lang.String,PartitionEntity> |
completeness(org.apache.kafka.common.Cluster cluster,
long from,
long to,
ModelCompletenessRequirements requirements)
Get the metric sample completeness for a given period.
|
boolean |
isValidLeader(PartitionMetricSample sample) |
double |
monitoredPercentage(MetadataClient.ClusterAndGeneration clusterAndGeneration)
Get the valid partitions percentage across all the windows.
|
java.util.SortedMap<java.lang.Long,java.lang.Float> |
validPartitionRatioByWindows(MetadataClient.ClusterAndGeneration clusterAndGeneration)
Get the monitored partition percentage in each window.
|
java.util.SortedSet<java.lang.Long> |
validWindows(MetadataClient.ClusterAndGeneration clusterAndGeneration,
double minMonitoredPartitionsPercentage)
Get a sorted set of valid windows in the aggregator.
|
addSample, aggregate, allWindows, availableWindows, clear, completeness, earliestWindow, monitoringPeriodMs, numAvailableWindows, numAvailableWindows, numSamples, peekCurrentWindow, removeEntities, removeEntityGroup, retainEntities, retainEntityGroup
compareGeneration, compareGeneration, generation, setGeneration
public KafkaPartitionMetricSampleAggregator(KafkaCruiseControlConfig config, MetadataClient metadataClient)
config
- The load monitor configurations.metadataClient
- A client to fetch metadata for the cluster.public boolean addSample(PartitionMetricSample sample)
sample
- The metric sample to add.public boolean addSample(PartitionMetricSample sample, boolean leaderValidation)
sample
- The metric sample to add.leaderValidation
- whether perform the leader validation or not.public MetricSampleAggregationResult<java.lang.String,PartitionEntity> aggregate(MetadataClient.ClusterAndGeneration clusterAndGeneration, long now, OperationProgress operationProgress) throws NotEnoughValidWindowsException
If a topic has at least one window that does not have enough samples, that topic will be excluded from the returned aggregated metrics. This is because:
clusterAndGeneration
- The current cluster information.now
- the current time.operationProgress
- to report the async operation progress.MetricSampleAggregationResult
for all the partitions.NotEnoughValidWindowsException
public MetricSampleAggregationResult<java.lang.String,PartitionEntity> aggregate(MetadataClient.ClusterAndGeneration clusterAndGeneration, long from, long to, ModelCompletenessRequirements requirements, OperationProgress operationProgress) throws NotEnoughValidWindowsException
If a topic has at least one window that does not have enough samples, that topic will be excluded from the returned aggregated metrics. This is because:
clusterAndGeneration
- The current cluster information.from
- the start of the time windowto
- the end of the time windowrequirements
- the ModelCompletenessRequirements
for the aggregation result.operationProgress
- to report the operation progress.MetricSampleAggregationResult
for all the partitions.NotEnoughValidWindowsException
public MetricSampleCompleteness<java.lang.String,PartitionEntity> completeness(org.apache.kafka.common.Cluster cluster, long from, long to, ModelCompletenessRequirements requirements)
cluster
- the current cluster topologyfrom
- the start of the periodto
- the end of the periodrequirements
- the model completeness requirements.public java.util.SortedSet<java.lang.Long> validWindows(MetadataClient.ClusterAndGeneration clusterAndGeneration, double minMonitoredPartitionsPercentage)
enough valid partitions
being monitored. A valid partition must be valid in all the windows in the returned set.clusterAndGeneration
- The current cluster and generation.minMonitoredPartitionsPercentage
- the minimum required monitored partitions percentage.public double monitoredPercentage(MetadataClient.ClusterAndGeneration clusterAndGeneration)
clusterAndGeneration
- the current cluster and generation.public java.util.SortedMap<java.lang.Long,java.lang.Float> validPartitionRatioByWindows(MetadataClient.ClusterAndGeneration clusterAndGeneration)
clusterAndGeneration
- the current cluster and generation.public boolean isValidLeader(PartitionMetricSample sample)