public abstract class AbstractGoal extends Object implements Goal
Modifier and Type | Field and Description |
---|---|
protected BalancingConstraint |
balancingConstraint |
protected boolean |
finished |
protected double |
minMonitoredPartitionPercentage |
protected int |
numWindows |
protected GoalOptimizationResult.Builder |
optimizationResultBuilder |
protected ProposalStats.Builder |
proposalStatsBuilder |
Constructor and Description |
---|
AbstractGoal()
Constructor of Abstract Goal class sets the _finished flag to false to signal that the goal requirements have not
been satisfied, yet.
|
Modifier and Type | Method and Description |
---|---|
protected abstract SortedSet<Broker> |
brokersToBalance(ClusterModel clusterModel)
Get sorted brokers that the rebalance process will go over to apply balancing actions to replicas they contain.
|
protected void |
changeObservership(ClusterModel clusterModel,
org.apache.kafka.common.TopicPartition tp,
int replicaId) |
void |
configure(Map<String,?> configs)
Configure this class with the given key-value pairs
|
protected abstract void |
initGoalState(ClusterModel clusterModel,
OptimizationOptions optimizationOptions,
Optional<OptimizationMetrics> optimizationMetricsOpt)
Initialize states that this goal requires -- e.g.
|
protected Broker |
maybeApplyBalancingAction(ClusterModel clusterModel,
Replica replica,
Collection<Broker> candidateBrokers,
ActionType action,
Set<Goal> optimizedGoals,
OptimizationOptions optimizationOptions,
DetailedProposal.DetailedReasonBuilder detailedReasonBuilder,
Optional<DetailedProposal.Builder> detailedProposalOptional)
Same with
this#maybeApplyBalancingAction(ClusterModel, Replica, Collection, ActionType, Set, OptimizationOptions, Optional |
protected Broker |
maybeApplyBalancingAction(ClusterModel clusterModel,
Replica replica,
Collection<Broker> candidateBrokers,
ActionType action,
Set<Goal> optimizedGoals,
OptimizationOptions optimizationOptions,
Optional<DetailedProposal.Builder> detailedProposalOptional)
Attempt to apply the given balancing action to the given replica in the given cluster.
|
protected Disk |
maybeMoveReplicaBetweenDisks(ClusterModel clusterModel,
Replica replica,
Collection<Disk> candidateDisks,
Set<Goal> optimizedGoals)
Attempt to move replica between disks of the same broker.
|
abstract String |
name()
Get the name of this goal.
|
GoalOptimizationResult |
optimize(ClusterModel clusterModel,
Set<Goal> optimizedGoals,
OptimizationOptions optimizationOptions,
Optional<OptimizationMetrics> optimizationMetricsOpt)
The entry-point to a goal, this method mutates the given #
ClusterModel object by optimizing it
in accordance to the goal |
protected ProposalObservabilityTrackingOptions |
proposalTrackingOptions()
Top proposal tracking enables goals to track the top
ProposalObservabilityTrackingOptions.maxNumberOfProposals proposals they generated, separated by the ones that were accepted and rejected. |
protected abstract void |
rebalanceForBroker(Broker broker,
ClusterModel clusterModel,
Set<Goal> optimizedGoals,
OptimizationOptions optimizationOptions)
Rebalance the given broker without violating the constraints of the current goal and optimized goals.
|
protected void |
relocateReplica(ClusterModel model,
org.apache.kafka.common.TopicPartition tp,
int sourceBrokerId,
int destinationBrokerId) |
boolean |
selfSatisfied(ClusterModel clusterModel,
BalancingAction action)
Check if requirements of this goal are not violated if this action is applied to the given cluster state,
false otherwise.
|
protected static boolean |
shouldExclude(BalancingAction balancingAction,
List<Broker> sourceBrokers,
Set<String> excludedTopics)
Check whether the replica should be excluded from the rebalance given the excluded topics
excludedTopics . |
protected static boolean |
shouldExclude(Replica replica,
Set<String> excludedTopics)
Check whether the replica should be excluded from the rebalance.
|
protected boolean |
shouldTryLeadershipMovement(Resource resource) |
String |
toString() |
protected ActionAcceptance |
tryAcceptProposal(Set<Goal> optimizedGoals,
BalancingAction proposal,
ClusterModel clusterModel,
Optional<DetailedProposal> detailedProposalOptional) |
protected abstract void |
updateGoalState(ClusterModel clusterModel,
Set<String> excludedTopics)
Update goal state after one round of self-healing / rebalance.
|
protected void |
updateReplicationFactor(ClusterModel clusterModel,
Map<Short,Set<String>> topicsByReplicationFactor,
Set<Integer> brokersExcludedForReplicaMovement) |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
actionAcceptance, canChangeReplicationFactor, clusterModelCompletenessRequirements, clusterModelStatsComparator, finish, isHardGoal, optimize
partitionActionAcceptance, partitionActionSelfSatisfied, replicaActionAcceptance, replicaActionSelfSatisfied
protected boolean finished
protected BalancingConstraint balancingConstraint
protected int numWindows
protected double minMonitoredPartitionPercentage
protected GoalOptimizationResult.Builder optimizationResultBuilder
protected ProposalStats.Builder proposalStatsBuilder
public AbstractGoal()
public void configure(Map<String,?> configs)
CruiseControlConfigurable
configure
in interface CruiseControlConfigurable
public GoalOptimizationResult optimize(ClusterModel clusterModel, Set<Goal> optimizedGoals, OptimizationOptions optimizationOptions, Optional<OptimizationMetrics> optimizationMetricsOpt) throws OptimizationFailureException
ClusterModel
object by optimizing it
in accordance to the goaloptimize
in interface Goal
clusterModel
- The cluster model reflecting the current state of the cluster. It is a result of the
optimization of the previously optimized goals.optimizedGoals
- Goals that have already been optimized. These goals cannot be violated.optimizationOptions
- Options to take into account during optimization -- e.g. excluded topics.optimizationMetricsOpt
- An optional of a metrics object used to track various metrics related to this goal's optimization run.GoalOptimizationResult
denoting whether the goal was met after the optimization and whether any replica movements were made.
Note that for hard goals, the implementation should just throw exceptions if the goal is not met.OptimizationFailureException
- if the goal is a hard goal and it failed to optimize the given cluster modelpublic abstract String name()
Goal
protected static boolean shouldExclude(Replica replica, Set<String> excludedTopics)
replica
- the replica to check.excludedTopics
- the excluded topics set.protected static boolean shouldExclude(BalancingAction balancingAction, List<Broker> sourceBrokers, Set<String> excludedTopics)
excludedTopics
.
A replica should be excluded if both its topic is in the excluded topics and all its original brokers are alive.
If all the brokers are not alive, the topic should be moved off the dead brokers despite being excluded.balancingAction
- the balancing action to check.sourceBrokers
- the set of source brokers that the balancing actions aims to move things away from.excludedTopics
- the excluded topics set to validate against.protected abstract SortedSet<Broker> brokersToBalance(ClusterModel clusterModel)
clusterModel
- The state of the cluster.public final boolean selfSatisfied(ClusterModel clusterModel, BalancingAction action)
clusterModel
- The state of the cluster.action
- Action containing information about potential modification to the given cluster model.protected abstract void initGoalState(ClusterModel clusterModel, OptimizationOptions optimizationOptions, Optional<OptimizationMetrics> optimizationMetricsOpt) throws OptimizationFailureException
clusterModel
- The state of the cluster.optimizationOptions
- Options to take into account during optimization -- e.g. excluded topics.optimizationMetricsOpt
- An optional of OptimizationMetrics
used to capture various metrics throughout this goal's optimization run.OptimizationFailureException
protected abstract void updateGoalState(ClusterModel clusterModel, Set<String> excludedTopics) throws OptimizationFailureException
clusterModel
- The state of the cluster.excludedTopics
- The topics that should be excluded from the optimization action.OptimizationFailureException
protected abstract void rebalanceForBroker(Broker broker, ClusterModel clusterModel, Set<Goal> optimizedGoals, OptimizationOptions optimizationOptions) throws OptimizationFailureException
broker
- Broker to be balanced.clusterModel
- The state of the cluster.optimizedGoals
- Optimized goals.optimizationOptions
- Options to take into account during optimization -- e.g. excluded topics.OptimizationFailureException
protected Broker maybeApplyBalancingAction(ClusterModel clusterModel, Replica replica, Collection<Broker> candidateBrokers, ActionType action, Set<Goal> optimizedGoals, OptimizationOptions optimizationOptions, Optional<DetailedProposal.Builder> detailedProposalOptional)
clusterModel
- The state of the cluster.replica
- Replica to be applied the given balancing action.candidateBrokers
- Candidate brokers as the potential destination brokers for replica movement or the location
of followers for leadership transfer. This list can be empty.action
- Balancing action.optimizedGoals
- Optimized goals.optimizationOptions
- Options to take into account during optimization -- e.g. excluded brokers for leadership.detailedProposalOptional
- An optional containing a DetailedProposal.Builder
to build a DetailedProposal
for observability purposes.protected Broker maybeApplyBalancingAction(ClusterModel clusterModel, Replica replica, Collection<Broker> candidateBrokers, ActionType action, Set<Goal> optimizedGoals, OptimizationOptions optimizationOptions, DetailedProposal.DetailedReasonBuilder detailedReasonBuilder, Optional<DetailedProposal.Builder> detailedProposalOptional)
this#maybeApplyBalancingAction(ClusterModel, Replica, Collection, ActionType, Set, OptimizationOptions, Optional
with the additional functionality of providing a string as the reason of movement. For example, incremental CPU
balancing goal might provide a reason as "Decrease the source broker's CPU usage by X".
clusterModel
- The state of the cluster.replica
- Replica to be applied the given balancing action.candidateBrokers
- Candidate brokers as the potential destination brokers for replica movement or the
location of followers for leadership transfer. This list can be empty.action
- Balancing action.optimizedGoals
- Optimized goals.optimizationOptions
- Options to take into account during optimization -- e.g. excluded brokers for
leadership.detailedReasonBuilder
- - a function used to build the reason for the proposal. e.g "Moving 100MB of disk from broker 1 to broker 2".
Used for observability purposes to show why such moves are proposed.detailedProposalOptional
- An optional containing a DetailedProposal.Builder
to build a DetailedProposal
for observability purposes.protected ActionAcceptance tryAcceptProposal(Set<Goal> optimizedGoals, BalancingAction proposal, ClusterModel clusterModel, Optional<DetailedProposal> detailedProposalOptional)
protected void relocateReplica(ClusterModel model, org.apache.kafka.common.TopicPartition tp, int sourceBrokerId, int destinationBrokerId)
protected void changeObservership(ClusterModel clusterModel, org.apache.kafka.common.TopicPartition tp, int replicaId)
protected void updateReplicationFactor(ClusterModel clusterModel, Map<Short,Set<String>> topicsByReplicationFactor, Set<Integer> brokersExcludedForReplicaMovement) throws OptimizationFailureException
OptimizationFailureException
protected Disk maybeMoveReplicaBetweenDisks(ClusterModel clusterModel, Replica replica, Collection<Disk> candidateDisks, Set<Goal> optimizedGoals)
clusterModel
- The state of the cluster.replica
- Replica to be moved.candidateDisks
- Candidate disks as the potential destination for replica movement.optimizedGoals
- Optimized goals.protected boolean shouldTryLeadershipMovement(Resource resource)
resource
- - the resource this goal is balancing onResource
s.protected ProposalObservabilityTrackingOptions proposalTrackingOptions()
ProposalObservabilityTrackingOptions.maxNumberOfProposals
proposals they generated, separated by the ones that were accepted and rejected.
A top proposal is defined as one that moves the most amount of resource load.
It is up to each implementation to potentially enable this functionality.ProposalObservabilityTrackingOptions
.