public class KafkaCruiseControl extends Object
Modifier and Type | Class and Description |
---|---|
static class |
KafkaCruiseControl.CcStartupMode |
Modifier and Type | Field and Description |
---|---|
protected KafkaCruiseControlContext |
kafkaCruiseControlContext |
Constructor and Description |
---|
KafkaCruiseControl(Integer brokerId,
KafkaCruiseControlConfig config,
DataBalancerMetricsRegistry metricRegistry,
BlockingSendClient.Builder blockingSendClientBuilder,
KafkaCruiseControl.CcStartupMode startupMode)
Construct the Cruise Control
|
Modifier and Type | Method and Description |
---|---|
OptimizerResult |
addBrokers(MultiBrokerAdditionOperation additionCallback,
BalanceOpExecutionCompletionCallback completionCallback,
String uid)
Add brokers.
|
ExecutorState.State |
executionState() |
boolean |
executorIsReserved() |
AnomalyDetector |
getAnomalyDetector() |
KafkaCruiseControlContext |
getContext() |
OptimizerResult |
getProposals(List<String> goals,
ModelCompletenessRequirements requirements,
OperationProgress operationProgress,
boolean allowCapacityEstimation,
boolean excludeRecentlyDemotedBrokers,
boolean excludeRecentlyRemovedBrokers,
boolean isTriggeredByGoalViolation,
Set<Integer> requestedDestinationBrokerIds,
boolean isRebalanceDiskMode)
Optimize a cluster workload model.
|
boolean |
meetCompletenessRequirements(List<String> goals)
Check if the completeness requirements are met for the given goals.
|
void |
notifyDeadBrokers(Set<Integer> deadBrokers) |
void |
notifyNewBrokers(Set<Integer> newBrokers) |
RebalanceResult |
rebalance(List<String> goals,
boolean dryRun,
ModelCompletenessRequirements requirements,
OperationProgress operationProgress,
boolean allowCapacityEstimation,
String uuid,
boolean excludeRecentlyDemotedBrokers,
boolean excludeRecentlyRemovedBrokers,
boolean isTriggeredByGoalViolation,
Collection<String> goalsToImprove,
Set<Integer> requestedDestinationBrokerIds,
boolean isRebalanceDiskMode)
Rebalance the cluster
|
Set<Integer> |
recentlyDemotedBrokers()
Get the set of broker IDs that have recently been demoted (had all leadership responsibilities removed)
|
Set<Integer> |
recentlyRemovedBrokers()
Get the set of broker IDs that have recently been removed from the cluster via a removeBrokers call.
|
BrokerRemovalFuture |
removeBroker(int broker,
Optional<Long> brokerEpoch,
BalanceOpExecutionCompletionCallback executionCompletionCallback,
BrokerRemovalCallback progressCallback,
String uuid)
A broker removal consists of 4 steps:
1.
|
OptimizerResult |
removeBrokers(Set<Integer> removedBrokers,
List<String> goals,
String uuid,
PlanComputationOptions opts)
Drain brokers #
removedBrokers of all of their partition replicas,
moving them to other brokers in the cluster. |
void |
sanityCheckBrokerPresence(Set<Integer> brokerIds)
Sanity check whether the provided brokers exist in cluster or not.
|
static void |
sanityCheckCapacityEstimation(boolean allowCapacityEstimation,
Map<Integer,String> capacityEstimationInfoByBrokerId)
Check whether the given capacity estimation info indicates estimations for any broker when capacity estimation is
not permitted.
|
void |
setEvenClusterLoadStateManager(boolean selfHealingEnabled) |
void |
setGoalViolationSelfHealing(boolean selfHealingEnabled)
Enable or disable self healing for the GOAL_VIOLATION anomaly type.
|
void |
shutdown() |
void |
startUp(ApiStatePersistenceStore persistenceStore)
Start up the Cruise Control.
|
void |
updateThrottle(long newThrottle)
Update the throttle used for an ongoing execution.
|
void |
userTriggeredStopExecution()
Request the executor to stop any ongoing execution.
|
protected final KafkaCruiseControlContext kafkaCruiseControlContext
public KafkaCruiseControl(Integer brokerId, KafkaCruiseControlConfig config, DataBalancerMetricsRegistry metricRegistry, BlockingSendClient.Builder blockingSendClientBuilder, KafkaCruiseControl.CcStartupMode startupMode)
config
- the configuration of Cruise Control.startupMode
- -- how CruiseControl was startedpublic void startUp(ApiStatePersistenceStore persistenceStore) throws ExecutionException, InterruptedException
public void shutdown()
public AnomalyDetector getAnomalyDetector()
public OptimizerResult removeBrokers(Set<Integer> removedBrokers, List<String> goals, String uuid, PlanComputationOptions opts) throws Exception
removedBrokers
of all of their partition replicas,
moving them to other brokers in the cluster. Drain brokers should not be used
for live brokers, since execution will ignore the broker's static throttleremovedBrokers
- The brokers to decommission.goals
- The goal names (i.e. each matching Goal.name()
) to be met when decommissioning the brokers.
When empty all goals will be used.uuid
- UUID of the execution.opts
- the options to compute the plan withException
- When any exception occurred during the broker removal process.public BrokerRemovalFuture removeBroker(int broker, Optional<Long> brokerEpoch, @Nonnull BalanceOpExecutionCompletionCallback executionCompletionCallback, @Nonnull BrokerRemovalCallback progressCallback, String uuid)
broker
- - the ID of the broker to removebrokerEpoch
- - the epoch of the broker to remove, needed for the shutdown requestprogressCallback
- - a callback utilized for tracking the progress of the remove broker callexecutionCompletionCallback
- -- a callback utilized to notify the operation scheduler when this has
finished.uuid
- - the unique ID of this operationpublic static void sanityCheckCapacityEstimation(boolean allowCapacityEstimation, Map<Integer,String> capacityEstimationInfoByBrokerId)
allowCapacityEstimation
- Allow capacity estimation in cluster model if the requested broker capacity is unavailable.capacityEstimationInfoByBrokerId
- Capacity estimation info by broker id for which there has been an estimation.public RebalanceResult rebalance(List<String> goals, boolean dryRun, ModelCompletenessRequirements requirements, OperationProgress operationProgress, boolean allowCapacityEstimation, String uuid, boolean excludeRecentlyDemotedBrokers, boolean excludeRecentlyRemovedBrokers, boolean isTriggeredByGoalViolation, Collection<String> goalsToImprove, Set<Integer> requestedDestinationBrokerIds, boolean isRebalanceDiskMode) throws KafkaCruiseControlException
goals
- The goal names (i.e. each matching Goal.name()
) to be met during the rebalance.
When empty all goals will be used.dryRun
- Whether it is a dry run or not.requirements
- The cluster model completeness requirements.operationProgress
- The progress of the job to report.allowCapacityEstimation
- Allow capacity estimation in cluster model if the requested broker capacity is unavailable.uuid
- UUID of the execution.excludeRecentlyDemotedBrokers
- Exclude recently demoted brokers from proposal generation for leadership transfer.excludeRecentlyRemovedBrokers
- Exclude recently removed brokers from proposal generation for replica transfer.isTriggeredByGoalViolation
- True if rebalance is triggered by goal violation, false otherwise.goalsToImprove
- -- a set of goals at least some of which must be fixable before executing the rebalance.requestedDestinationBrokerIds
- Explicitly requested destination broker Ids to limit the replica movement to
these brokers (if empty, no explicit filter is enforced -- cannot be null).isRebalanceDiskMode
- Whether rebalance between brokers or disks within the brokers.KafkaCruiseControlException
- When the rebalance encounter errors.public OptimizerResult addBrokers(MultiBrokerAdditionOperation additionCallback, @Nonnull BalanceOpExecutionCompletionCallback completionCallback, String uid) throws Exception
additionCallback
- - the callback for the operationcompletionCallback
- routine to be invoked on completion of the proposal execution. Accepts success/failure and Exception of the result.uid
- Unique ID of the execution.InterruptedException
- if the thread was interrupted.KafkaCruiseControlException
- When any exception other than InterruptedException occurred during the broker addition.Exception
public OptimizerResult getProposals(List<String> goals, ModelCompletenessRequirements requirements, OperationProgress operationProgress, boolean allowCapacityEstimation, boolean excludeRecentlyDemotedBrokers, boolean excludeRecentlyRemovedBrokers, boolean isTriggeredByGoalViolation, Set<Integer> requestedDestinationBrokerIds, boolean isRebalanceDiskMode) throws KafkaCruiseControlException
goals
- A list of goal names (i.e. each matching Goal.name()
) to optimize. When empty all goals will be used.requirements
- The model completeness requirements to enforce when generating the proposals.operationProgress
- The progress of the job to report.allowCapacityEstimation
- Allow capacity estimation in cluster model if the requested broker capacity is unavailable.excludeRecentlyDemotedBrokers
- Exclude recently demoted brokers from proposal generation for leadership transfer.excludeRecentlyRemovedBrokers
- Exclude recently removed brokers from proposal generation for replica transfer.isTriggeredByGoalViolation
- True if proposals is triggered by goal violation, false otherwise.requestedDestinationBrokerIds
- Explicitly requested destination broker Ids to limit the replica movement to
these brokers (if empty, no explicit filter is enforced -- cannot be null).isRebalanceDiskMode
- True to generate proposal to rebalance between disks within the brokers, false otherwise.KafkaCruiseControlException
- If anything goes wrong in optimization proposal calculation.public KafkaCruiseControlContext getContext()
public Set<Integer> recentlyDemotedBrokers()
public Set<Integer> recentlyRemovedBrokers()
public void userTriggeredStopExecution()
public ExecutorState.State executionState()
public boolean executorIsReserved()
public boolean meetCompletenessRequirements(List<String> goals)
goals
- A list of goals to check completeness for.public void sanityCheckBrokerPresence(Set<Integer> brokerIds)
brokerIds
- A set of broker ids.public void updateThrottle(long newThrottle)
newThrottle
- The new value to be used for throttlingpublic void setGoalViolationSelfHealing(boolean selfHealingEnabled)
selfHealingEnabled
- #True
if self healing should be enabled for the GOAL_VIOLATION anomaly type, #false
otherwise.public void setEvenClusterLoadStateManager(boolean selfHealingEnabled)