public class KafkaCruiseControl
extends java.lang.Object
Modifier and Type | Field and Description |
---|---|
protected KafkaCruiseControlConfig |
_config |
Constructor and Description |
---|
KafkaCruiseControl(KafkaCruiseControlConfig config,
scala.Option<org.apache.zookeeper.client.ZKClientConfig> zkClientConfig,
DataBalancerMetricsRegistry metricRegistry,
BlockingSendClient.Builder blockingSendClientBuilder)
Construct the Cruise Control
|
Modifier and Type | Method and Description |
---|---|
OptimizerResult |
addBrokers(MultiBrokerAdditionOperation additionCallback,
BalanceOpExecutionCompletionCallback completionCallback,
java.lang.String uid)
Add brokers.
|
KafkaCruiseControlConfig |
config() |
ExecutorState.State |
executionState() |
boolean |
executorIsReserved() |
OptimizerResult |
getProposals(java.util.List<java.lang.String> goals,
ModelCompletenessRequirements requirements,
OperationProgress operationProgress,
boolean allowCapacityEstimation,
boolean excludeRecentlyDemotedBrokers,
boolean excludeRecentlyRemovedBrokers,
boolean isTriggeredByGoalViolation,
java.util.Set<java.lang.Integer> requestedDestinationBrokerIds,
boolean isRebalanceDiskMode)
Optimize a cluster workload model.
|
boolean |
meetCompletenessRequirements(java.util.List<java.lang.String> goals)
Check if the completeness requirements are met for the given goals.
|
OptimizerResult |
rebalance(java.util.List<java.lang.String> goals,
boolean dryRun,
ModelCompletenessRequirements requirements,
OperationProgress operationProgress,
boolean allowCapacityEstimation,
java.lang.Integer concurrentInterBrokerPartitionMovements,
java.lang.Integer concurrentIntraBrokerPartitionMovements,
java.lang.Integer concurrentLeaderMovements,
ReplicaMovementStrategy replicaMovementStrategy,
java.lang.String uuid,
boolean excludeRecentlyDemotedBrokers,
boolean excludeRecentlyRemovedBrokers,
boolean isTriggeredByGoalViolation,
java.util.Set<java.lang.Integer> requestedDestinationBrokerIds,
boolean isRebalanceDiskMode)
Rebalance the cluster
|
BrokerRemovalFuture |
removeBroker(int broker,
java.util.Optional<java.lang.Long> brokerEpoch,
BalanceOpExecutionCompletionCallback executionCompletionCallback,
BrokerRemovalCallback progressCallback,
java.lang.String uuid)
A broker removal consists of 4 steps:
1.
|
OptimizerResult |
removeBrokers(java.util.Set<java.lang.Integer> removedBrokers,
java.util.List<java.lang.String> goals,
java.lang.String uuid,
PlanComputationOptions opts)
Drain brokers #
removedBrokers of all of their partition replicas,
moving them to other brokers in the cluster. |
void |
sanityCheckBrokerPresence(java.util.Set<java.lang.Integer> brokerIds)
Sanity check whether the provided brokers exist in cluster or not.
|
static void |
sanityCheckCapacityEstimation(boolean allowCapacityEstimation,
java.util.Map<java.lang.Integer,java.lang.String> capacityEstimationInfoByBrokerId)
Check whether the given capacity estimation info indicates estimations for any broker when capacity estimation is
not permitted.
|
void |
setGoalViolationSelfHealing(boolean setSelfHealingEnabled)
Enable or disable self healing for the GOAL_VIOLATION anomaly type.
|
void |
shutdown() |
void |
startUp()
Start up the Cruise Control.
|
CruiseControlState |
state(OperationProgress operationProgress,
java.util.Set<CruiseControlState.SubState> substates)
Get the state with selected substates for Kafka Cruise Control.
|
void |
updateThrottle(long newThrottle)
Update the throttle used for an ongoing execution.
|
void |
userTriggeredStopExecution()
Request the executor to stop any ongoing execution.
|
protected final KafkaCruiseControlConfig _config
public KafkaCruiseControl(KafkaCruiseControlConfig config, scala.Option<org.apache.zookeeper.client.ZKClientConfig> zkClientConfig, DataBalancerMetricsRegistry metricRegistry, BlockingSendClient.Builder blockingSendClientBuilder)
config
- the configuration of Cruise Control.public void startUp() throws java.util.concurrent.ExecutionException, java.lang.InterruptedException
java.util.concurrent.ExecutionException
java.lang.InterruptedException
public void shutdown()
public OptimizerResult removeBrokers(java.util.Set<java.lang.Integer> removedBrokers, java.util.List<java.lang.String> goals, java.lang.String uuid, PlanComputationOptions opts) throws java.lang.Exception
removedBrokers
of all of their partition replicas,
moving them to other brokers in the cluster. Drain brokers should not be used
for live brokers, since execution will ignore the broker's static throttleremovedBrokers
- The brokers to decommission.goals
- The goal names (i.e. each matching Goal.name()
) to be met when decommissioning the brokers.
When empty all goals will be used.uuid
- UUID of the execution.opts
- the options to compute the plan withjava.lang.Exception
- When any exception occurred during the broker removal process.public BrokerRemovalFuture removeBroker(int broker, java.util.Optional<java.lang.Long> brokerEpoch, @Nonnull BalanceOpExecutionCompletionCallback executionCompletionCallback, @Nonnull BrokerRemovalCallback progressCallback, java.lang.String uuid)
broker
- - the ID of the broker to removebrokerEpoch
- - the epoch of the broker to remove, needed for the shutdown requestprogressCallback
- - a callback utilized for tracking the progress of the remove broker callexecutionCompletionCallback
- -- a callback utilized to notify the operation scheduler when this has
finished.uuid
- - the unique ID of this operationpublic static void sanityCheckCapacityEstimation(boolean allowCapacityEstimation, java.util.Map<java.lang.Integer,java.lang.String> capacityEstimationInfoByBrokerId)
allowCapacityEstimation
- Allow capacity estimation in cluster model if the requested broker capacity is unavailable.capacityEstimationInfoByBrokerId
- Capacity estimation info by broker id for which there has been an estimation.public OptimizerResult rebalance(java.util.List<java.lang.String> goals, boolean dryRun, ModelCompletenessRequirements requirements, OperationProgress operationProgress, boolean allowCapacityEstimation, java.lang.Integer concurrentInterBrokerPartitionMovements, java.lang.Integer concurrentIntraBrokerPartitionMovements, java.lang.Integer concurrentLeaderMovements, ReplicaMovementStrategy replicaMovementStrategy, java.lang.String uuid, boolean excludeRecentlyDemotedBrokers, boolean excludeRecentlyRemovedBrokers, boolean isTriggeredByGoalViolation, java.util.Set<java.lang.Integer> requestedDestinationBrokerIds, boolean isRebalanceDiskMode) throws KafkaCruiseControlException
goals
- The goal names (i.e. each matching Goal.name()
) to be met during the rebalance.
When empty all goals will be used.dryRun
- Whether it is a dry run or not.requirements
- The cluster model completeness requirements.operationProgress
- The progress of the job to report.allowCapacityEstimation
- Allow capacity estimation in cluster model if the requested broker capacity is unavailable.concurrentInterBrokerPartitionMovements
- The maximum number of concurrent inter-broker partition movements per broker
(if null, use num.concurrent.partition.movements.per.broker).concurrentIntraBrokerPartitionMovements
- The maximum number of concurrent intra-broker partition movements
(if null, use num.concurrent.intra.broker.partition.movements).concurrentLeaderMovements
- The maximum number of concurrent leader movements
(if null, use num.concurrent.leader.movements).replicaMovementStrategy
- The strategy used to determine the execution order of generated replica movement tasks
(if null, use default.replica.movement.strategies).uuid
- UUID of the execution.excludeRecentlyDemotedBrokers
- Exclude recently demoted brokers from proposal generation for leadership transfer.excludeRecentlyRemovedBrokers
- Exclude recently removed brokers from proposal generation for replica transfer.isTriggeredByGoalViolation
- True if rebalance is triggered by goal violation, false otherwise.requestedDestinationBrokerIds
- Explicitly requested destination broker Ids to limit the replica movement to
these brokers (if empty, no explicit filter is enforced -- cannot be null).isRebalanceDiskMode
- Whether rebalance between brokers or disks within the brokers.KafkaCruiseControlException
- When the rebalance encounter errors.public OptimizerResult addBrokers(MultiBrokerAdditionOperation additionCallback, @Nonnull BalanceOpExecutionCompletionCallback completionCallback, java.lang.String uid) throws java.lang.Exception
additionCallback
- - the callback for the operationcompletionCallback
- routine to be invoked on completion of the proposal execution. Accepts success/failure and Exception of the result.uid
- Unique ID of the execution.java.lang.InterruptedException
- if the thread was interrupted.KafkaCruiseControlException
- When any exception other than InterruptedException occurred during the broker addition.java.lang.Exception
public OptimizerResult getProposals(java.util.List<java.lang.String> goals, ModelCompletenessRequirements requirements, OperationProgress operationProgress, boolean allowCapacityEstimation, boolean excludeRecentlyDemotedBrokers, boolean excludeRecentlyRemovedBrokers, boolean isTriggeredByGoalViolation, java.util.Set<java.lang.Integer> requestedDestinationBrokerIds, boolean isRebalanceDiskMode) throws KafkaCruiseControlException
goals
- A list of goal names (i.e. each matching Goal.name()
) to optimize. When empty all goals will be used.requirements
- The model completeness requirements to enforce when generating the proposals.operationProgress
- The progress of the job to report.allowCapacityEstimation
- Allow capacity estimation in cluster model if the requested broker capacity is unavailable.excludeRecentlyDemotedBrokers
- Exclude recently demoted brokers from proposal generation for leadership transfer.excludeRecentlyRemovedBrokers
- Exclude recently removed brokers from proposal generation for replica transfer.isTriggeredByGoalViolation
- True if proposals is triggered by goal violation, false otherwise.requestedDestinationBrokerIds
- Explicitly requested destination broker Ids to limit the replica movement to
these brokers (if empty, no explicit filter is enforced -- cannot be null).isRebalanceDiskMode
- True to generate proposal to rebalance between disks within the brokers, false otherwise.KafkaCruiseControlException
- If anything goes wrong in optimization proposal calculation.public KafkaCruiseControlConfig config()
public void userTriggeredStopExecution()
public CruiseControlState state(OperationProgress operationProgress, java.util.Set<CruiseControlState.SubState> substates)
public ExecutorState.State executionState()
public boolean executorIsReserved()
public boolean meetCompletenessRequirements(java.util.List<java.lang.String> goals)
goals
- A list of goals to check completeness for.public void sanityCheckBrokerPresence(java.util.Set<java.lang.Integer> brokerIds)
brokerIds
- A set of broker ids.public void updateThrottle(long newThrottle)
newThrottle
- The new value to be used for throttlingjava.lang.Exception
public void setGoalViolationSelfHealing(boolean setSelfHealingEnabled)
setSelfHealingEnabled
- #True
if self healing should be enabled for the GOAL_VIOLATION anomaly type, #false
otherwise.