public class Executor extends Object
The executor class is responsible for talking to the Kafka cluster to execute the rebalance proposals. The executor is thread-safe.
Modifier and Type | Class and Description |
---|---|
static interface |
Executor.ExecutionTaskWaiter |
static class |
Executor.PartitionReplicas |
Constructor and Description |
---|
Executor(KafkaCruiseControlConfig config,
org.apache.kafka.common.utils.Time time,
DataBalancerMetricsRegistry metricRegistry,
MetadataClient metadataClient,
long removalHistoryRetentionTimeMs,
ExecutorNotifier executorNotifier,
AnomalyDetector anomalyDetector,
org.apache.kafka.clients.admin.ConfluentAdmin adminClient,
com.linkedin.kafka.cruisecontrol.executor.ReplicationThrottleHelper throttleHelper)
Package private for unit testing.
|
Modifier and Type | Method and Description |
---|---|
void |
dropRecentlyRemovedBrokers(Set<Integer> brokersToDrop)
Drop the given brokers from the recently removed brokers.
|
Future<?> |
executeProposals(Collection<ExecutionProposal> proposals,
Set<Integer> unthrottledBrokers,
Set<Integer> removedBrokers,
LoadMonitor loadMonitor,
String uuid,
BalanceOpExecutionCompletionCallback completionCallback)
Initialize proposal execution and start execution.
|
boolean |
hasOngoingExecution()
Whether there is an ongoing operation triggered by current Cruise Control deployment.
|
boolean |
hasOngoingPartitionReassignments()
Whether there is any ongoing partition reassignment.
|
void |
init() |
boolean |
isReservedByOther()
Whether the Executor's is reserved (paused) by another thread.
|
static Set<org.apache.kafka.common.TopicPartition> |
partitionsBeingReassigned(SbkAdminUtils adminUtils)
Returns the
TopicPartition s currently being reassigned |
Set<Integer> |
recentlyRemovedBrokers()
Recently removed brokers are the ones for which a removal was started, regardless of how the process was completed.
|
ExecutorReservationHandle |
reserveAndAbortOngoingExecutions(Duration executionAbortTimeout,
boolean shouldInvalidateMetrics,
String reason)
An overloaded variant of
reserveAndAbortOngoingExecutions(Duration, String) that
provides the option to explicitly tell whether metrics should be invalidated or not. |
ExecutorReservationHandle |
reserveAndAbortOngoingExecutions(Duration executionAbortTimeout,
String reason)
Attempts to acquire a reservation on the Executor, blocking other threads from executing proposals
and, if successfully reserved, aborts all of its ongoing executions.
|
void |
shutdown()
Shutdown the executor.
|
void |
startUp() |
ExecutorState |
state()
Check whether the executor is executing a set of proposals.
|
void |
triggerStopExecution(boolean shouldInvalidateMetrics,
String reason)
An overloaded variant of
triggerStopExecution(String) that provides the
possibility to optionally invalidate metrics windows. |
void |
triggerStopExecution(String reason)
Request the executor to stop any ongoing execution.
|
boolean |
updateThrottle(long newThrottle)
Update the throttle rate in ZK (for ongoing executions)
and set a new throttle value, persisted in memory until the process dies
|
public Executor(KafkaCruiseControlConfig config, org.apache.kafka.common.utils.Time time, DataBalancerMetricsRegistry metricRegistry, MetadataClient metadataClient, long removalHistoryRetentionTimeMs, ExecutorNotifier executorNotifier, AnomalyDetector anomalyDetector, org.apache.kafka.clients.admin.ConfluentAdmin adminClient, com.linkedin.kafka.cruisecontrol.executor.ReplicationThrottleHelper throttleHelper)
public void init()
public void startUp() throws ExecutionException, InterruptedException
public ExecutorReservationHandle reserveAndAbortOngoingExecutions(Duration executionAbortTimeout, String reason) throws TimeoutException
reserveAndAbortOngoingExecutions(Duration, boolean, String)
.
Additionally, awaits a certain refresh time if an ongoing execution was aborted to allow metadata
to reset in the underlying Kafka brokers.
This method is blocking. After this method returns, the caller is ensured that the Executor is not executing any proposals.
The callee is expected to check #isReservedByOther()
prior to calling this method.
executionAbortTimeout
- the amount of time to wait until the executino is successfully aborted.reason
- the reason for aborting the current executionAutoCloseableReservationHandle
if the reservation is taken successfullyIllegalStateException
- if the Executor is already reserved by another threadTimeoutException
- if the aborted proposal execution doesn't stop before #executionAbortTimeout
public ExecutorReservationHandle reserveAndAbortOngoingExecutions(Duration executionAbortTimeout, boolean shouldInvalidateMetrics, String reason) throws TimeoutException
reserveAndAbortOngoingExecutions(Duration, String)
that
provides the option to explicitly tell whether metrics should be invalidated or not.executionAbortTimeout
- the amount of time to wait until the executino is successfully aborted.reason
- the reason for aborting the current executionshouldInvalidateMetrics
- for optional metrics invalidationAutoCloseableReservationHandle
if the reservation is taken successfullyIllegalStateException
- if the Executor is already reserved by another threadTimeoutException
- if the aborted proposal execution doesn't stop before #executionAbortTimeout
public void triggerStopExecution(String reason)
triggerStopExecution(boolean, String)
instead.public void triggerStopExecution(boolean shouldInvalidateMetrics, String reason)
triggerStopExecution(String)
that provides the
possibility to optionally invalidate metrics windows.shouldInvalidateMetrics
- whether to invalidate metrics windowspublic void dropRecentlyRemovedBrokers(Set<Integer> brokersToDrop)
brokersToDrop
- Brokers to drop from the latestRemoveStartTimeMsByBrokerId
.public void shutdown()
public boolean hasOngoingExecution()
public boolean isReservedByOther()
public boolean updateThrottle(long newThrottle)
newThrottle
- The new throttle rate, in bytespublic boolean hasOngoingPartitionReassignments()
public Set<Integer> recentlyRemovedBrokers()
removalHistoryRetentionTimeMs
.public ExecutorState state()
public Future<?> executeProposals(Collection<ExecutionProposal> proposals, Set<Integer> unthrottledBrokers, Set<Integer> removedBrokers, LoadMonitor loadMonitor, String uuid, BalanceOpExecutionCompletionCallback completionCallback)
proposals
- Proposals to be executed.unthrottledBrokers
- Brokers that are not throttled in terms of the number of in/out replica movements.removedBrokers
- Removed brokers, null if no brokers has been removed.loadMonitor
- Load monitor.uuid
- UUID of the execution.completionCallback
- -- a Consumer of (success, failure exception) to be invoked when the execution completes. NOT
to be invoked if this fails with an exception.public static Set<org.apache.kafka.common.TopicPartition> partitionsBeingReassigned(SbkAdminUtils adminUtils)
TopicPartition
s currently being reassigned