public class ExecutionTaskManager extends Object
Constructor and Description |
---|
ExecutionTaskManager(int defaultInterBrokerPartitionMovementConcurrency,
int defaultIntraBrokerPartitionMovementConcurrency,
int defaultLeadershipMovementConcurrency,
List<String> replicaMovementStrategies,
org.apache.kafka.clients.admin.ConfluentAdmin adminClient,
DataBalancerMetricsRegistry metricRegistry,
org.apache.kafka.common.utils.Time time,
KafkaCruiseControlConfig config)
The constructor of The Execution task manager.
|
Modifier and Type | Method and Description |
---|---|
void |
addExecutionProposals(Collection<ExecutionProposal> proposals,
Collection<Integer> brokersToSkipConcurrencyCheck,
org.apache.kafka.common.Cluster cluster)
Add a collection of execution proposals for execution.
|
void |
clear() |
List<ExecutionTask> |
drainInterBrokerReplicaMovementTasks()
Returns a list of execution tasks that move the replicas cross brokers.
|
long |
finishedInterBrokerDataMovementInMB() |
ExecutionTaskTracker.ExecutionTasksSummary |
getExecutionTasksSummary(Set<ExecutionTask.TaskType> taskTypesToGetFullList) |
List<ExecutionTask> |
getIntraBrokerReplicaMovementTasks()
Returns a list of execution tasks that move the replicas cross disks of the same broker.
|
List<ExecutionTask> |
getLeadershipMovementTasks()
Returns a list of execution tasks that move the leadership.
|
long |
inExecutionInterBrokerDataToMoveInMB() |
Set<ExecutionTask> |
inExecutionTasks() |
Set<ExecutionTask> |
inExecutionTasks(Collection<ExecutionTask.TaskType> types) |
int |
interBrokerPartitionMovementConcurrency() |
int |
intraBrokerPartitionMovementConcurrency() |
int |
leadershipMovementConcurrency() |
void |
markTaskAborting(ExecutionTask task)
Mark an in-progress task as aborting (1) if an error is encountered and (2) the rollback is possible.
|
void |
markTaskDead(ExecutionTask task)
Mark an in-progress task as aborting (1) if an error is encountered and (2) the rollback is not possible.
|
void |
markTaskDone(ExecutionTask task)
Mark the successful completion of a given task.
|
void |
markTasksAborting(List<ExecutionTask> tasks)
Mark the given tasks as aborting.
|
void |
markTasksInProgress(List<ExecutionTask> tasks)
Mark the given tasks as in progress.
|
void |
markTasksToBeRetried(List<ExecutionTask> tasks)
Mark the given tasks as to be retried.
|
int |
numFinishedInterBrokerPartitionMovements() |
int |
numFinishedIntraBrokerPartitionMovements() |
int |
numFinishedLeadershipMovements() |
int |
numInterBrokerPartitionMovementsToBeRetried() |
int |
numPendingInterBrokerPartitionMovements()
Returns the number of execution tasks that are pending to be executed
|
int |
numPendingIntraBrokerPartitionMovements() |
int |
numPendingLeadershipMovements() |
void |
reloadInterBrokerTasksToBeRetried(org.apache.kafka.common.Cluster cluster)
Reloads the inter-broker tasks that are queued up for retries
|
Map<Integer,Long> |
remainingInterBrokerDataToMoveByDestinationBroker() |
long |
remainingInterBrokerDataToMoveInMB() |
void |
setStopRequested() |
Set<String> |
throttledTopics()
Returns the topics that have had a replication throttle applied
|
public ExecutionTaskManager(int defaultInterBrokerPartitionMovementConcurrency, int defaultIntraBrokerPartitionMovementConcurrency, int defaultLeadershipMovementConcurrency, List<String> replicaMovementStrategies, org.apache.kafka.clients.admin.ConfluentAdmin adminClient, DataBalancerMetricsRegistry metricRegistry, org.apache.kafka.common.utils.Time time, KafkaCruiseControlConfig config)
defaultInterBrokerPartitionMovementConcurrency
- The maximum number of concurrent inter-broker partition movements per broker.
It can be overwritten by user parameter upon post request.defaultIntraBrokerPartitionMovementConcurrency
- The maximum number of concurrent intra-broker partition movements.
It can be overwritten by user parameter upon post request.defaultLeadershipMovementConcurrency
- The maximum number of concurrent leadership movements per batch. It can
be overwritten by user parameter upon post request.replicaMovementStrategies
- The strategies used to determine the execution order of inter-broker replica movement tasks.adminClient
- The adminClient use to query logdir information of replicas.metricRegistry
- The metric registry.time
- The time object to get the time.config
- config object that holds all Kafka Cruise control related configspublic int interBrokerPartitionMovementConcurrency()
public int intraBrokerPartitionMovementConcurrency()
public int leadershipMovementConcurrency()
public List<ExecutionTask> drainInterBrokerReplicaMovementTasks()
ExecutionTaskManager
- subsequent calls to this method will not return the same tasks.
The caller is expected to mark them in progress (markTasksInProgress(List)
), execute them and manage their lifecycle.public List<ExecutionTask> getIntraBrokerReplicaMovementTasks()
public List<ExecutionTask> getLeadershipMovementTasks()
public Set<String> throttledTopics()
public void addExecutionProposals(Collection<ExecutionProposal> proposals, Collection<Integer> brokersToSkipConcurrencyCheck, org.apache.kafka.common.Cluster cluster)
proposals
- the execution proposals to execute.brokersToSkipConcurrencyCheck
- the brokers that does not need to be throttled when move the partitions.cluster
- Cluster state.public void reloadInterBrokerTasksToBeRetried(org.apache.kafka.common.Cluster cluster)
public void markTasksInProgress(List<ExecutionTask> tasks)
public void markTasksToBeRetried(List<ExecutionTask> tasks)
public void markTasksAborting(List<ExecutionTask> tasks)
public void markTaskDone(ExecutionTask task)
public void markTaskAborting(ExecutionTask task)
public void markTaskDead(ExecutionTask task)
public int numInterBrokerPartitionMovementsToBeRetried()
public int numPendingInterBrokerPartitionMovements()
public long remainingInterBrokerDataToMoveInMB()
public Map<Integer,Long> remainingInterBrokerDataToMoveByDestinationBroker()
public int numFinishedInterBrokerPartitionMovements()
public long finishedInterBrokerDataMovementInMB()
public Set<ExecutionTask> inExecutionTasks()
public Set<ExecutionTask> inExecutionTasks(Collection<ExecutionTask.TaskType> types)
public long inExecutionInterBrokerDataToMoveInMB()
public int numPendingLeadershipMovements()
public int numFinishedLeadershipMovements()
public int numPendingIntraBrokerPartitionMovements()
public int numFinishedIntraBrokerPartitionMovements()
public void clear()
public void setStopRequested()
public ExecutionTaskTracker.ExecutionTasksSummary getExecutionTasksSummary(Set<ExecutionTask.TaskType> taskTypesToGetFullList)