public class ExecutionTaskManager extends Object
Constructor and Description |
---|
ExecutionTaskManager(int defaultInterBrokerPartitionMovementConcurrency,
int defaultIntraBrokerPartitionMovementConcurrency,
int defaultLeadershipMovementConcurrency,
List<String> replicaMovementStrategies,
org.apache.kafka.clients.admin.ConfluentAdmin adminClient,
DataBalancerMetricsRegistry metricRegistry,
org.apache.kafka.common.utils.Time time,
KafkaCruiseControlConfig config)
The constructor of The Execution task manager.
|
Modifier and Type | Method and Description |
---|---|
void |
addExecutionProposals(Collection<ExecutionProposal> proposals,
Collection<Integer> brokersToSkipConcurrencyCheck,
org.apache.kafka.common.Cluster cluster)
Add a collection of execution proposals for execution.
|
void |
clear() |
List<ExecutionTask> |
drainInterBrokerReplicaMovementTasks()
Returns a list of execution tasks that move the replicas cross brokers.
|
long |
finishedInterBrokerDataMovementInMB() |
long |
finishedIntraBrokerDataToMoveInMB() |
ExecutionTaskTracker.ExecutionTasksSummary |
getExecutionTasksSummary(Set<ExecutionTask.TaskType> taskTypesToGetFullList) |
List<ExecutionTask> |
getIntraBrokerReplicaMovementTasks()
Returns a list of execution tasks that move the replicas cross disks of the same broker.
|
List<ExecutionTask> |
getLeadershipMovementTasks()
Returns a list of execution tasks that move the leadership.
|
long |
inExecutionInterBrokerDataToMoveInMB() |
long |
inExecutionIntraBrokerDataMovementInMB() |
Set<ExecutionTask> |
inExecutionTasks() |
Set<ExecutionTask> |
inExecutionTasks(Collection<ExecutionTask.TaskType> types) |
int |
interBrokerPartitionMovementConcurrency() |
int |
intraBrokerPartitionMovementConcurrency() |
int |
leadershipMovementConcurrency() |
void |
markTaskAborting(ExecutionTask task)
Mark an in-progress task as aborting (1) if an error is encountered and (2) the rollback is possible.
|
void |
markTaskDead(ExecutionTask task)
Mark an in-progress task as aborting (1) if an error is encountered and (2) the rollback is not possible.
|
void |
markTaskDone(ExecutionTask task)
Mark the successful completion of a given task.
|
void |
markTasksAborting(List<ExecutionTask> tasks)
Mark the given tasks as aborting.
|
void |
markTasksInProgress(List<ExecutionTask> tasks)
Mark the given tasks as in progress.
|
void |
markTasksToBeRetried(List<ExecutionTask> tasks)
Mark the given tasks as to be retried.
|
int |
numFinishedInterBrokerPartitionMovements() |
int |
numFinishedIntraBrokerPartitionMovements() |
int |
numFinishedLeadershipMovements() |
int |
numInterBrokerPartitionMovementsToBeRetried() |
int |
numPendingInterBrokerPartitionMovements()
Returns the number of execution tasks that are pending to be executed
|
int |
numPendingIntraBrokerPartitionMovements() |
int |
numPendingLeadershipMovements() |
void |
reloadInterBrokerTasksToBeRetried(org.apache.kafka.common.Cluster cluster)
Reloads the inter-broker tasks that are queued up for retries
|
long |
remainingInterBrokerDataToMoveInMB() |
long |
remainingIntraBrokerDataToMoveInMB() |
void |
setStopRequested() |
Set<String> |
throttledTopics()
Returns the topics that have had a replication throttle applied
|
public ExecutionTaskManager(int defaultInterBrokerPartitionMovementConcurrency, int defaultIntraBrokerPartitionMovementConcurrency, int defaultLeadershipMovementConcurrency, List<String> replicaMovementStrategies, org.apache.kafka.clients.admin.ConfluentAdmin adminClient, DataBalancerMetricsRegistry metricRegistry, org.apache.kafka.common.utils.Time time, KafkaCruiseControlConfig config)
defaultInterBrokerPartitionMovementConcurrency
- The maximum number of concurrent inter-broker partition movements per broker.
It can be overwritten by user parameter upon post request.defaultIntraBrokerPartitionMovementConcurrency
- The maximum number of concurrent intra-broker partition movements.
It can be overwritten by user parameter upon post request.defaultLeadershipMovementConcurrency
- The maximum number of concurrent leadership movements per batch. It can
be overwritten by user parameter upon post request.replicaMovementStrategies
- The strategies used to determine the execution order of inter-broker replica movement tasks.adminClient
- The adminClient use to query logdir information of replicas.metricRegistry
- The metric registry.time
- The time object to get the time.config
- config object that holds all Kafka Cruise control related configspublic int interBrokerPartitionMovementConcurrency()
public int intraBrokerPartitionMovementConcurrency()
public int leadershipMovementConcurrency()
public List<ExecutionTask> drainInterBrokerReplicaMovementTasks()
ExecutionTaskManager
- subsequent calls to this method will not return the same tasks.
The caller is expected to mark them in progress (markTasksInProgress(List)
), execute them and manage their lifecycle.public List<ExecutionTask> getIntraBrokerReplicaMovementTasks()
public List<ExecutionTask> getLeadershipMovementTasks()
public Set<String> throttledTopics()
public void addExecutionProposals(Collection<ExecutionProposal> proposals, Collection<Integer> brokersToSkipConcurrencyCheck, org.apache.kafka.common.Cluster cluster)
proposals
- the execution proposals to execute.brokersToSkipConcurrencyCheck
- the brokers that does not need to be throttled when move the partitions.cluster
- Cluster state.public void reloadInterBrokerTasksToBeRetried(org.apache.kafka.common.Cluster cluster)
public void markTasksInProgress(List<ExecutionTask> tasks)
public void markTasksToBeRetried(List<ExecutionTask> tasks)
public void markTasksAborting(List<ExecutionTask> tasks)
public void markTaskDone(ExecutionTask task)
public void markTaskAborting(ExecutionTask task)
public void markTaskDead(ExecutionTask task)
public int numInterBrokerPartitionMovementsToBeRetried()
public int numPendingInterBrokerPartitionMovements()
public long remainingInterBrokerDataToMoveInMB()
public int numFinishedInterBrokerPartitionMovements()
public long finishedInterBrokerDataMovementInMB()
public Set<ExecutionTask> inExecutionTasks()
public Set<ExecutionTask> inExecutionTasks(Collection<ExecutionTask.TaskType> types)
public long inExecutionInterBrokerDataToMoveInMB()
public int numPendingLeadershipMovements()
public int numFinishedLeadershipMovements()
public int numPendingIntraBrokerPartitionMovements()
public long remainingIntraBrokerDataToMoveInMB()
public int numFinishedIntraBrokerPartitionMovements()
public long finishedIntraBrokerDataToMoveInMB()
public long inExecutionIntraBrokerDataMovementInMB()
public void clear()
public void setStopRequested()
public ExecutionTaskTracker.ExecutionTasksSummary getExecutionTasksSummary(Set<ExecutionTask.TaskType> taskTypesToGetFullList)