public final class PushManagerImpl extends Object implements PushManager
Constructor and Description |
---|
PushManagerImpl(ReplicationConfig config,
org.apache.kafka.common.utils.Time time,
PushReplicationManagerMetrics pushReplicationManagerMetrics,
Function<Integer,org.apache.kafka.clients.KafkaClient> networkClientResolver) |
Modifier and Type | Method and Description |
---|---|
Pusher |
getPusher(org.apache.kafka.server.common.TopicIdPartition topicIdPartition,
int destinationBrokerId) |
boolean |
isActive() |
boolean |
isPushReplicationSupported(boolean isInternalTopic,
boolean isClusterLinkDestination) |
void |
onHighWatermarkUpdate(org.apache.kafka.server.common.TopicIdPartition topicIdPartition,
Set<Integer> replicaIds,
long updatedHighWatermark)
Invoked every time the high watermark of the given partition is updated.
|
void |
onLeaderAppend(org.apache.kafka.server.common.TopicIdPartition topicIdPartition,
Set<Integer> replicaIds,
long appendOffset,
org.apache.kafka.common.record.AbstractRecords records)
Invoked every time this broker as the leader of the given partition appends memory
records to its log (if the corresponding partition replica has transitioned to push
replication), as well as the first time after starting a push session with the
FileRecords needed to have the follower catch up to the leader. |
void |
onLogStartOffsetUpdate(org.apache.kafka.server.common.TopicIdPartition topicIdPartition,
Set<Integer> replicaIds,
long updatedLogStartOffset)
Invoked every time the log start offset of the given partition is updated.
|
void |
recordFollowerNotCaughtUp(org.apache.kafka.server.common.TopicIdPartition topicIdPartition,
Integer replicaId) |
boolean |
shutdown()
Shuts down the manager, cleaning up any used resources (buffered records, initialized
pusher threads, etc.).
|
void |
startPush(org.apache.kafka.server.common.TopicIdPartition topicIdPartition,
PushSession pushSession)
Initiates push replication for the given partition replica with the given replication
session identifiers (leader epoch + replica epoch + replication session ID).
|
boolean |
startup()
Starts up the push manager, including the pusher threads.
|
void |
stopPush(org.apache.kafka.server.common.TopicIdPartition topicIdPartition,
Set<Integer> replicaIds,
PushSessionEndReason pushSessionEndReason)
Stops push replication for the given partition replicas.
|
public PushManagerImpl(ReplicationConfig config, org.apache.kafka.common.utils.Time time, PushReplicationManagerMetrics pushReplicationManagerMetrics, Function<Integer,org.apache.kafka.clients.KafkaClient> networkClientResolver)
public void onLeaderAppend(org.apache.kafka.server.common.TopicIdPartition topicIdPartition, Set<Integer> replicaIds, long appendOffset, org.apache.kafka.common.record.AbstractRecords records)
PushManager
Invoked every time this broker as the leader of the given partition appends memory
records to its log (if the corresponding partition replica has transitioned to push
replication), as well as the first time after starting a push session with the
FileRecords
needed to have the follower catch up to the leader.
Subsequent calls to this for the same partition are expected to happen-before one another so that the append order within the partition is preserved. For partition replicas for which there is no active push replication session, this will be a no-op.
Throws IllegalStateException
if the manager has already been stopped.
onLeaderAppend
in interface PushManager
public void onHighWatermarkUpdate(org.apache.kafka.server.common.TopicIdPartition topicIdPartition, Set<Integer> replicaIds, long updatedHighWatermark)
PushManager
Invoked every time the high watermark of the given partition is updated. Similarly to
PushManager.onLeaderAppend(org.apache.kafka.server.common.TopicIdPartition, java.util.Set<java.lang.Integer>, long, org.apache.kafka.common.record.AbstractRecords)
, (1) subsequent calls to this for the same partition are expected to
happen-before one another so that the HWM update order within the partition is
preserved; and (2) for partition replicas for which there is no active push replication
session, this will be a no-op.
Throws IllegalStateException
if the manager has already been stopped.
onHighWatermarkUpdate
in interface PushManager
public void onLogStartOffsetUpdate(org.apache.kafka.server.common.TopicIdPartition topicIdPartition, Set<Integer> replicaIds, long updatedLogStartOffset)
PushManager
Invoked every time the log start offset of the given partition is updated. Similarly to
PushManager.onLeaderAppend(org.apache.kafka.server.common.TopicIdPartition, java.util.Set<java.lang.Integer>, long, org.apache.kafka.common.record.AbstractRecords)
, (1) subsequent calls to this for the same partition are expected to
happen-before one another so that the LSO update order within the partition is
preserved; and (2) for partition replicas for which there is no active push replication
session, this will be a no-op.
Throws IllegalStateException
if the manager has already been stopped.
onLogStartOffsetUpdate
in interface PushManager
public void startPush(org.apache.kafka.server.common.TopicIdPartition topicIdPartition, PushSession pushSession)
PushManager
Initiates push replication for the given partition replica with the given replication
session identifiers (leader epoch + replica epoch + replication session ID). Should be called
only after a successful partition replica state change, which could happen only if the
transition condition for the given replication session has been satisfied. Calling this
should happen-before calling PushManager.stopPush(org.apache.kafka.server.common.TopicIdPartition, java.util.Set<java.lang.Integer>, io.confluent.kafka.replication.push.PushSessionEndReason)
for the same replication session. It
should also happen-before any subsequent partition updates (appended records,
updated high watermark or log start offset) called for the same replication session.
Calling this would register the PushSession
for the partition replica,
initiate sending an AppendRecords request with the FileRecords
from the transition
condition, and if that request is successful, subsequent AppendRecords requests with the
buffered records for that partition replica.
Throws IllegalStateException
if the manager has already been stopped.
startPush
in interface PushManager
public void stopPush(org.apache.kafka.server.common.TopicIdPartition topicIdPartition, Set<Integer> replicaIds, PushSessionEndReason pushSessionEndReason)
PushManager
Stops push replication for the given partition replicas. Should be called only after a
successful state change for each of the partition replicas. Calling PushManager.startPush(org.apache.kafka.server.common.TopicIdPartition, io.confluent.kafka.replication.push.PushSession)
for
the same partition and replication session ID should happen-before calling this.
Some of the conditions that would necessitate downgrade to pull replication are:
Calling this would deregister PushSession
s for the partition replicas,
evict any buffered records for them, and if sendEndSessionRequest
is set will
send an end replication session signal to the corresponding replicas in subsequent
AppendRecords requests.
Throws IllegalStateException
if the manager has already been stopped.
stopPush
in interface PushManager
topicIdPartition
- The partition for which some push replication sessions are being stopped.replicaIds
- The broker IDs of the replicas for the stopped sessions.pushSessionEndReason
- Indicates the reason to end a push session and whether the
replica should be explicitly notified in a subsequent
AppendRecords request.public boolean isPushReplicationSupported(boolean isInternalTopic, boolean isClusterLinkDestination)
isPushReplicationSupported
in interface PushManager
isInternalTopic
- Whether the topic is an internal topicisClusterLinkDestination
- Whether the topic is a cluster link destination topicpublic void recordFollowerNotCaughtUp(org.apache.kafka.server.common.TopicIdPartition topicIdPartition, Integer replicaId)
recordFollowerNotCaughtUp
in interface PushManager
public boolean startup()
PushManager
startup
in interface PushManager
true
if the manager has been started from the invocation.public boolean shutdown()
PushManager
shutdown
in interface PushManager
true
if the manager has successfully been "stopped" from this invocation or
false
if the push manager is not activepublic boolean isActive()
isActive
in interface PushManager
public Pusher getPusher(org.apache.kafka.server.common.TopicIdPartition topicIdPartition, int destinationBrokerId)