Interface Pusher

public interface Pusher

Used for proactively replicating (pushing) records for partitions led by this broker to its follower brokers. Records for partitions handled by the pusher are buffered within it and are regularly sent to followers.

Method Summary

Modifier and Type	Method	Description
int	id()	An identifier of this pusher thread expected to be between 0 and N-1 where N is the total number of pusher threads on this broker.
void	onHighWatermarkUpdate(org.apache.kafka.server.common.TopicIdPartition partition, int destinationBrokerId, long highWatermark)	Should be called whenever the high watermark for a push-replicated partition replica is updated.
void	onLeaderAppend(org.apache.kafka.server.common.TopicIdPartition partition, int destinationBrokerId, long appendOffset, org.apache.kafka.common.record.AbstractRecords records)	Buffers the given records to be replicated with some subsequent AppendRecords request.
void	onLogStartOffsetUpdate(org.apache.kafka.server.common.TopicIdPartition topicPartition, int destinationBrokerId, long logStartOffset)	Should be called whenever the log start offset for a push-replicated partition replica is updated.
void	shutdown()	Shuts this pusher down, terminating all its push replication processing.
void	start()	Starts the pusher thread.
void	startPush(org.apache.kafka.server.common.TopicIdPartition partition, PushSession pushSession)	Adds this partition replica with its push replication session metadata to the set of partitions for which this pusher will be sending AppendRecords requests.
void	stopPush(org.apache.kafka.server.common.TopicIdPartition partition, int destinationBrokerId, PushSessionEndReason pushSessionEndReason)	Stops the push replication session for the given partition, optionally preparing an AppendRecords request signaling the stoppage.

Method Details

id

int id()

An identifier of this pusher thread expected to be between 0 and N-1 where N is the total number of pusher threads on this broker.

startPush

void startPush(org.apache.kafka.server.common.TopicIdPartition partition,
 PushSession pushSession)

Adds this partition replica with its push replication session metadata to the set of partitions for which this pusher will be sending AppendRecords requests. Calling this would set up an initial AppendRecords request for this partition if/when the transitional records for the session are received, and will register a handler for any results of the initial as well as any subsequent AppendRecords requests within this replication session for the partition replica.

Thread-safe, must be called before any subsequent onLeaderAppend(org.apache.kafka.server.common.TopicIdPartition, int, long, org.apache.kafka.common.record.AbstractRecords), onHighWatermarkUpdate(org.apache.kafka.server.common.TopicIdPartition, int, long), or onLogStartOffsetUpdate(org.apache.kafka.server.common.TopicIdPartition, int, long) calls for this partition replica.

onLeaderAppend

void onLeaderAppend(org.apache.kafka.server.common.TopicIdPartition partition,
 int destinationBrokerId,
 long appendOffset,
 org.apache.kafka.common.record.AbstractRecords records)

Buffers the given records to be replicated with some subsequent AppendRecords request.

Thread-safe, must be called after startPush(org.apache.kafka.server.common.TopicIdPartition, io.confluent.kafka.replication.push.PushSession) and before stopPush(org.apache.kafka.server.common.TopicIdPartition, int, io.confluent.kafka.replication.push.PushSessionEndReason) have been called for a particular replication session for this partition replica, maintaining the in-partition record append order.

onHighWatermarkUpdate

void onHighWatermarkUpdate(org.apache.kafka.server.common.TopicIdPartition partition,
 int destinationBrokerId,
 long highWatermark)

Should be called whenever the high watermark for a push-replicated partition replica is updated.

Thread-safe, must be called after startPush(org.apache.kafka.server.common.TopicIdPartition, io.confluent.kafka.replication.push.PushSession) and before stopPush(org.apache.kafka.server.common.TopicIdPartition, int, io.confluent.kafka.replication.push.PushSessionEndReason) have been called for a particular replication session for this partition replica, maintaining the in-partition HWM update order.

onLogStartOffsetUpdate

void onLogStartOffsetUpdate(org.apache.kafka.server.common.TopicIdPartition topicPartition,
 int destinationBrokerId,
 long logStartOffset)

Should be called whenever the log start offset for a push-replicated partition replica is updated.

Thread-safe, must be called after startPush(org.apache.kafka.server.common.TopicIdPartition, io.confluent.kafka.replication.push.PushSession) and before stopPush(org.apache.kafka.server.common.TopicIdPartition, int, io.confluent.kafka.replication.push.PushSessionEndReason) have been called for a particular replication session for this partition replica, maintaining the in-partition LSO update order.

stopPush

void stopPush(org.apache.kafka.server.common.TopicIdPartition partition,
 int destinationBrokerId,
 PushSessionEndReason pushSessionEndReason)

Stops the push replication session for the given partition, optionally preparing an AppendRecords request signaling the stoppage. Subsequent partition update events won't be transformed into AppendRecords requests until a new push replication session has been started, and any buffered state related to the stopped session will be cleared.

Thread-safe, this can be called even without a preceding startPush(org.apache.kafka.server.common.TopicIdPartition, io.confluent.kafka.replication.push.PushSession) and can be called multiple times in succession. If requested so (via the sendEndSessionRequest flag) this will still prepare an AppendRecords request signaling the stoppage.

Parameters:

partition - The partition for which a push replication session is being stopped.

destinationBrokerId - The broker ID of the replica for the stopped session.

pushSessionEndReason - Indicates the reason to end a push session and whether the replica should be explicitly notified in a subsequent AppendRecords request.

start

void start()

Starts the pusher thread.

shutdown

void shutdown()

Shuts this pusher down, terminating all its push replication processing. Once shut down, a pusher can no longer be used - any subsequent calls to startPush(org.apache.kafka.server.common.TopicIdPartition, io.confluent.kafka.replication.push.PushSession) or onLeaderAppend(org.apache.kafka.server.common.TopicIdPartition, int, long, org.apache.kafka.common.record.AbstractRecords)/onHighWatermarkUpdate(org.apache.kafka.server.common.TopicIdPartition, int, long)/onLogStartOffsetUpdate(org.apache.kafka.server.common.TopicIdPartition, int, long) would throw and any subsequent calls to stopPush(org.apache.kafka.server.common.TopicIdPartition, int, io.confluent.kafka.replication.push.PushSessionEndReason) won't have any effect.