Interface Pusher

public interface Pusher
Used for proactively replicating (pushing) records for partitions led by this broker to its follower brokers. Records for partitions handled by the pusher are buffered within it and are regularly sent to followers.

  • Method Summary

    Modifier and Type
    Method
    Description
    int
    id()
    An identifier of this pusher thread expected to be between 0 and N-1 where N is the total number of pusher threads on this broker.
    void
    onHighWatermarkUpdate(org.apache.kafka.server.common.TopicIdPartition partition, int destinationBrokerId, long highWatermark)
    Should be called whenever the high watermark for a push-replicated partition replica is updated.
    void
    onLeaderAppend(org.apache.kafka.server.common.TopicIdPartition partition, int destinationBrokerId, long appendOffset, org.apache.kafka.common.record.internal.AbstractRecords records)
    Buffers the given records to be replicated with some subsequent AppendRecords request.
    void
    onLogStartOffsetUpdate(org.apache.kafka.server.common.TopicIdPartition topicPartition, int destinationBrokerId, long logStartOffset)
    Should be called whenever the log start offset for a push-replicated partition replica is updated.
    void
    shutdown()
    Shuts this pusher down, terminating all its push replication processing.
    void
    start()
    Starts the pusher thread.
    void
    startPush(org.apache.kafka.server.common.TopicIdPartition partition, PushSession pushSession)
    Adds this partition replica with its push replication session metadata to the set of partitions for which this pusher will be sending AppendRecords requests.
    void
    stopPush(org.apache.kafka.server.common.TopicIdPartition partition, int destinationBrokerId, PushSessionEndReason pushSessionEndReason)
    Stops the push replication session for the given partition, optionally preparing an AppendRecords request signaling the stoppage.

  • Method Details

    • id

      int id()
      An identifier of this pusher thread expected to be between 0 and N-1 where N is the total number of pusher threads on this broker.

    • startPush

      void startPush(org.apache.kafka.server.common.TopicIdPartition partition, PushSession pushSession)

      Adds this partition replica with its push replication session metadata to the set of partitions for which this pusher will be sending AppendRecords requests. Calling this would set up an initial AppendRecords request for this partition if/when the transitional records for the session are received, and will register a handler for any results of the initial as well as any subsequent AppendRecords requests within this replication session for the partition replica.

      Thread-safe, must be called before any subsequent onLeaderAppend(TopicIdPartition, int, long, AbstractRecords), onHighWatermarkUpdate(TopicIdPartition, int, long), or onLogStartOffsetUpdate(TopicIdPartition, int, long) calls for this partition replica.

    • onLeaderAppend

      void onLeaderAppend(org.apache.kafka.server.common.TopicIdPartition partition, int destinationBrokerId, long appendOffset, org.apache.kafka.common.record.internal.AbstractRecords records)

      Buffers the given records to be replicated with some subsequent AppendRecords request.

      Thread-safe, must be called after startPush(TopicIdPartition, PushSession) and before stopPush(TopicIdPartition, int, PushSessionEndReason) have been called for a particular replication session for this partition replica, maintaining the in-partition record append order.

    • onHighWatermarkUpdate

      void onHighWatermarkUpdate(org.apache.kafka.server.common.TopicIdPartition partition, int destinationBrokerId, long highWatermark)

      Should be called whenever the high watermark for a push-replicated partition replica is updated.

      Thread-safe, must be called after startPush(TopicIdPartition, PushSession) and before stopPush(TopicIdPartition, int, PushSessionEndReason) have been called for a particular replication session for this partition replica, maintaining the in-partition HWM update order.

    • onLogStartOffsetUpdate

      void onLogStartOffsetUpdate(org.apache.kafka.server.common.TopicIdPartition topicPartition, int destinationBrokerId, long logStartOffset)

      Should be called whenever the log start offset for a push-replicated partition replica is updated.

      Thread-safe, must be called after startPush(TopicIdPartition, PushSession) and before stopPush(TopicIdPartition, int, PushSessionEndReason) have been called for a particular replication session for this partition replica, maintaining the in-partition LSO update order.

    • stopPush

      void stopPush(org.apache.kafka.server.common.TopicIdPartition partition, int destinationBrokerId, PushSessionEndReason pushSessionEndReason)

      Stops the push replication session for the given partition, optionally preparing an AppendRecords request signaling the stoppage. Subsequent partition update events won't be transformed into AppendRecords requests until a new push replication session has been started, and any buffered state related to the stopped session will be cleared.

      Thread-safe, this can be called even without a preceding startPush(TopicIdPartition, PushSession) and can be called multiple times in succession. If requested so (via the sendEndSessionRequest flag) this will still prepare an AppendRecords request signaling the stoppage.

      Parameters:
      partition - The partition for which a push replication session is being stopped.
      destinationBrokerId - The broker ID of the replica for the stopped session.
      pushSessionEndReason - Indicates the reason to end a push session and whether the replica should be explicitly notified in a subsequent AppendRecords request.

    • start

      void start()
      Starts the pusher thread.

    • shutdown

      void shutdown()
      Shuts this pusher down, terminating all its push replication processing. Once shut down, a pusher can no longer be used - any subsequent calls to startPush(TopicIdPartition, PushSession) or onLeaderAppend(TopicIdPartition, int, long, AbstractRecords)/onHighWatermarkUpdate(TopicIdPartition, int, long)/onLogStartOffsetUpdate(TopicIdPartition, int, long) would throw and any subsequent calls to stopPush(TopicIdPartition, int, PushSessionEndReason) won't have any effect.