Class HsBufferContext


  • public class HsBufferContext
    extends Object
    This class maintains the buffer's reference count and its status for hybrid shuffle mode.

    Each buffer has three status: released, spillStarted, consumed.

    • released indicates that buffer has been released from the memory data manager, and can no longer be spilled or consumed.
    • spillStarted indicates that spilling of the buffer has started, either completed or not.
    • consumed indicates that buffer has been consumed by these consumers.

    Reference count of the buffer is maintained as follows: *

    • +1 when the buffer is obtained by memory data manager (from the buffer pool), and -1 when it is released from memory data manager.
    • +1 when spilling of the buffer is tarted, and -1 when it is completed.
    • +1 when the buffer is being consumed, and -1 when consuming is completed (by the downstream).

    Note: This class is not thread-safe.

    • Constructor Detail

      • HsBufferContext

        public HsBufferContext​(Buffer buffer,
                               int bufferIndex,
                               int subpartitionId)
    • Method Detail

      • getBuffer

        public Buffer getBuffer()
      • isReleased

        public boolean isReleased()
      • isSpillStarted

        public boolean isSpillStarted()
      • isConsumed

        public boolean isConsumed​(HsConsumerId consumerId)
      • release

        public void release()
        Mark buffer status to release.
      • startSpilling

        public boolean startSpilling​(CompletableFuture<Void> spilledFuture)
        Mark buffer status to startSpilling.
        Parameters:
        spilledFuture - completable future of this buffer's spilling operation.
        Returns:
        false, if spilling of the buffer has been started before or the buffer has been released already; true, otherwise.
      • consumed

        public void consumed​(HsConsumerId consumerId)