Class HsMemoryDataSpiller

  • All Implemented Interfaces:
    AutoCloseable

    public class HsMemoryDataSpiller
    extends Object
    implements AutoCloseable
    This component is responsible for asynchronously writing in-memory data to disk. Each spilling operation will write the disk file sequentially.
    • Method Detail

      • spillAsync

        public CompletableFuture<List<HsFileDataIndex.SpilledBuffer>> spillAsync​(List<BufferWithIdentity> bufferToSpill)
        Spilling buffers to disk asynchronously.
        Parameters:
        bufferToSpill - buffers need to be spilled, must ensure that it is sorted by (subpartitionId, bufferIndex).
        Returns:
        the completable future contains spilled buffers information.
      • close

        public void close()
        Close this HsMemoryDataSpiller. It means spiller will no longer accept new spilling operation and wait for all previous spilling operation done blocking.
        Specified by:
        close in interface AutoCloseable