Class HsMemoryDataSpiller
- java.lang.Object
-
- org.apache.flink.runtime.io.network.partition.hybrid.HsMemoryDataSpiller
-
- All Implemented Interfaces:
AutoCloseable
public class HsMemoryDataSpiller extends Object implements AutoCloseable
This component is responsible for asynchronously writing in-memory data to disk. Each spilling operation will write the disk file sequentially.
-
-
Constructor Summary
Constructors Constructor Description HsMemoryDataSpiller(Path dataFilePath)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description voidclose()Close thisHsMemoryDataSpiller.CompletableFuture<List<HsFileDataIndex.SpilledBuffer>>spillAsync(List<BufferWithIdentity> bufferToSpill)Spilling buffers to disk asynchronously.
-
-
-
Constructor Detail
-
HsMemoryDataSpiller
public HsMemoryDataSpiller(Path dataFilePath) throws IOException
- Throws:
IOException
-
-
Method Detail
-
spillAsync
public CompletableFuture<List<HsFileDataIndex.SpilledBuffer>> spillAsync(List<BufferWithIdentity> bufferToSpill)
Spilling buffers to disk asynchronously.- Parameters:
bufferToSpill- buffers need to be spilled, must ensure that it is sorted by (subpartitionId, bufferIndex).- Returns:
- the completable future contains spilled buffers information.
-
close
public void close()
Close thisHsMemoryDataSpiller. It means spiller will no longer accept new spilling operation and wait for all previous spilling operation done blocking.- Specified by:
closein interfaceAutoCloseable
-
-