Package org.apache.flink.runtime.io.network.partition.hybrid

Class HsFileDataManager

java.lang.Object

org.apache.flink.runtime.io.network.partition.hybrid.HsFileDataManager

All Implemented Interfaces:

Runnable, BufferRecycler

@ThreadSafe
public class HsFileDataManager
extends Object
implements Runnable, BufferRecycler

File data manager for HsResultPartition, which schedules HsSubpartitionFileReaderImpl for loading data w.r.t. their offset in the file.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.io.network.buffer.BufferRecycler

BufferRecycler.DummyBufferRecycler

Constructor Summary

Constructors

Constructor	Description
HsFileDataManager(BatchShuffleReadBufferPool bufferPool, ScheduledExecutorService ioExecutor, HsFileDataIndex dataIndex, Path dataFilePath, HsSubpartitionFileReader.Factory fileReaderFactory, HybridShuffleConfiguration hybridShuffleConfiguration)

Method Summary

Modifier and Type	Method	Description
void	closeDataIndexAndDeleteShuffleFile()
void	recycle(org.apache.flink.core.memory.MemorySegment segment)	Recycles the MemorySegment to its original BufferPool instance.
HsDataView	registerNewConsumer(int subpartitionId, HsConsumerId consumerId, HsSubpartitionConsumerInternalOperations operation)	This method only called by result partition to create subpartitionFileReader.
void	release()	Releases this file data manager and delete shuffle data after all readers is removed.
void	releaseSubpartitionReader(HsSubpartitionFileReader subpartitionFileReader)	Release specific HsSubpartitionFileReader from HsFileDataManager.
void	run()
void	setup()	Setup read buffer pool.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HsFileDataManager

public HsFileDataManager(BatchShuffleReadBufferPool bufferPool,
                         ScheduledExecutorService ioExecutor,
                         HsFileDataIndex dataIndex,
                         Path dataFilePath,
                         HsSubpartitionFileReader.Factory fileReaderFactory,
                         HybridShuffleConfiguration hybridShuffleConfiguration)

Method Detail

setup

public void setup()

Setup read buffer pool.

run

public void run()

Specified by:

run in interface Runnable

registerNewConsumer

public HsDataView registerNewConsumer(int subpartitionId,
                                      HsConsumerId consumerId,
                                      HsSubpartitionConsumerInternalOperations operation)
                               throws IOException

This method only called by result partition to create subpartitionFileReader.

Throws:

IOException

closeDataIndexAndDeleteShuffleFile

public void closeDataIndexAndDeleteShuffleFile()

releaseSubpartitionReader

public void releaseSubpartitionReader(HsSubpartitionFileReader subpartitionFileReader)

Release specific HsSubpartitionFileReader from HsFileDataManager.

Parameters:

subpartitionFileReader - to release.

release

public void release()

Releases this file data manager and delete shuffle data after all readers is removed.

recycle

public void recycle(org.apache.flink.core.memory.MemorySegment segment)

Description copied from interface: BufferRecycler

Recycles the MemorySegment to its original BufferPool instance.

Specified by:

recycle in interface BufferRecycler

Parameters:

segment - The memory segment to be recycled.