Package org.apache.flink.runtime.io.network.partition.hybrid

Class HsSelectiveSpillingStrategy

java.lang.Object

org.apache.flink.runtime.io.network.partition.hybrid.HsSelectiveSpillingStrategy

All Implemented Interfaces:

HsSpillingStrategy

public class HsSelectiveSpillingStrategy
extends Object
implements HsSpillingStrategy

A special implementation of HsSpillingStrategy that reduce disk writes as much as possible.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.io.network.partition.hybrid.HsSpillingStrategy

HsSpillingStrategy.Decision

Constructor Summary

Constructors

Constructor	Description
HsSelectiveSpillingStrategy(HybridShuffleConfiguration hybridShuffleConfiguration)

Method Summary

Modifier and Type	Method	Description
HsSpillingStrategy.Decision	decideActionWithGlobalInfo(HsSpillingInfoProvider spillingInfoProvider)	Make a decision based on global information.
Optional<HsSpillingStrategy.Decision>	onBufferConsumed(BufferIndexAndChannel consumedBuffer)	Make a decision when a buffer is consumed.
Optional<HsSpillingStrategy.Decision>	onBufferFinished(int numTotalUnSpillBuffers, int currentPoolSize)	Make a decision when a buffer becomes finished.
Optional<HsSpillingStrategy.Decision>	onMemoryUsageChanged(int numTotalRequestedBuffers, int currentPoolSize)	Make a decision when memory usage is changed.
HsSpillingStrategy.Decision	onResultPartitionClosed(HsSpillingInfoProvider spillingInfoProvider)	Make a decision when result partition is closed.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HsSelectiveSpillingStrategy

public HsSelectiveSpillingStrategy(HybridShuffleConfiguration hybridShuffleConfiguration)

Method Detail

onBufferFinished

public Optional<HsSpillingStrategy.Decision> onBufferFinished(int numTotalUnSpillBuffers,
                                                              int currentPoolSize)

Description copied from interface: HsSpillingStrategy

Make a decision when a buffer becomes finished.

Specified by:

onBufferFinished in interface HsSpillingStrategy

Parameters:

numTotalUnSpillBuffers - total number of buffers not spill.

Returns:

A HsSpillingStrategy.Decision based on the provided information, or Optional.empty() if the decision cannot be made, which indicates global information is needed.

onBufferConsumed

public Optional<HsSpillingStrategy.Decision> onBufferConsumed(BufferIndexAndChannel consumedBuffer)

Description copied from interface: HsSpillingStrategy

Make a decision when a buffer is consumed.

Specified by:

onBufferConsumed in interface HsSpillingStrategy

Parameters:

consumedBuffer - the buffer that is consumed.

Returns:

A HsSpillingStrategy.Decision based on the provided information, or Optional.empty() if the decision cannot be made, which indicates global information is needed.

onMemoryUsageChanged

public Optional<HsSpillingStrategy.Decision> onMemoryUsageChanged(int numTotalRequestedBuffers,
                                                                  int currentPoolSize)

Description copied from interface: HsSpillingStrategy

Make a decision when memory usage is changed.

Specified by:

onMemoryUsageChanged in interface HsSpillingStrategy

Parameters:

numTotalRequestedBuffers - total number of buffers requested.

currentPoolSize - current value of buffer pool size.

Returns:

A HsSpillingStrategy.Decision based on the provided information, or Optional.empty() if the decision cannot be made, which indicates global information is needed.

decideActionWithGlobalInfo

public HsSpillingStrategy.Decision decideActionWithGlobalInfo(HsSpillingInfoProvider spillingInfoProvider)

Description copied from interface: HsSpillingStrategy

Make a decision based on global information. Because this method will directly touch the HsSpillingInfoProvider, the caller should take care of the thread safety.

Specified by:

decideActionWithGlobalInfo in interface HsSpillingStrategy

Parameters:

spillingInfoProvider - that provides information about the current status.

Returns:

A HsSpillingStrategy.Decision based on the global information.

onResultPartitionClosed

public HsSpillingStrategy.Decision onResultPartitionClosed(HsSpillingInfoProvider spillingInfoProvider)

Description copied from interface: HsSpillingStrategy

Make a decision when result partition is closed. Because this method will directly touch the HsSpillingInfoProvider, the caller should take care of the thread safety.

Specified by:

onResultPartitionClosed in interface HsSpillingStrategy

Parameters:

spillingInfoProvider - that provides information about the current status.

Returns:

A HsSpillingStrategy.Decision based on the global information.