Package org.apache.flink.runtime.io.network.partition.hybrid.tiered.common

Class TieredStorageUtils

java.lang.Object

org.apache.flink.runtime.io.network.partition.hybrid.tiered.common.TieredStorageUtils

public class TieredStorageUtils
extends Object

Utils for reading from or writing to tiered storage.

Constructor Summary

Constructors

Constructor	Description
TieredStorageUtils()

Method Summary

Modifier and Type	Method	Description
static ByteBuffer[]	generateBufferWithHeaders(List<org.apache.flink.api.java.tuple.Tuple2<Buffer,Integer>> bufferWithIndexes)
static int	getAccumulatorExclusiveBuffers()	Get exclusive buffer number of accumulator.
static String	getDiskTierName()
static String	getMemoryTierName()
static int	getMinBuffersPerGate()	Get the number of minimum buffers per input gate.
static int	getMinBuffersPerResultPartition()	*
static float	getNumBuffersTriggerFlushRatio()	When the number of buffers that have been requested exceeds this threshold, trigger the flushing operation in each TierProducerAgent.
static long	getPoolSizeCheckInterval()	Get the pool size check interval.
static String	getRemoteTierName()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TieredStorageUtils

public TieredStorageUtils()

Method Detail

getNumBuffersTriggerFlushRatio

public static float getNumBuffersTriggerFlushRatio()

When the number of buffers that have been requested exceeds this threshold, trigger the flushing operation in each TierProducerAgent.

Returns:

flush ratio.

getAccumulatorExclusiveBuffers

public static int getAccumulatorExclusiveBuffers()

Get exclusive buffer number of accumulator.

The buffer number is used to compare with the subpartition number to determine the type of BufferAccumulator.

If the exclusive buffer number is larger than (subpartitionNum + 1), the accumulator will use HashBufferAccumulator. If the exclusive buffer number is equal to or smaller than (subpartitionNum + 1), the accumulator will use SortBufferAccumulator

Returns:

the buffer number.

getPoolSizeCheckInterval

public static long getPoolSizeCheckInterval()

Get the pool size check interval.

getMinBuffersPerGate

public static int getMinBuffersPerGate()

Get the number of minimum buffers per input gate. It is only used when taskmanager.network.hybrid-shuffle.memory-decoupling.enabled is set to true.

Returns:

the buffer number.

getMinBuffersPerResultPartition

public static int getMinBuffersPerResultPartition()

*

Get the number of minimum buffers per result partition.

Returns:

the buffer number.

getMemoryTierName

public static String getMemoryTierName()

getDiskTierName

public static String getDiskTierName()

getRemoteTierName

public static String getRemoteTierName()

generateBufferWithHeaders

public static ByteBuffer[] generateBufferWithHeaders(List<org.apache.flink.api.java.tuple.Tuple2<Buffer,Integer>> bufferWithIndexes)