Package org.apache.flink.runtime.taskmanager

Class NettyShuffleEnvironmentConfiguration

  • public class NettyShuffleEnvironmentConfiguration
extends Object
    Configuration object for the network stack.

    • Constructor Detail

      • NettyShuffleEnvironmentConfiguration

        public NettyShuffleEnvironmentConfiguration​(int numNetworkBuffers,
                                            int networkBufferSize,
                                            int partitionRequestInitialBackoff,
                                            int partitionRequestMaxBackoff,
                                            int networkBuffersPerChannel,
                                            int floatingNetworkBuffersPerGate,
                                            Optional<Integer> maxRequiredBuffersPerGate,
                                            java.time.Duration requestSegmentsTimeout,
                                            boolean isNetworkDetailedMetrics,
                                            @Nullable
                                            NettyConfig nettyConfig,
                                            String[] tempDirs,
                                            BoundedBlockingSubpartitionType blockingSubpartitionType,
                                            boolean batchShuffleCompressionEnabled,
                                            String compressionCodec,
                                            int maxBuffersPerChannel,
                                            long batchShuffleReadMemoryBytes,
                                            int sortShuffleMinBuffers,
                                            int sortShuffleMinParallelism,
                                            BufferDebloatConfiguration debloatConfiguration,
                                            int maxNumberOfConnections,
                                            boolean connectionReuseEnabled,
                                            int maxOverdraftBuffersPerGate,
                                            int hybridShuffleSpilledIndexRegionGroupSize,
                                            long hybridShuffleNumRetainedInMemoryRegionsMax,
                                            @Nullable
                                            TieredStorageConfiguration tieredStorageConfiguration)

    • Method Detail

      • numNetworkBuffers

        public int numNetworkBuffers()

      • networkBufferSize

        public int networkBufferSize()

      • partitionRequestInitialBackoff

        public int partitionRequestInitialBackoff()

      • partitionRequestMaxBackoff

        public int partitionRequestMaxBackoff()

      • networkBuffersPerChannel

        public int networkBuffersPerChannel()

      • floatingNetworkBuffersPerGate

        public int floatingNetworkBuffersPerGate()

      • maxRequiredBuffersPerGate

        public Optional<Integer> maxRequiredBuffersPerGate()

      • batchShuffleReadMemoryBytes

        public long batchShuffleReadMemoryBytes()

      • sortShuffleMinBuffers

        public int sortShuffleMinBuffers()

      • sortShuffleMinParallelism

        public int sortShuffleMinParallelism()

      • getRequestSegmentsTimeout

        public java.time.Duration getRequestSegmentsTimeout()

      • isNetworkDetailedMetrics

        public boolean isNetworkDetailedMetrics()

      • getTempDirs

        public String[] getTempDirs()

      • isConnectionReuseEnabled

        public boolean isConnectionReuseEnabled()

      • isBatchShuffleCompressionEnabled

        public boolean isBatchShuffleCompressionEnabled()

      • isSSLEnabled

        public boolean isSSLEnabled()

      • getCompressionCodec

        public String getCompressionCodec()

      • getMaxBuffersPerChannel

        public int getMaxBuffersPerChannel()

      • getMaxNumberOfConnections

        public int getMaxNumberOfConnections()

      • getMaxOverdraftBuffersPerGate

        public int getMaxOverdraftBuffersPerGate()

      • getHybridShuffleNumRetainedInMemoryRegionsMax

        public long getHybridShuffleNumRetainedInMemoryRegionsMax()

      • getHybridShuffleSpilledIndexRegionGroupSize

        public int getHybridShuffleSpilledIndexRegionGroupSize()

      • fromConfiguration

        public static NettyShuffleEnvironmentConfiguration fromConfiguration​(org.apache.flink.configuration.Configuration configuration,
                                                                     org.apache.flink.configuration.MemorySize networkMemorySize,
                                                                     boolean localTaskManagerCommunication,
                                                                     InetAddress taskManagerAddress)
        Utility method to extract network related parameters from the configuration and to sanity check them.
        Parameters:
        configuration - configuration object
        networkMemorySize - the size of memory reserved for shuffle environment
        localTaskManagerCommunication - true, to skip initializing the network stack
        taskManagerAddress - identifying the IP address under which the TaskManager will be accessible
        Returns:
        NettyShuffleEnvironmentConfiguration

      • hashCode

        public int hashCode()
        Overrides:
        hashCode in class Object