Package org.apache.flink.optimizer.dataproperties

Class RequestedGlobalProperties

  • All Implemented Interfaces:
    Cloneable
    public final class RequestedGlobalProperties
extends Object
implements Cloneable
    This class represents the global properties of the data that are requested by an operator. Operators request the global properties they need for correct execution. This list is an example of global properties requested by certain operators:
    • "groupBy/reduce" will request the data to be partitioned in some way after the key fields.
    • "map" will request the data to be in an arbitrary distribution - it has no prerequisites
    • "join" will request certain properties for each input. This class represents the properties on an input alone. The properties may be partitioning on the key fields, or a combination of replication on one input and anything-but-replication on the other input.

    • Constructor Detail

      • RequestedGlobalProperties

        public RequestedGlobalProperties()
        Initializes the global properties with no partitioning.

    • Method Detail

      • setHashPartitioned

        public void setHashPartitioned​(org.apache.flink.api.common.operators.util.FieldSet partitionedFields)
        Sets these properties to request a hash partitioning on the given fields.

        If the fields are provided as FieldSet, then any permutation of the fields is a valid partitioning, including subsets. If the fields are given as a FieldList, then only an exact partitioning on the fields matches this requested partitioning.

        Parameters:
        partitionedFields - The key fields for the partitioning.

      • setRangePartitioned

        public void setRangePartitioned​(org.apache.flink.api.common.operators.Ordering ordering)

      • setRangePartitioned

        public void setRangePartitioned​(org.apache.flink.api.common.operators.Ordering ordering,
                                org.apache.flink.api.common.distributions.DataDistribution dataDistribution)

      • setAnyPartitioning

        public void setAnyPartitioning​(org.apache.flink.api.common.operators.util.FieldSet partitionedFields)
        Sets these properties to request some partitioning on the given fields. This will allow both hash partitioning and range partitioning to match.

        If the fields are provided as FieldSet, then any permutation of the fields is a valid partitioning, including subsets. If the fields are given as a FieldList, then only an exact partitioning on the fields matches this requested partitioning.

        Parameters:
        partitionedFields - The key fields for the partitioning.

      • setRandomPartitioning

        public void setRandomPartitioning()

      • setAnyDistribution

        public void setAnyDistribution()

      • setFullyReplicated

        public void setFullyReplicated()

      • setForceRebalancing

        public void setForceRebalancing()

      • setCustomPartitioned

        public void setCustomPartitioned​(org.apache.flink.api.common.operators.util.FieldSet partitionedFields,
                                 org.apache.flink.api.common.functions.Partitioner<?> partitioner)
        Sets these properties to request a custom partitioning with the given Partitioner instance.

        If the fields are provided as FieldSet, then any permutation of the fields is a valid partitioning, including subsets. If the fields are given as a FieldList, then only an exact partitioning on the fields matches this requested partitioning.

        Parameters:
        partitionedFields - The key fields for the partitioning.

      • getPartitioning

        public PartitioningProperty getPartitioning()
        Gets the partitioning property.
        Returns:
        The partitioning property.

      • getPartitionedFields

        public org.apache.flink.api.common.operators.util.FieldSet getPartitionedFields()
        Gets the fields on which the data is partitioned.
        Returns:
        The partitioning fields.

      • getOrdering

        public org.apache.flink.api.common.operators.Ordering getOrdering()
        Gets the key order.
        Returns:
        The key order.

      • getDataDistribution

        public org.apache.flink.api.common.distributions.DataDistribution getDataDistribution()
        Gets the data distribution.
        Returns:
        The data distribution.

      • getCustomPartitioner

        public org.apache.flink.api.common.functions.Partitioner<?> getCustomPartitioner()
        Gets the custom partitioner associated with these properties.
        Returns:
        The custom partitioner associated with these properties.

      • isTrivial

        public boolean isTrivial()
        Checks, if the properties in this object are trivial, i.e. only standard values.

      • reset

        public void reset()
        This method resets the properties to a state where no properties are given.

      • filterBySemanticProperties

        public RequestedGlobalProperties filterBySemanticProperties​(org.apache.flink.api.common.operators.SemanticProperties props,
                                                            int input)
        Filters these properties by what can be preserved by the given SemanticProperties when propagated down to the given input.
        Parameters:
        props - The SemanticProperties which define which fields are preserved.
        input - The index of the operator's input.
        Returns:
        The filtered RequestedGlobalProperties

      • isMetBy

        public boolean isMetBy​(GlobalProperties props)
        Checks, if this set of interesting properties, is met by the given produced properties.
        Parameters:
        props - The properties for which to check whether they meet these properties.
        Returns:
        True, if the properties are met, false otherwise.

      • parameterizeChannel

        public void parameterizeChannel​(Channel channel,
                                boolean globalDopChange,
                                org.apache.flink.api.common.ExecutionMode exchangeMode,
                                boolean breakPipeline)
        Parametrizes the ship strategy fields of a channel such that the channel produces the desired global properties.
        Parameters:
        channel - The channel to parametrize.
        globalDopChange - Flag indicating whether the parallelism changes between sender and receiver.
        exchangeMode - The mode of data exchange (pipelined, always batch, batch only on shuffle, ...)
        breakPipeline - Indicates whether this data exchange should break pipelines (unless pipelines are forced).

      • hashCode

        public int hashCode()
        Overrides:
        hashCode in class Object