Class TupleComparatorBase<T>
- java.lang.Object
-
- org.apache.flink.api.common.typeutils.TypeComparator<T>
-
- org.apache.flink.api.common.typeutils.CompositeTypeComparator<T>
-
- org.apache.flink.api.java.typeutils.runtime.TupleComparatorBase<T>
-
- All Implemented Interfaces:
Serializable
- Direct Known Subclasses:
TupleComparator
@Internal public abstract class TupleComparatorBase<T> extends CompositeTypeComparator<T> implements Serializable
- See Also:
- Serialized Form
-
-
Field Summary
Fields Modifier and Type Field Description protected TypeComparator[]comparatorscomparators for the key fields, in the same order as the key fieldsprotected Object[]deserializedFields1protected Object[]deserializedFields2static int[]HASH_SALTA sequence of prime numbers to be used for salting the computed hash values.protected booleaninvertNormKeyprotected int[]keyPositionskey positions describe which fields are keys in what orderprotected intnormalizableKeyPrefixLenprotected int[]normalizedKeyLengthsprotected intnumLeadingNormalizableKeysprotected TypeSerializer[]serializersserializers to deserialize the first n fields for comparison
-
Constructor Summary
Constructors Modifier Constructor Description TupleComparatorBase(int[] keyPositions, TypeComparator<?>[] comparators, TypeSerializer<?>[] serializers)protectedTupleComparatorBase(TupleComparatorBase<T> toClone)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description intcompareSerialized(DataInputView firstSource, DataInputView secondSource)Compares two records in serialized form.intcompareToReference(TypeComparator<T> referencedComparator)This method compares the element that has been set as reference in this type accessor, to the element set as reference in the given type accessor.voidgetFlatComparator(List<TypeComparator> flatComparators)protected int[]getKeyPositions()intgetNormalizeKeyLen()Gets the number of bytes that the normalized key would maximally take.protected voidinstantiateDeserializationUtils()booleaninvertNormalizedKey()Flag whether normalized key comparisons should be inverted key should be interpreted inverted, i.e.booleanisNormalizedKeyPrefixOnly(int keyBytes)Checks, whether the given number of bytes for a normalized is only a prefix to determine the order of elements of the data type for which this comparator provides the comparison methods.protected voidprivateDuplicate(TupleComparatorBase<T> toClone)TreadWithKeyDenormalization(T reuse, DataInputView source)Reads the record back while de-normalizing the key fields.booleansupportsNormalizedKey()Checks whether the data type supports the creation of a normalized key for comparison.booleansupportsSerializationWithKeyNormalization()Check whether this comparator supports to serialize the record in a format that replaces its keys by a normalized key.voidwriteWithKeyNormalization(T record, DataOutputView target)Writes the record in such a fashion that all keys are normalizing and at the beginning of the serialized data.-
Methods inherited from class org.apache.flink.api.common.typeutils.CompositeTypeComparator
getFlatComparators
-
Methods inherited from class org.apache.flink.api.common.typeutils.TypeComparator
compare, compareAgainstReference, duplicate, equalToReference, extractKeys, hash, putNormalizedKey, setReference, supportsCompareAgainstReference
-
-
-
-
Field Detail
-
keyPositions
protected int[] keyPositions
key positions describe which fields are keys in what order
-
comparators
protected TypeComparator[] comparators
comparators for the key fields, in the same order as the key fields
-
normalizedKeyLengths
protected int[] normalizedKeyLengths
-
numLeadingNormalizableKeys
protected int numLeadingNormalizableKeys
-
normalizableKeyPrefixLen
protected int normalizableKeyPrefixLen
-
invertNormKey
protected boolean invertNormKey
-
serializers
protected TypeSerializer[] serializers
serializers to deserialize the first n fields for comparison
-
deserializedFields1
protected transient Object[] deserializedFields1
-
deserializedFields2
protected transient Object[] deserializedFields2
-
HASH_SALT
public static final int[] HASH_SALT
A sequence of prime numbers to be used for salting the computed hash values. Based on some empirical evidence, we are using a 32-element subsequence of the OEIS sequence #A068652 (numbers such that every cyclic permutation is a prime).
-
-
Constructor Detail
-
TupleComparatorBase
public TupleComparatorBase(int[] keyPositions, TypeComparator<?>[] comparators, TypeSerializer<?>[] serializers)
-
TupleComparatorBase
protected TupleComparatorBase(TupleComparatorBase<T> toClone)
-
-
Method Detail
-
privateDuplicate
protected void privateDuplicate(TupleComparatorBase<T> toClone)
-
getKeyPositions
protected int[] getKeyPositions()
-
getFlatComparator
public void getFlatComparator(List<TypeComparator> flatComparators)
- Specified by:
getFlatComparatorin classCompositeTypeComparator<T>
-
compareToReference
public int compareToReference(TypeComparator<T> referencedComparator)
Description copied from class:TypeComparatorThis method compares the element that has been set as reference in this type accessor, to the element set as reference in the given type accessor. Similar to comparing two elementse1ande2via a comparator, this method can be used the following way.
The rational behind this method is that elements are typically compared using certain features that are extracted from them, (such de-serializing as a subset of fields). When setting the reference, this extraction happens. The extraction needs happen only once per element, even though an element is typically compared to many other elements when establishing a sorted order. The actual comparison performed by this method may be very cheap, as it happens on the extracted features.E e1 = ...; E e2 = ...; TypeComparator<E> acc1 = ...; TypeComparator<E> acc2 = ...; acc1.setReference(e1); acc2.setReference(e2); int comp = acc1.compareToReference(acc2);- Specified by:
compareToReferencein classTypeComparator<T>- Parameters:
referencedComparator- The type accessors where the element for comparison has been set as reference.- Returns:
- A value smaller than zero, if the reference value of
referencedAccessorsis smaller than the reference value of this type accessor; a value greater than zero, if it is larger; zero, if both are equal. - See Also:
TypeComparator.setReference(Object)
-
compareSerialized
public int compareSerialized(DataInputView firstSource, DataInputView secondSource) throws IOException
Description copied from class:TypeComparatorCompares two records in serialized form. The return value indicates the order of the two in the same way as defined byComparator.compare(Object, Object).This method may de-serialize the records or compare them directly based on their binary representation.
- Specified by:
compareSerializedin classTypeComparator<T>- Parameters:
firstSource- The input view containing the first record.secondSource- The input view containing the second record.- Returns:
- An integer defining the oder among the objects in the same way as
Comparator.compare(Object, Object). - Throws:
IOException- Thrown, if any of the input views raised an exception when reading the records.- See Also:
Comparator.compare(Object, Object)
-
supportsNormalizedKey
public boolean supportsNormalizedKey()
Description copied from class:TypeComparatorChecks whether the data type supports the creation of a normalized key for comparison.- Specified by:
supportsNormalizedKeyin classTypeComparator<T>- Returns:
- True, if the data type supports the creation of a normalized key for comparison, false otherwise.
-
getNormalizeKeyLen
public int getNormalizeKeyLen()
Description copied from class:TypeComparatorGets the number of bytes that the normalized key would maximally take. A value ofInteger.MAX_VALUE is interpreted as infinite.- Specified by:
getNormalizeKeyLenin classTypeComparator<T>- Returns:
- The number of bytes that the normalized key would maximally take.
-
isNormalizedKeyPrefixOnly
public boolean isNormalizedKeyPrefixOnly(int keyBytes)
Description copied from class:TypeComparatorChecks, whether the given number of bytes for a normalized is only a prefix to determine the order of elements of the data type for which this comparator provides the comparison methods. For example, if the data type is ordered with respect to an integer value it contains, then this method would return true, if the number of key bytes is smaller than four.- Specified by:
isNormalizedKeyPrefixOnlyin classTypeComparator<T>- Returns:
- True, if the given number of bytes is only a prefix, false otherwise.
-
invertNormalizedKey
public boolean invertNormalizedKey()
Description copied from class:TypeComparatorFlag whether normalized key comparisons should be inverted key should be interpreted inverted, i.e. descending.- Specified by:
invertNormalizedKeyin classTypeComparator<T>- Returns:
- True, if all normalized key comparisons should invert the sign of the comparison result, false if the normalized key should be used as is.
-
supportsSerializationWithKeyNormalization
public boolean supportsSerializationWithKeyNormalization()
Description copied from class:TypeComparatorCheck whether this comparator supports to serialize the record in a format that replaces its keys by a normalized key.- Specified by:
supportsSerializationWithKeyNormalizationin classTypeComparator<T>- Returns:
- True, if the comparator supports that specific form of serialization, false if not.
-
writeWithKeyNormalization
public void writeWithKeyNormalization(T record, DataOutputView target) throws IOException
Description copied from class:TypeComparatorWrites the record in such a fashion that all keys are normalizing and at the beginning of the serialized data. This must only be used when for all the key fields the full normalized key is used. The method#supportsSerializationWithKeyNormalization()allows to check that.- Specified by:
writeWithKeyNormalizationin classTypeComparator<T>- Parameters:
record- The record object into which to read the record data.target- The stream to which to write the data,- Throws:
IOException- See Also:
TypeComparator.supportsSerializationWithKeyNormalization(),TypeComparator.readWithKeyDenormalization(Object, DataInputView),NormalizableKey.copyNormalizedKey(MemorySegment, int, int)
-
readWithKeyDenormalization
public T readWithKeyDenormalization(T reuse, DataInputView source) throws IOException
Description copied from class:TypeComparatorReads the record back while de-normalizing the key fields. This must only be used when for all the key fields the full normalized key is used, which is hinted by the#supportsSerializationWithKeyNormalization()method.- Specified by:
readWithKeyDenormalizationin classTypeComparator<T>- Parameters:
reuse- The reuse object into which to read the record data.source- The stream from which to read the data,- Throws:
IOException- See Also:
TypeComparator.supportsSerializationWithKeyNormalization(),TypeComparator.writeWithKeyNormalization(Object, DataOutputView),NormalizableKey.copyNormalizedKey(MemorySegment, int, int)
-
instantiateDeserializationUtils
protected final void instantiateDeserializationUtils()
-
-