Package org.apache.flink.api.common.typeutils

Class CompositeTypeSerializerSnapshot<T,​S extends TypeSerializer<T>>

  • Type Parameters:
    T - The data type that the originating serializer of this snapshot serializes.
    S - The type of the originating serializer.
    All Implemented Interfaces:
    TypeSerializerSnapshot<T>
    Direct Known Subclasses:
    GenericArraySerializerSnapshot, JavaEitherSerializerSnapshot, ListSerializerSnapshot, MapSerializerSnapshot, NullableSerializer.NullableSerializerSnapshot, RowSerializer.RowSerializerSnapshot, SingleThreadAccessCheckingTypeSerializer.SingleThreadAccessCheckingTypeSerializerSnapshot, TupleSerializerSnapshot
    @PublicEvolving
public abstract class CompositeTypeSerializerSnapshot<T,​S extends TypeSerializer<T>>
extends Object
implements TypeSerializerSnapshot<T>
    A CompositeTypeSerializerSnapshot is a convenient serializer snapshot class that can be used by simple serializers which 1) delegates its serialization to multiple nested serializers, and 2) may contain some extra static information that needs to be persisted as part of its snapshot.

    Examples for this would be the ListSerializer, MapSerializer, EitherSerializer, etc., in which case the serializer, called the "outer" serializer in this context, has only some nested serializers that needs to be persisted as its snapshot, and nothing else that needs to be persisted as the "outer" snapshot. An example which has non-empty outer snapshots would be the GenericArraySerializer, which beyond the nested component serializer, also contains a class of the component type that needs to be persisted.

    Serializers that do have some outer snapshot needs to make sure to implement the methods writeOuterSnapshot(DataOutputView), readOuterSnapshot(int, DataInputView, ClassLoader), and resolveOuterSchemaCompatibility(TypeSerializer) when using this class as the base for its serializer snapshot class. By default, the base implementations of these methods are empty, i.e. this class assumes that subclasses do not have any outer snapshot that needs to be persisted.

    Snapshot Versioning

    This base class has its own versioning for the format in which it writes the outer snapshot and the nested serializer snapshots. The version of the serialization format of this based class is defined by getCurrentVersion(). This is independent of the version in which subclasses writes their outer snapshot, defined by getCurrentOuterSnapshotVersion(). This means that the outer snapshot's version can be maintained only taking into account changes in how the outer snapshot is written. Any changes in the base format does not require upticks in the outer snapshot's version.

    Serialization Format

    The current version of the serialization format of a CompositeTypeSerializerSnapshot is as follows: 

    
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 | CompositeTypeSerializerSnapshot | CompositeTypeSerializerSnapshot |          Outer snapshot         |
 |           version               |          MAGIC_NUMBER           |              version            |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                                               Outer snapshot                                        |
 |                                   #writeOuterSnapshot(DataOutputView out)                           |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |      Delegate MAGIC_NUMBER      |         Delegate version        |     Num. nested serializers     |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                                     Nested serializer snapshots                                     |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

    • Constructor Detail

      • CompositeTypeSerializerSnapshot

        public CompositeTypeSerializerSnapshot​(Class<? extends TypeSerializer> correspondingSerializerClass)
        Constructor to be used for read instantiation.
        Parameters:
        correspondingSerializerClass - the expected class of the new serializer.

      • CompositeTypeSerializerSnapshot

        public CompositeTypeSerializerSnapshot​(S serializerInstance)
        Constructor to be used for writing the snapshot.
        Parameters:
        serializerInstance - an instance of the originating serializer of this snapshot.

    • Method Detail

      • getCurrentVersion

        public final int getCurrentVersion()
        Description copied from interface: TypeSerializerSnapshot
        Returns the version of the current snapshot's written binary format.
        Specified by:
        getCurrentVersion in interface TypeSerializerSnapshot<T>
        Returns:
        the version of the current snapshot's written binary format.

      • resolveSchemaCompatibility

        public final TypeSerializerSchemaCompatibility<T> resolveSchemaCompatibility​(TypeSerializer<T> newSerializer)
        Description copied from interface: TypeSerializerSnapshot
        Checks a new serializer's compatibility to read data written by the prior serializer.

        When a checkpoint/savepoint is restored, this method checks whether the serialization format of the data in the checkpoint/savepoint is compatible for the format of the serializer used by the program that restores the checkpoint/savepoint. The outcome can be that the serialization format is compatible, that the program's serializer needs to reconfigure itself (meaning to incorporate some information from the TypeSerializerSnapshot to be compatible), that the format is outright incompatible, or that a migration needed. In the latter case, the TypeSerializerSnapshot produces a serializer to deserialize the data, and the restoring program's serializer re-serializes the data, thus converting the format during the restore operation.

        Specified by:
        resolveSchemaCompatibility in interface TypeSerializerSnapshot<T>
        Parameters:
        newSerializer - the new serializer to check.
        Returns:
        the serializer compatibility result.

      • restoreSerializer

        public final TypeSerializer<T> restoreSerializer()
        Description copied from interface: TypeSerializerSnapshot
        Recreates a serializer instance from this snapshot. The returned serializer can be safely used to read data written by the prior serializer (i.e., the serializer that created this snapshot).
        Specified by:
        restoreSerializer in interface TypeSerializerSnapshot<T>
        Returns:
        a serializer instance restored from this serializer snapshot.

      • getCurrentOuterSnapshotVersion

        protected abstract int getCurrentOuterSnapshotVersion()
        Returns the version of the current outer snapshot's written binary format.
        Returns:
        the version of the current outer snapshot's written binary format.

      • getNestedSerializers

        protected abstract TypeSerializer<?>[] getNestedSerializers​(S outerSerializer)
        Gets the nested serializers from the outer serializer.
        Parameters:
        outerSerializer - the outer serializer.
        Returns:
        the nested serializers.

      • createOuterSerializerWithNestedSerializers

        protected abstract S createOuterSerializerWithNestedSerializers​(TypeSerializer<?>[] nestedSerializers)
        Creates an instance of the outer serializer with a given array of its nested serializers.
        Parameters:
        nestedSerializers - array of nested serializers to create the outer serializer with.
        Returns:
        an instance of the outer serializer.

      • writeOuterSnapshot

        protected void writeOuterSnapshot​(DataOutputView out)
                           throws IOException
        Writes the outer snapshot, i.e. any information beyond the nested serializers of the outer serializer.

        The base implementation of this methods writes nothing, i.e. it assumes that the outer serializer only has nested serializers and no extra information. Otherwise, if the outer serializer contains some extra information that needs to be persisted as part of the serializer snapshot, this must be overridden. Note that this method and the corresponding methods readOuterSnapshot(int, DataInputView, ClassLoader), resolveOuterSchemaCompatibility(TypeSerializer) needs to be implemented.

        Parameters:
        out - the DataOutputView to write the outer snapshot to.
        Throws:
        IOException

      • readOuterSnapshot

        protected void readOuterSnapshot​(int readOuterSnapshotVersion,
                                 DataInputView in,
                                 ClassLoader userCodeClassLoader)
                          throws IOException
        Reads the outer snapshot, i.e. any information beyond the nested serializers of the outer serializer.

        The base implementation of this methods reads nothing, i.e. it assumes that the outer serializer only has nested serializers and no extra information. Otherwise, if the outer serializer contains some extra information that has been persisted as part of the serializer snapshot, this must be overridden. Note that this method and the corresponding methods writeOuterSnapshot(DataOutputView), resolveOuterSchemaCompatibility(TypeSerializer) needs to be implemented.

        Parameters:
        readOuterSnapshotVersion - the read version of the outer snapshot.
        in - the DataInputView to read the outer snapshot from.
        userCodeClassLoader - the user code class loader.
        Throws:
        IOException

      • isOuterSnapshotCompatible

        @Deprecated
protected boolean isOuterSnapshotCompatible​(S newSerializer)
        Deprecated.
        this method is deprecated, and will be removed in the future. Please implement resolveOuterSchemaCompatibility(TypeSerializer) instead.
        Checks whether the outer snapshot is compatible with a given new serializer.

        The base implementation of this method just returns true, i.e. it assumes that the outer serializer only has nested serializers and no extra information, and therefore the result of the check must always be true. Otherwise, if the outer serializer contains some extra information that has been persisted as part of the serializer snapshot, this must be overridden. Note that this method and the corresponding methods writeOuterSnapshot(DataOutputView), readOuterSnapshot(int, DataInputView, ClassLoader) needs to be implemented.

        Parameters:
        newSerializer - the new serializer, which contains the new outer information to check against.
        Returns:
        a flag indicating whether or not the new serializer's outer information is compatible with the one written in this snapshot.