Class IncrementalRemoteKeyedStateHandle
- java.lang.Object
-
- org.apache.flink.runtime.state.IncrementalRemoteKeyedStateHandle
-
- All Implemented Interfaces:
Serializable,CheckpointBoundKeyedStateHandle,CompositeStateHandle,IncrementalKeyedStateHandle,KeyedStateHandle,StateObject
public class IncrementalRemoteKeyedStateHandle extends Object implements IncrementalKeyedStateHandle
The handle to states of an incremental snapshot.The states contained in an incremental snapshot include:
- Created shared state which includes shared files produced since the last completed checkpoint. These files can be referenced by succeeding checkpoints if the checkpoint succeeds to complete.
- Referenced shared state which includes the shared files materialized in previous
checkpoints. Until we this is registered to a
SharedStateRegistry, all referenced shared state handles are only placeholders, so that we do not send state handles twice from which we know that they already exist on the checkpoint coordinator. - Private state which includes all other files, typically mutable, that cannot be shared by other checkpoints.
- Backend meta state which includes the information of existing states.
SharedStateRegistry, so that all placeholder state handles to previously existing state are replaced with the originals.IMPORTANT: This class currently overrides equals and hash code only for testing purposes. They should not be called from production code. This means this class is also not suited to serve as a key, e.g. in hash maps.
- See Also:
- Serialized Form
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from interface org.apache.flink.runtime.state.IncrementalKeyedStateHandle
IncrementalKeyedStateHandle.HandleAndLocalPath
-
-
Field Summary
Fields Modifier and Type Field Description static longUNKNOWN_CHECKPOINTED_SIZE
-
Constructor Summary
Constructors Modifier Constructor Description IncrementalRemoteKeyedStateHandle(UUID backendIdentifier, KeyGroupRange keyGroupRange, long checkpointId, List<IncrementalKeyedStateHandle.HandleAndLocalPath> sharedState, List<IncrementalKeyedStateHandle.HandleAndLocalPath> privateState, StreamStateHandle metaStateHandle)IncrementalRemoteKeyedStateHandle(UUID backendIdentifier, KeyGroupRange keyGroupRange, long checkpointId, List<IncrementalKeyedStateHandle.HandleAndLocalPath> sharedState, List<IncrementalKeyedStateHandle.HandleAndLocalPath> privateState, StreamStateHandle metaStateHandle, long persistedSizeOfThisCheckpoint)protectedIncrementalRemoteKeyedStateHandle(UUID backendIdentifier, KeyGroupRange keyGroupRange, long checkpointId, List<IncrementalKeyedStateHandle.HandleAndLocalPath> sharedState, List<IncrementalKeyedStateHandle.HandleAndLocalPath> privateState, StreamStateHandle metaStateHandle, long persistedSizeOfThisCheckpoint, StateHandleID stateHandleId)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description SharedStateRegistryKeycreateSharedStateRegistryKey(StreamStateHandle handle)Create a unique key based on physical id to register one of our shared state handles.voiddiscardState()Discards the state referred to and solemnly owned by this handle, to free up resources in the persistent storage.booleanequals(Object o)This method is should only be called in tests! This should never serve as key in a hash map.UUIDgetBackendIdentifier()Returns the identifier of the state backend from which this handle was created.longgetCheckpointedSize()Returns the persisted data size during checkpoint execution in bytes.longgetCheckpointId()Returns the ID of the checkpoint for which the handle was created or used.KeyedStateHandlegetIntersection(KeyGroupRange keyGroupRange)Returns a state over a range that is the intersection between this handle's key-group range and the provided key-group range.KeyGroupRangegetKeyGroupRange()Returns the range of the key groups contained in the state.StreamStateHandlegetMetaStateHandle()List<IncrementalKeyedStateHandle.HandleAndLocalPath>getPrivateState()List<IncrementalKeyedStateHandle.HandleAndLocalPath>getSharedState()List<IncrementalKeyedStateHandle.HandleAndLocalPath>getSharedStateHandles()Returns a list of all shared states and the corresponding localPath in the backend at the time this was created.SharedStateRegistrygetSharedStateRegistry()StateHandleIDgetStateHandleId()Returns a unique state handle id to distinguish with other keyed state handles.longgetStateSize()Returns the size of the state in bytes.inthashCode()This method should only be called in tests! This should never serve as key in a hash map.CheckpointBoundKeyedStateHandlerebound(long checkpointId)Returns a newCheckpointBoundKeyedStateHandleRebounding checkpoint id to a specific checkpoint id.voidregisterSharedStates(SharedStateRegistry stateRegistry, long checkpointID)Register both newly created and already referenced shared states in the givenSharedStateRegistry.static IncrementalRemoteKeyedStateHandlerestore(UUID backendIdentifier, KeyGroupRange keyGroupRange, long checkpointId, List<IncrementalKeyedStateHandle.HandleAndLocalPath> sharedState, List<IncrementalKeyedStateHandle.HandleAndLocalPath> privateState, StreamStateHandle metaStateHandle, long persistedSizeOfThisCheckpoint, StateHandleID stateHandleId)StringtoString()
-
-
-
Field Detail
-
UNKNOWN_CHECKPOINTED_SIZE
public static final long UNKNOWN_CHECKPOINTED_SIZE
- See Also:
- Constant Field Values
-
-
Constructor Detail
-
IncrementalRemoteKeyedStateHandle
@VisibleForTesting public IncrementalRemoteKeyedStateHandle(UUID backendIdentifier, KeyGroupRange keyGroupRange, long checkpointId, List<IncrementalKeyedStateHandle.HandleAndLocalPath> sharedState, List<IncrementalKeyedStateHandle.HandleAndLocalPath> privateState, StreamStateHandle metaStateHandle)
-
IncrementalRemoteKeyedStateHandle
public IncrementalRemoteKeyedStateHandle(UUID backendIdentifier, KeyGroupRange keyGroupRange, long checkpointId, List<IncrementalKeyedStateHandle.HandleAndLocalPath> sharedState, List<IncrementalKeyedStateHandle.HandleAndLocalPath> privateState, StreamStateHandle metaStateHandle, long persistedSizeOfThisCheckpoint)
-
IncrementalRemoteKeyedStateHandle
protected IncrementalRemoteKeyedStateHandle(UUID backendIdentifier, KeyGroupRange keyGroupRange, long checkpointId, List<IncrementalKeyedStateHandle.HandleAndLocalPath> sharedState, List<IncrementalKeyedStateHandle.HandleAndLocalPath> privateState, StreamStateHandle metaStateHandle, long persistedSizeOfThisCheckpoint, StateHandleID stateHandleId)
-
-
Method Detail
-
restore
public static IncrementalRemoteKeyedStateHandle restore(UUID backendIdentifier, KeyGroupRange keyGroupRange, long checkpointId, List<IncrementalKeyedStateHandle.HandleAndLocalPath> sharedState, List<IncrementalKeyedStateHandle.HandleAndLocalPath> privateState, StreamStateHandle metaStateHandle, long persistedSizeOfThisCheckpoint, StateHandleID stateHandleId)
-
getKeyGroupRange
public KeyGroupRange getKeyGroupRange()
Description copied from interface:KeyedStateHandleReturns the range of the key groups contained in the state.- Specified by:
getKeyGroupRangein interfaceKeyedStateHandle
-
getCheckpointId
public long getCheckpointId()
Description copied from interface:CheckpointBoundKeyedStateHandleReturns the ID of the checkpoint for which the handle was created or used.- Specified by:
getCheckpointIdin interfaceCheckpointBoundKeyedStateHandle
-
rebound
public CheckpointBoundKeyedStateHandle rebound(long checkpointId)
Description copied from interface:CheckpointBoundKeyedStateHandleReturns a newCheckpointBoundKeyedStateHandleRebounding checkpoint id to a specific checkpoint id.- Specified by:
reboundin interfaceCheckpointBoundKeyedStateHandle- Parameters:
checkpointId- rebounded checkpoint id.
-
getSharedState
public List<IncrementalKeyedStateHandle.HandleAndLocalPath> getSharedState()
-
getPrivateState
public List<IncrementalKeyedStateHandle.HandleAndLocalPath> getPrivateState()
-
getMetaStateHandle
public StreamStateHandle getMetaStateHandle()
-
getBackendIdentifier
@Nonnull public UUID getBackendIdentifier()
Description copied from interface:IncrementalKeyedStateHandleReturns the identifier of the state backend from which this handle was created.- Specified by:
getBackendIdentifierin interfaceIncrementalKeyedStateHandle
-
getSharedStateHandles
@Nonnull public List<IncrementalKeyedStateHandle.HandleAndLocalPath> getSharedStateHandles()
Description copied from interface:IncrementalKeyedStateHandleReturns a list of all shared states and the corresponding localPath in the backend at the time this was created.- Specified by:
getSharedStateHandlesin interfaceIncrementalKeyedStateHandle
-
getSharedStateRegistry
public SharedStateRegistry getSharedStateRegistry()
-
getIntersection
public KeyedStateHandle getIntersection(KeyGroupRange keyGroupRange)
Description copied from interface:KeyedStateHandleReturns a state over a range that is the intersection between this handle's key-group range and the provided key-group range.- Specified by:
getIntersectionin interfaceKeyedStateHandle- Parameters:
keyGroupRange- The key group range to intersect with, will return null if the intersection of this handle's key-group and the provided key-group is empty.
-
getStateHandleId
public StateHandleID getStateHandleId()
Description copied from interface:KeyedStateHandleReturns a unique state handle id to distinguish with other keyed state handles.If this state handle would be used as materialized part of ChangelogStateBackendHandle, it should persist this state handle id when serializing of checkpoint and deserialize it back to ensure the state handle id is constant.- Specified by:
getStateHandleIdin interfaceKeyedStateHandle- Returns:
- A unique state handle id.
-
discardState
public void discardState() throws ExceptionDescription copied from interface:StateObjectDiscards the state referred to and solemnly owned by this handle, to free up resources in the persistent storage. This method is called when the state represented by this object will not be used anymore.- Specified by:
discardStatein interfaceStateObject- Throws:
Exception
-
getStateSize
public long getStateSize()
Description copied from interface:StateObjectReturns the size of the state in bytes. If the size is not known, this method should return0.The values produced by this method are only used for informational purposes and for metrics/monitoring. If this method returns wrong values, the checkpoints and recovery will still behave correctly. However, efficiency may be impacted (wrong space pre-allocation) and functionality that depends on metrics (like monitoring) will be impacted.
Note for implementors: This method should not perform any I/O operations while obtaining the state size (hence it does not declare throwing an
IOException). Instead, the state size should be stored in the state object, or should be computable from the state stored in this object. The reason is that this method is called frequently by several parts of the checkpointing and issuing I/O requests from this method accumulates a heavy I/O load on the storage system at higher scale.- Specified by:
getStateSizein interfaceStateObject- Returns:
- Size of the state in bytes.
-
getCheckpointedSize
public long getCheckpointedSize()
Description copied from interface:CompositeStateHandleReturns the persisted data size during checkpoint execution in bytes. If incremental checkpoint is enabled, this value represents the incremental persisted data size, and usually smaller thanStateObject.getStateSize(). If the size is unknown, this method would return same result asStateObject.getStateSize().- Specified by:
getCheckpointedSizein interfaceCompositeStateHandle- Returns:
- The persisted data size during checkpoint execution in bytes.
-
registerSharedStates
public void registerSharedStates(SharedStateRegistry stateRegistry, long checkpointID)
Description copied from interface:CompositeStateHandleRegister both newly created and already referenced shared states in the givenSharedStateRegistry. This method is called when the checkpoint successfully completes or is recovered from failures.After this is completed, newly created shared state is considered as published is no longer owned by this handle. This means that it should no longer be deleted as part of calls to
StateObject.discardState(). Instead,StateObject.discardState()will trigger an unregistration from the registry.- Specified by:
registerSharedStatesin interfaceCompositeStateHandle- Parameters:
stateRegistry- The registry where shared states are registered.
-
createSharedStateRegistryKey
@VisibleForTesting public SharedStateRegistryKey createSharedStateRegistryKey(StreamStateHandle handle)
Create a unique key based on physical id to register one of our shared state handles.
-
equals
@VisibleForTesting public boolean equals(Object o)
This method is should only be called in tests! This should never serve as key in a hash map.
-
hashCode
@VisibleForTesting public int hashCode()
This method should only be called in tests! This should never serve as key in a hash map.
-
-