Class RocksDBSnapshotStrategyBase.PreviousSnapshot

  • Enclosing class:
    RocksDBSnapshotStrategyBase<K,​R extends org.apache.flink.runtime.state.SnapshotResources>

    protected static class RocksDBSnapshotStrategyBase.PreviousSnapshot
    extends Object
    Previous snapshot with uploaded sst files.
    • Constructor Detail

      • PreviousSnapshot

        protected PreviousSnapshot​(@Nullable
                                   SortedMap<Long,​Collection<org.apache.flink.runtime.state.IncrementalKeyedStateHandle.HandleAndLocalPath>> currentUploadedSstFiles,
                                   long lastCompletedCheckpoint)
        Constructor of PreviousSnapshot. Giving a map of uploaded sst files in previous checkpoints, prune the sst files which have been re-uploaded in the following checkpoints. The prune logic is used to resolve the mismatch between TM and JM due to notification delay. Following steps for example:
        • 1) checkpoint 1 uses file 00001.SST uploaded as xxx.sst.
        • 2) checkpoint 2 uses the same file 00001.SST but re-uploads it as yyy.sst because CP 1 wasn't yet confirmed.
        • 3) TM get a confirmation of checkpoint 1.
        • 4) JM completes checkpoint 2 and subsumes checkpoint 1 - removing xxx.sst.
        • 5) checkpoint 3 tries to re-use file 00001.SST uploaded as xxx.sst in checkpoint 1, but it was deleted in (4) by JM.
        Parameters:
        currentUploadedSstFiles - the sst files uploaded in previous checkpoints.
        lastCompletedCheckpoint - the last completed checkpoint id.
    • Method Detail

      • getUploaded

        protected Optional<org.apache.flink.runtime.state.StreamStateHandle> getUploaded​(String filename)