public class SnapshotObjectStoreUtils extends Object
Constructor and Description |
---|
SnapshotObjectStoreUtils(TierObjectStore store,
ThreadPoolExecutor pool,
RetryPolicy retryPolicy,
String keyPrefix,
org.apache.kafka.common.utils.Time time,
RestoreMetricsManager metricsManager) |
Modifier and Type | Method and Description |
---|---|
TierObjectStore.TierPartitionStateSnapshotMetadata |
decodeFtpsSnapshotPath(String path,
String topicName) |
static AbstractTierMetadata |
deserializeRecord(org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]> record) |
Map<TopicIdPartition,Future<Path>> |
downloadFtpsSnapshotsInParallel(Map<TopicIdPartition,TierObjectStore.TierPartitionStateSnapshotMetadata> ftpsSnapshots,
Path toLocalDir)
Initiates downloads of Tier Partition Snapshots (one for each TopicIdPartition) to the returned Paths.
|
List<Future<Path>> |
downloadTierTopicSnapshotsInParallel(List<TierObjectStore.TierTopicSnapshotMetadata> allObjectMetadata,
Path toLocalDir) |
ByteBuffer |
fetchRecoverSnapshot(TierPartitionForceRestore forceRestoreEvent) |
PointInTimeTierPartitionStateBuilder.FtpsSnapshotsMetadata |
locateFtpsSnapshotsByTimestamp(Map<org.apache.kafka.common.TopicPartition,PartitionRestoreContext> partitionRestoreContextMap)
Fetches the latest available FTPS filenames prior to the given timestamp from the given topic-partitions using
TierObjectStore.listObject and converts them to TierPartitionStateSnapshotMetadata objects,
storing the results in a mapping from topic-partition to FTPS metadata objects. |
List<TierObjectStore.TierTopicSnapshotMetadata> |
locateTierTopicSnapshotsByTimestamp(long startTimestamp,
OptionalLong endTimestamp)
Fetches all TTPS filenames from object store using
TierObjectStore.listObject , filters them to those that
are within the range from [startTimestamp, endTimestamp], and converts them to objects of type
TierTopicPartitionSnapshotMetadata . |
public SnapshotObjectStoreUtils(TierObjectStore store, ThreadPoolExecutor pool, RetryPolicy retryPolicy, String keyPrefix, org.apache.kafka.common.utils.Time time, RestoreMetricsManager metricsManager)
public Map<TopicIdPartition,Future<Path>> downloadFtpsSnapshotsInParallel(Map<TopicIdPartition,TierObjectStore.TierPartitionStateSnapshotMetadata> ftpsSnapshots, Path toLocalDir)
ftpsSnapshots
- The object metadata corresponding to each Tier Partition Snapshot that should be
downloadedtoLocalDir
- The directory into which the snapshots should be downloadedpublic List<Future<Path>> downloadTierTopicSnapshotsInParallel(List<TierObjectStore.TierTopicSnapshotMetadata> allObjectMetadata, Path toLocalDir)
public PointInTimeTierPartitionStateBuilder.FtpsSnapshotsMetadata locateFtpsSnapshotsByTimestamp(Map<org.apache.kafka.common.TopicPartition,PartitionRestoreContext> partitionRestoreContextMap)
TierObjectStore.listObject
and converts them to TierPartitionStateSnapshotMetadata
objects,
storing the results in a mapping from topic-partition to FTPS metadata objects.
These metadata objects can be used as inputs to TierObjectStore.getObject
.
partitionRestoreContextMap
- partition contexts for which FTPS snapshot filenames are to be retrievedFetchTierPartitionSnapshotMetadataResult
containing metadataMap
and
minLastMaterializedEventTs
. The metadataMap
is a mapping from topic-partition to corresponding
FTPS metadata that has the highest timestamp that is less than or equal to the given timestamp; if timestamp is
empty, maps to the latest available filename; if none found, maps to null. The minLastMaterializedEventTs
is the minimum value of the last materialized event timestamp encountered when collecting the
metadataMap
, or else null if all values in metadataMap
are null.public List<TierObjectStore.TierTopicSnapshotMetadata> locateTierTopicSnapshotsByTimestamp(long startTimestamp, OptionalLong endTimestamp)
TierObjectStore.listObject
, filters them to those that
are within the range from [startTimestamp, endTimestamp], and converts them to objects of type
TierTopicPartitionSnapshotMetadata
.
These metadata objects can be used as inputs to TierObjectStore.getObject
.
startTimestamp
- minimum snapshot timestamp of FTPS filesendTimestamp
- optional ending point-in-time timestamp of TTPS filesstartTimestamp
(up to endTimestamp
, if present).
An empty list indicates none could be found, likely meaning that input startTimestamp
is too low or
endTimestamp <= startTimestamp. Results are sorted in ascending order by (minTs, maxTs, leaderEpoch).public TierObjectStore.TierPartitionStateSnapshotMetadata decodeFtpsSnapshotPath(String path, String topicName)
public ByteBuffer fetchRecoverSnapshot(TierPartitionForceRestore forceRestoreEvent) throws InterruptedException
InterruptedException
public static AbstractTierMetadata deserializeRecord(org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]> record)