public interface TierPartitionState
Modifier and Type | Interface and Description |
---|---|
static class |
TierPartitionState.AppendResult
The result of an attempt to append a tier metadata entry.
|
static class |
TierPartitionState.RecoveryOperation
Various operations required while recovering a tiered partition from data loss
|
static class |
TierPartitionState.RestoreResult
The result of an attempt to restore a TierPartitionState via a PartitionForceRestore event
|
Modifier and Type | Method and Description |
---|---|
TierPartitionState.AppendResult |
append(AbstractTierMetadata tierMetadata,
OffsetAndEpoch sourceOffsetAndEpoch)
Appends abstract metadata to the tier partition.
|
void |
beginCatchup()
Begin catchup phase for tier partition state.
|
void |
close()
Close TierPartition, flushing to disk.
|
void |
closeHandlers() |
long |
committedEndOffset()
Return the end offset spanned by the TierPartitionState that has been committed to disk.
|
void |
delete()
Delete this TierPartitionState from local storage.
|
File |
dir()
The directory where the TierPartition is stored on disk.
|
long |
endOffset()
Return the uncommitted end offset spanned by the TierPartitionState.
|
Collection<kafka.log.TierLogSegment> |
fencedSegments() |
boolean |
flush()
Flush data contained in this TierPartitionState to disk.
|
kafka.common.FollowerRestorePoint |
followerRestorePoint(long targetStartOffset)
Get the TierObjectMetadata for the immediate previous segment (even beyond retention) to the target startOffset.
|
boolean |
isTieringEnabled() |
OffsetAndEpoch |
lastLocalMaterializedSrcOffsetAndEpoch()
Return the last materialized source offset and epoch
|
long |
materializationLag()
Return the current tier materialization lag.
|
CompletableFuture<Optional<kafka.log.TierLogSegment>> |
materializeUptoLeaderEpoch(int targetEpoch)
Sets up a listener for this tier partition state.
|
CompletableFuture<kafka.log.TierLogSegment> |
materializeUptoObjectIdAndRestoreEpoch(long upperBoundEndOffset,
UUID targetObjectId,
int targetRestoreEpoch)
Sets up a listener for this tier partition stae for use by the replica fetcher.
|
CompletableFuture<kafka.log.TierLogSegment> |
materializeUptoOffset(long targetOffset)
Sets up a listener for this tier partition state for use by the replica fetcher.
|
boolean |
maybeOpenChannelOnOffsetTieredException()
Called when a replica receives OFFSET_TIERED exception while replicating from leader.
|
boolean |
mayContainTieredData()
Indicates if a partition may have some tiered data, essentially whether tiered storage is currently enabled or was enabled
earlier for this partition.
|
Optional<kafka.log.TierLogSegment> |
metadata(long targetOffset)
Lookup the TierObjectMetadata which will contain data for a target offset.
|
int |
numSegments()
Scan the ObjectMetadata (segment) entries in this tier partition, and return the count.
|
int |
numSegments(long from,
long to)
Count of the number of segments lying within a range
|
void |
onCatchUpComplete()
Mark catchup completed for tier partition state.
|
Optional<SegmentState> |
previousMetadataBeforeOffset(long targetStartOffset)
Get the in-memory metadata for the immediate previous segment (even beyond retention) to target startOffset
|
TierPartitionState.RestoreResult |
processRestoreEvents(AbstractTierMetadata metadata,
Optional<ByteBuffer> targetStateOpt,
TierPartitionStatus targetStatus,
OffsetAndEpoch sourceOffsetAndEpoch)
Processes one of the recovery events TierPartitionForceRestore or TierPartitionUnfreezeLogStartOffset.
|
List<SegmentState> |
segmentInMemoryMetadataRange(long from,
long to)
List of all the segment (minimal) metadata whose startOffset is in the range (even if they are beyond retention).
|
Iterator<kafka.log.TierLogSegment> |
segments()
Get an iterator for all readable tiered segments.
|
Iterator<kafka.log.TierLogSegment> |
segments(long from,
long to)
Get an iterator for all readable tiered segments in a given range.
|
void |
setTieredPartitionRecoveryWorkflowCb(Consumer<TierPartitionState.RecoveryOperation> cb)
Sets the callback function used by tier partition state to communicate to the MergedLog layer during recovery of
a tiered partition.
|
void |
setTieringDisabled()
Reset the tier(ing) enabled flag at TierPartitionState.
|
boolean |
setTieringEnabled()
Called when tiering is enabled for this tier topic partition.
|
boolean |
setTopicId(UUID topicId)
Sets the TopicIdPartition for this TierPartitionState.
|
Optional<Long> |
startOffset()
Determine start offset spanned by the TierPartitionState.
|
TierPartitionStatus |
status()
Return the current status of the TierPartitionState.
|
int |
tierEpoch()
Return the current tierEpoch.
|
Optional<TopicIdPartition> |
topicIdPartition()
Optional TopicIdPartition corresponding to this TierPartition
If one has not been set, returns empty
|
org.apache.kafka.common.TopicPartition |
topicPartition()
The topic-partition corresponding to this TierPartition.
|
long |
totalSize()
Sum the size of all segment spanned by this TierPartitionState.
|
CompletableFuture<Boolean> |
trackMetadataInitialization(int targetLeaderEpoch)
Tracks the completion of tier metadata initialization.
|
void |
updateDir(File dir)
Update the directory reference for the log and indices in this segment.
|
boolean |
validateChecksum()
Validates the integrity by computing the checksum and validating the checksum.
|
void setTieredPartitionRecoveryWorkflowCb(Consumer<TierPartitionState.RecoveryOperation> cb)
cb
- callback function that takes a TierPartitionState.RecoveryOperation as a parameterorg.apache.kafka.common.TopicPartition topicPartition()
Optional<TopicIdPartition> topicIdPartition()
File dir()
Optional<Long> startOffset() throws IOException
IOException
boolean setTopicId(UUID topicId) throws IOException
topicId
- The topic id to assignIOException
IllegalStateException
- on topic id mismatch, if it had already been set beforelong committedEndOffset() throws IOException
IOException
long endOffset() throws IOException
IOException
OffsetAndEpoch lastLocalMaterializedSrcOffsetAndEpoch()
IOException
int numSegments()
int numSegments(long from, long to)
from
- Start of the range, include segment which contains "from" (inclusive)to
- End of the range, upper bound exclusive offset to include or the end of the log if "to" is past the endIterator<kafka.log.TierLogSegment> segments()
Iterator<kafka.log.TierLogSegment> segments(long from, long to)
from
- Start of the range, include segment which contains "from" (inclusive)to
- End of the range, upper bound exclusive offset to include or the end of the log if "to" is past the endList<SegmentState> segmentInMemoryMetadataRange(long from, long to)
from
- Inclusive startOffset positionto
- Inclusive endOffset positionOptional<SegmentState> previousMetadataBeforeOffset(long targetStartOffset)
targetStartOffset
- target startOffsetkafka.common.FollowerRestorePoint followerRestorePoint(long targetStartOffset)
targetStartOffset
- target startOffsetOptional<kafka.log.TierLogSegment> metadata(long targetOffset) throws IOException
targetOffset
- the target offset to lookup the overlapping or next metadata for.IOException
- if disk error encounteredCollection<kafka.log.TierLogSegment> fencedSegments()
TierPartitionState.AppendResult append(AbstractTierMetadata tierMetadata, OffsetAndEpoch sourceOffsetAndEpoch)
tierMetadata
- AbstractTierMetadata entry to be appended to the tier partition log.sourceOffsetAndEpoch
- Offset and epoch corresponding to this metadata entryTierPartitionState.RestoreResult processRestoreEvents(AbstractTierMetadata metadata, Optional<ByteBuffer> targetStateOpt, TierPartitionStatus targetStatus, OffsetAndEpoch sourceOffsetAndEpoch)
metadata
- the metadata event to be processedtargetStateOpt
- ByteBuffer containing the contents of the TierPartitionState to restoretargetStatus
- the status that the TierPartitionState will be transitioned to upon
successful restorationsourceOffsetAndEpoch
- Offset and epoch corresponding to this metadata entrylong totalSize() throws IOException
IOException
int tierEpoch() throws IOException
IOException
boolean isTieringEnabled()
boolean mayContainTieredData()
boolean maybeOpenChannelOnOffsetTieredException() throws IOException
IOException
boolean setTieringEnabled() throws IOException
IOException
void setTieringDisabled()
boolean flush() throws IOException
IOException
- if the flush operation failedboolean validateChecksum() throws IOException, InstantiationException, IllegalAccessException
IOException
InstantiationException
IllegalAccessException
void beginCatchup()
void onCatchUpComplete()
CompletableFuture<kafka.log.TierLogSegment> materializeUptoOffset(long targetOffset) throws IOException
targetOffset
- the offset awaiting materializationIOException
CompletableFuture<kafka.log.TierLogSegment> materializeUptoObjectIdAndRestoreEpoch(long upperBoundEndOffset, UUID targetObjectId, int targetRestoreEpoch) throws IOException
IOException
CompletableFuture<Optional<kafka.log.TierLogSegment>> materializeUptoLeaderEpoch(int targetEpoch) throws IOException
targetEpoch
- the leader epoch awaiting materializationIOException
CompletableFuture<Boolean> trackMetadataInitialization(int targetLeaderEpoch) throws IOException
targetLeaderEpoch
- completes the initialization once the tier metadata state reaches equal or greater than targetEpoch.IOException
TierPartitionStatus status()
long materializationLag()
void updateDir(File dir)
dir
- The new directoryvoid delete() throws IOException
IOException
void close() throws IOException
IOException
void closeHandlers() throws IOException
IOException