Package org.apache.flink.streaming.runtime.operators.sink.committables

Class CommittableCollector<CommT>

java.lang.Object

org.apache.flink.streaming.runtime.operators.sink.committables.CommittableCollector<CommT>

Type Parameters:

CommT - type of committable

@Internal
public class CommittableCollector<CommT>
extends Object

This class is responsible to book-keep the committing progress across checkpoints and subtasks. It handles the emission of committables and the CommittableSummary.

Constructor Summary

Constructors

Constructor	Description
CommittableCollector(int subtaskId, int numberOfSubtasks)

Method Summary

Modifier and Type	Method	Description
void	addMessage(CommittableMessage<CommT> message)	Adds a CommittableMessage to the collector to hold it until emission.
CommittableCollector<CommT>	copy()	Returns a new committable collector that deep copies all internals.
Collection<? extends CheckpointCommittableManager<CommT>>	getCheckpointCommittablesUpTo(long checkpointId)	Returns all CheckpointCommittableManager until the requested checkpoint id.
CommittableManager<CommT>	getEndOfInputCommittable()	Returns CheckpointCommittableManager that is currently hold by the collector and associated with the EOI checkpoint id.
int	getNumberOfSubtasks()	Returns number of subtasks.
int	getSubtaskId()	Returns subtask id.
boolean	isFinished()	Returns whether all CheckpointCommittableManager currently hold by the collector are either committed or failed.
void	merge(CommittableCollector<CommT> cc)	Merges all information from an external collector into this collector.
static <CommT> CommittableCollector<CommT>	of(org.apache.flink.api.common.functions.RuntimeContext context)	Creates a CommittableCollector based on the current runtime information.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CommittableCollector

public CommittableCollector(int subtaskId,
                            int numberOfSubtasks)

Method Detail

of

public static <CommT> CommittableCollector<CommT> of(org.apache.flink.api.common.functions.RuntimeContext context)

Creates a CommittableCollector based on the current runtime information. This method should be used for to instantiate a collector for all Sink V2.

Type Parameters:

CommT - type of the committable

Parameters:

context - holding runtime of information

Returns:

CommittableCollector

addMessage

public void addMessage(CommittableMessage<CommT> message)

Adds a CommittableMessage to the collector to hold it until emission.

Parameters:

message - either CommittableSummary or CommittableWithLineage

getCheckpointCommittablesUpTo

public Collection<? extends CheckpointCommittableManager<CommT>> getCheckpointCommittablesUpTo(long checkpointId)

Returns all CheckpointCommittableManager until the requested checkpoint id.

Parameters:

checkpointId - counter

Returns:

collection of CheckpointCommittableManager

getEndOfInputCommittable

@Nullable
public CommittableManager<CommT> getEndOfInputCommittable()

Returns CheckpointCommittableManager that is currently hold by the collector and associated with the EOI checkpoint id.

Returns:

CheckpointCommittableManager

isFinished

public boolean isFinished()

Returns whether all CheckpointCommittableManager currently hold by the collector are either committed or failed.

Returns:

state of the CheckpointCommittableManager

merge

public void merge(CommittableCollector<CommT> cc)

Merges all information from an external collector into this collector.

This method is important during recovery from existing state.

Parameters:

cc - other CommittableCollector

getNumberOfSubtasks

public int getNumberOfSubtasks()

Returns number of subtasks.

Returns:

number of subtasks

getSubtaskId

public int getSubtaskId()

Returns subtask id.

Returns:

subtask id.

copy

public CommittableCollector<CommT> copy()

Returns a new committable collector that deep copies all internals.

Returns:

CommittableCollector