Package org.apache.flink.table.runtime.operators.join.lookup.keyordered

Class TableAsyncExecutionController<IN,OUT,KEY>

java.lang.Object

org.apache.flink.table.runtime.operators.join.lookup.keyordered.TableAsyncExecutionController<IN,OUT,KEY>

Type Parameters:

IN - Input type for the controller.

OUT - Output type for the controller.

KEY - The key type for the controller.

public class TableAsyncExecutionController<IN,OUT,KEY>
extends Object

The TableAsyncExecutionController is used to keep key ordered process mode for async operator. It allows for out of order processing on different keys and sequential processing of StreamElement on the same key.

Existing AsyncExecutionController is tightly coupled with the concept of state and this is why we add TableAsyncExecutionController.

TODO: Refactor this for less deduplication in the FLINK-37921.

Constructor Summary

Constructors

Constructor

Description

TableAsyncExecutionController(org.apache.flink.util.function.ThrowingConsumer<AecRecord<IN,OUT>,Exception> asyncInvoke, org.apache.flink.util.function.ThrowingConsumer<org.apache.flink.streaming.api.watermark.Watermark,Exception> emitWatermark, Consumer<org.apache.flink.streaming.api.operators.async.queue.StreamElementQueueEntry<OUT>> emitResult, Function<org.apache.flink.streaming.api.operators.async.queue.StreamElementQueueEntry<OUT>,Integer> inferDrivenInputIndex, org.apache.flink.util.function.BiFunctionWithException<org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN>,Integer,KEY,Exception> inferBlockingKey)

Method Summary

Modifier and Type	Method	Description
void	close()
void	completeRecord(org.apache.flink.streaming.api.operators.async.queue.StreamElementQueueEntry<OUT> resultFuture, AecRecord<IN,OUT> aecRecord)	Used for an element to be completed.
Epoch<OUT>	getActiveEpoch()
org.apache.flink.util.function.ThrowingConsumer<AecRecord<IN,OUT>,Exception>	getAsyncInvoke()
int	getBlockingSize()
Consumer<org.apache.flink.streaming.api.operators.async.queue.StreamElementQueueEntry<OUT>>	getEmitResult()
org.apache.flink.util.function.ThrowingConsumer<org.apache.flink.streaming.api.watermark.Watermark,Exception>	getEmitWatermark()
EpochManager<OUT>	getEpochManager()
int	getFinishSize()
org.apache.flink.util.function.BiFunctionWithException<org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN>,Integer,KEY,Exception>	getInferBlockingKey()
Function<org.apache.flink.streaming.api.operators.async.queue.StreamElementQueueEntry<OUT>,Integer>	getInferDrivenInputIndex()
int	getInFlightSize()
RecordsBuffer<AecRecord<IN,OUT>,KEY>	getRecordsBuffer()
void	outputRecord(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN> record, Epoch<OUT> epoch, int inputIndex)
Map<KEY,Deque<AecRecord<IN,OUT>>>	pendingElements()
void	recovery(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN> record, org.apache.flink.streaming.api.watermark.Watermark watermark, int inputIndex)
void	registerMetrics(org.apache.flink.metrics.MetricGroup metricGroup)
void	submitRecord(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN> record, Epoch<OUT> epoch, int inputIndex)
void	submitWatermark(org.apache.flink.streaming.api.watermark.Watermark watermark)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TableAsyncExecutionController

public TableAsyncExecutionController(org.apache.flink.util.function.ThrowingConsumer<AecRecord<IN,OUT>,Exception> asyncInvoke,
                                     org.apache.flink.util.function.ThrowingConsumer<org.apache.flink.streaming.api.watermark.Watermark,Exception> emitWatermark,
                                     Consumer<org.apache.flink.streaming.api.operators.async.queue.StreamElementQueueEntry<OUT>> emitResult,
                                     Function<org.apache.flink.streaming.api.operators.async.queue.StreamElementQueueEntry<OUT>,Integer> inferDrivenInputIndex,
                                     org.apache.flink.util.function.BiFunctionWithException<org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN>,Integer,KEY,Exception> inferBlockingKey)

Method Detail

registerMetrics

public void registerMetrics(org.apache.flink.metrics.MetricGroup metricGroup)

completeRecord

public void completeRecord(org.apache.flink.streaming.api.operators.async.queue.StreamElementQueueEntry<OUT> resultFuture,
                           AecRecord<IN,OUT> aecRecord)
                    throws Exception

Used for an element to be completed. This is used to release the occupied key and notify the epoch of the element to collect the result future.

Throws:

Exception

recovery

public void recovery(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN> record,
                     org.apache.flink.streaming.api.watermark.Watermark watermark,
                     int inputIndex)
              throws Exception

Throws:

Exception

submitRecord

public void submitRecord(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN> record,
                         @Nullable
                         Epoch<OUT> epoch,
                         int inputIndex)
                  throws Exception

Throws:

Exception

submitWatermark

public void submitWatermark(org.apache.flink.streaming.api.watermark.Watermark watermark)

pendingElements

public Map<KEY,Deque<AecRecord<IN,OUT>>> pendingElements()

outputRecord

public void outputRecord(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN> record,
                         Epoch<OUT> epoch,
                         int inputIndex)

close

public void close()

getActiveEpoch

@VisibleForTesting
public Epoch<OUT> getActiveEpoch()

getEpochManager

@VisibleForTesting
public EpochManager<OUT> getEpochManager()

getBlockingSize

@VisibleForTesting
public int getBlockingSize()

getFinishSize

@VisibleForTesting
public int getFinishSize()

getInFlightSize

@VisibleForTesting
public int getInFlightSize()

getRecordsBuffer

@VisibleForTesting
public RecordsBuffer<AecRecord<IN,OUT>,KEY> getRecordsBuffer()

getAsyncInvoke

@VisibleForTesting
public org.apache.flink.util.function.ThrowingConsumer<AecRecord<IN,OUT>,Exception> getAsyncInvoke()

getEmitWatermark

@VisibleForTesting
public org.apache.flink.util.function.ThrowingConsumer<org.apache.flink.streaming.api.watermark.Watermark,Exception> getEmitWatermark()

getEmitResult

@VisibleForTesting
public Consumer<org.apache.flink.streaming.api.operators.async.queue.StreamElementQueueEntry<OUT>> getEmitResult()

getInferDrivenInputIndex

@VisibleForTesting
public Function<org.apache.flink.streaming.api.operators.async.queue.StreamElementQueueEntry<OUT>,Integer> getInferDrivenInputIndex()

getInferBlockingKey

@VisibleForTesting
public org.apache.flink.util.function.BiFunctionWithException<org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN>,Integer,KEY,Exception> getInferBlockingKey()