Package org.apache.flink.table.runtime.operators.join.deltajoin

Class StreamingDeltaJoinOperator

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.flink.table.runtime.operators.TableAbstractCoUdfStreamOperator<org.apache.flink.table.data.RowData,AsyncDeltaJoinRunner,AsyncDeltaJoinRunner>

org.apache.flink.table.runtime.operators.join.deltajoin.StreamingDeltaJoinOperator

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.BoundedMultiInput, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.KeyContextHandler, org.apache.flink.streaming.api.operators.OutputTypeConfigurable<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.streaming.api.operators.TwoInputStreamOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.YieldingOperator<org.apache.flink.table.data.RowData>

public class StreamingDeltaJoinOperator
extends TableAbstractCoUdfStreamOperator<org.apache.flink.table.data.RowData,AsyncDeltaJoinRunner,AsyncDeltaJoinRunner>
implements org.apache.flink.streaming.api.operators.TwoInputStreamOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.BoundedMultiInput

This operator supports delta join in table layer.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	StreamingDeltaJoinOperator.InputIndexAwareStreamRecordQueueEntry	A StreamElementQueueEntry with the input index.

Field Summary

Fields inherited from class org.apache.flink.table.runtime.operators.TableAbstractCoUdfStreamOperator

leftTriggeredUserFunction, rightTriggeredUserFunction

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

combinedWatermark, config, lastRecordAttributes1, lastRecordAttributes2, latencyStats, metrics, output, processingTimeService, stateHandler, stateKeySelector1, stateKeySelector2, timeServiceManager

Constructor Summary

Constructors

Constructor

Description

StreamingDeltaJoinOperator(AsyncDeltaJoinRunner rightLookupTableAsyncFunction, AsyncDeltaJoinRunner leftLookupTableAsyncFunction, RowDataKeySelector leftJoinKeySelector, RowDataKeySelector rightJoinKeySelector, long timeout, int capacity, org.apache.flink.streaming.runtime.tasks.ProcessingTimeService processingTimeService, org.apache.flink.api.common.operators.MailboxExecutor mailboxExecutor, org.apache.flink.table.types.logical.RowType leftStreamType, org.apache.flink.table.types.logical.RowType rightStreamType)

Method Summary

Modifier and Type	Method	Description
void	close()
void	emitWatermark(org.apache.flink.streaming.api.watermark.Watermark mark)
void	endInput(int inputId)
TableAsyncExecutionController<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>	getAsyncExecutionController()
void	initializeState(org.apache.flink.runtime.state.StateInitializationContext context)
void	invoke(AecRecord<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData> element)
void	open()
void	processElement1(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)
void	processElement2(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)
void	processWatermark(org.apache.flink.streaming.api.watermark.Watermark mark)
void	setAsyncExecutionController(TableAsyncExecutionController<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData> asyncExecutionController)
void	setup(org.apache.flink.streaming.runtime.tasks.StreamTask<?,?> containingTask, org.apache.flink.streaming.api.graph.StreamConfig config, org.apache.flink.streaming.api.operators.Output<org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData>> output)
org.apache.flink.streaming.api.operators.OperatorSnapshotFutures	snapshotState(long checkpointId, long timestamp, org.apache.flink.runtime.checkpoint.CheckpointOptions checkpointOptions, org.apache.flink.runtime.state.CheckpointStreamFactory factory)
void	waitInFlightInputsFinished()

Methods inherited from class org.apache.flink.table.runtime.operators.TableAbstractCoUdfStreamOperator

finish, getLeftTriggeredUserFunction, getRightTriggeredUserFunction, notifyCheckpointAborted, notifyCheckpointComplete, setOutputType, snapshotState

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

beforeInitializeStateHandler, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getStateKeySelector1, getStateKeySelector2, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, isAsyncKeyOrderedProcessingEnabled, isUsingCustomRawKeyedState, prepareSnapshotPreBarrier, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes, processRecordAttributes1, processRecordAttributes2, processWatermark, processWatermark1, processWatermark1, processWatermark2, processWatermark2, processWatermarkStatus, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setMailboxExecutor, setProcessingTimeService, useSplittableTimers

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted, notifyCheckpointComplete

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

hasKeyContext

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

finish, getMetricGroup, getOperatorAttributes, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2

Methods inherited from interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator

processLatencyMarker1, processLatencyMarker2, processRecordAttributes1, processRecordAttributes2, processWatermark1, processWatermark1, processWatermark2, processWatermark2, processWatermarkStatus1, processWatermarkStatus2

Constructor Detail

StreamingDeltaJoinOperator

public StreamingDeltaJoinOperator(AsyncDeltaJoinRunner rightLookupTableAsyncFunction,
                                  AsyncDeltaJoinRunner leftLookupTableAsyncFunction,
                                  RowDataKeySelector leftJoinKeySelector,
                                  RowDataKeySelector rightJoinKeySelector,
                                  long timeout,
                                  int capacity,
                                  org.apache.flink.streaming.runtime.tasks.ProcessingTimeService processingTimeService,
                                  org.apache.flink.api.common.operators.MailboxExecutor mailboxExecutor,
                                  org.apache.flink.table.types.logical.RowType leftStreamType,
                                  org.apache.flink.table.types.logical.RowType rightStreamType)

Method Detail

setup

public void setup(org.apache.flink.streaming.runtime.tasks.StreamTask<?,?> containingTask,
                  org.apache.flink.streaming.api.graph.StreamConfig config,
                  org.apache.flink.streaming.api.operators.Output<org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData>> output)

Overrides:

setup in class TableAbstractCoUdfStreamOperator<org.apache.flink.table.data.RowData,AsyncDeltaJoinRunner,AsyncDeltaJoinRunner>

initializeState

public void initializeState(org.apache.flink.runtime.state.StateInitializationContext context)
                     throws Exception

Specified by:

initializeState in interface org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator

Overrides:

initializeState in class TableAbstractCoUdfStreamOperator<org.apache.flink.table.data.RowData,AsyncDeltaJoinRunner,AsyncDeltaJoinRunner>

Throws:

Exception

processWatermark

public void processWatermark(org.apache.flink.streaming.api.watermark.Watermark mark)
                      throws Exception

Overrides:

processWatermark in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>

Throws:

Exception

open

public void open()
          throws Exception

Specified by:

open in interface org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>

Overrides:

open in class TableAbstractCoUdfStreamOperator<org.apache.flink.table.data.RowData,AsyncDeltaJoinRunner,AsyncDeltaJoinRunner>

Throws:

Exception

processElement1

public void processElement1(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)
                     throws Exception

Specified by:

processElement1 in interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>

Throws:

Exception

processElement2

public void processElement2(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)
                     throws Exception

Specified by:

processElement2 in interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>

Throws:

Exception

emitWatermark

public void emitWatermark(org.apache.flink.streaming.api.watermark.Watermark mark)
                   throws Exception

Throws:

Exception

invoke

public void invoke(AecRecord<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData> element)
            throws Exception

Throws:

Exception

snapshotState

public org.apache.flink.streaming.api.operators.OperatorSnapshotFutures snapshotState(long checkpointId,
                                                                                      long timestamp,
                                                                                      org.apache.flink.runtime.checkpoint.CheckpointOptions checkpointOptions,
                                                                                      org.apache.flink.runtime.state.CheckpointStreamFactory factory)
                                                                               throws Exception

Specified by:

snapshotState in interface org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>

Overrides:

snapshotState in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>

Throws:

Exception

endInput

public void endInput(int inputId)
              throws Exception

Specified by:

endInput in interface org.apache.flink.streaming.api.operators.BoundedMultiInput

Throws:

Exception

close

public void close()
           throws Exception

Specified by:

close in interface org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>

Overrides:

close in class TableAbstractCoUdfStreamOperator<org.apache.flink.table.data.RowData,AsyncDeltaJoinRunner,AsyncDeltaJoinRunner>

Throws:

Exception

waitInFlightInputsFinished

public void waitInFlightInputsFinished()
                                throws InterruptedException

Throws:

InterruptedException

getAsyncExecutionController

@VisibleForTesting
public TableAsyncExecutionController<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData> getAsyncExecutionController()

setAsyncExecutionController

@VisibleForTesting
public void setAsyncExecutionController(TableAsyncExecutionController<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData> asyncExecutionController)