Package org.apache.flink.streaming.api.operators.python.embedded

Class AbstractTwoInputEmbeddedPythonFunctionOperator<IN1,IN2,OUT>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedDataStreamPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.embedded.AbstractTwoInputEmbeddedPythonFunctionOperator<IN1,IN2,OUT>

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.api.java.typeutils.ResultTypeQueryable<OUT>, org.apache.flink.streaming.api.operators.BoundedMultiInput, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.KeyContextHandler, DataStreamPythonFunctionOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.streaming.api.operators.TwoInputStreamOperator<IN1,IN2,OUT>, org.apache.flink.streaming.api.operators.YieldingOperator<OUT>

Direct Known Subclasses:

EmbeddedPythonCoProcessOperator, EmbeddedPythonKeyedCoProcessOperator

@Internal
public abstract class AbstractTwoInputEmbeddedPythonFunctionOperator<IN1,IN2,OUT>
extends AbstractEmbeddedDataStreamPythonFunctionOperator<OUT>
implements org.apache.flink.streaming.api.operators.TwoInputStreamOperator<IN1,IN2,OUT>, org.apache.flink.streaming.api.operators.BoundedMultiInput

AbstractTwoInputEmbeddedPythonFunctionOperator is responsible for run Python DataStream operators with two input user defined python function in Embedded Python environment.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field	Description
protected long	timestamp

Fields inherited from class org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedDataStreamPythonFunctionOperator

collector, hasSideOutput, outputTypeInfo, sideOutputContext

Fields inherited from class org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedPythonFunctionOperator

interpreter

Fields inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

bundleFinishedCallback, config, elementCount, lastFinishBundleTime, maxBundleSize, systemEnvEnabled

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

combinedWatermark, lastRecordAttributes1, lastRecordAttributes2, latencyStats, LOG, metrics, output, processingTimeService, stateHandler, stateKeySelector1, stateKeySelector2, timeServiceManager

Constructor Summary

Constructors

Constructor	Description
AbstractTwoInputEmbeddedPythonFunctionOperator(org.apache.flink.configuration.Configuration config, DataStreamPythonFunctionInfo pythonFunctionInfo, org.apache.flink.api.common.typeinfo.TypeInformation<IN1> inputTypeInfo1, org.apache.flink.api.common.typeinfo.TypeInformation<IN2> inputTypeInfo2, org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outputTypeInfo)

Method Summary

Modifier and Type	Method	Description
void	close()
abstract List<FlinkFnApi.UserDefinedDataStreamFunction>	createUserDefinedFunctionsProto()	Gets the proto representation of the Python user-defined functions to be executed.
void	endInput(int inputId)
abstract Object	getFunctionContext()	Gets The function context.
abstract Object	getTimerContext()	Gets The Timer context.
void	open()
void	openPythonInterpreter()	Setup method for Python Interpreter.
void	processElement1(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN1> element)
void	processElement2(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN2> element)

Methods inherited from class org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedDataStreamPythonFunctionOperator

addSideOutputTags, getJobParameters, getProducedType, getPythonFunctionInfo, getSideOutputTags, setNumPartitions

Methods inherited from class org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedPythonFunctionOperator

createPythonEnvironmentManager, invokeFinishBundle

Methods inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

checkInvokeFinishBundleByCount, drainUnregisteredTimers, finish, getConfiguration, getFlinkMetricContainer, isBundleFinished, prepareSnapshotPreBarrier, processWatermark, setCurrentKey

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

beforeInitializeStateHandler, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getStateKeySelector1, getStateKeySelector2, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isAsyncStateProcessingEnabled, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes, processRecordAttributes1, processRecordAttributes2, processWatermark, processWatermark1, processWatermark1, processWatermark2, processWatermark2, processWatermarkStatus, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setKeyContextElement1, setKeyContextElement2, setMailboxExecutor, setProcessingTimeService, setup, snapshotState, snapshotState, useSplittableTimers

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted, notifyCheckpointComplete

Methods inherited from interface org.apache.flink.streaming.api.operators.python.DataStreamPythonFunctionOperator

copy

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

hasKeyContext

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

finish, getMetricGroup, getOperatorAttributes, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState

Methods inherited from interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator

processLatencyMarker1, processLatencyMarker2, processRecordAttributes1, processRecordAttributes2, processWatermark1, processWatermark1, processWatermark2, processWatermark2, processWatermarkStatus1, processWatermarkStatus2

Field Detail

timestamp

protected transient long timestamp

Constructor Detail

AbstractTwoInputEmbeddedPythonFunctionOperator

public AbstractTwoInputEmbeddedPythonFunctionOperator(org.apache.flink.configuration.Configuration config,
                                                      DataStreamPythonFunctionInfo pythonFunctionInfo,
                                                      org.apache.flink.api.common.typeinfo.TypeInformation<IN1> inputTypeInfo1,
                                                      org.apache.flink.api.common.typeinfo.TypeInformation<IN2> inputTypeInfo2,
                                                      org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outputTypeInfo)

Method Detail

open

public void open()
          throws Exception

Specified by:

open in interface org.apache.flink.streaming.api.operators.StreamOperator<IN1>

Overrides:

open in class AbstractEmbeddedDataStreamPythonFunctionOperator<OUT>

Throws:

Exception

openPythonInterpreter

public void openPythonInterpreter()

Description copied from class: AbstractEmbeddedPythonFunctionOperator

Setup method for Python Interpreter.

Specified by:

openPythonInterpreter in class AbstractEmbeddedPythonFunctionOperator<OUT>

endInput

public void endInput(int inputId)
              throws Exception

Specified by:

endInput in interface org.apache.flink.streaming.api.operators.BoundedMultiInput

Throws:

Exception

close

public void close()
           throws Exception

Specified by:

close in interface org.apache.flink.streaming.api.operators.StreamOperator<IN1>

Overrides:

close in class AbstractEmbeddedPythonFunctionOperator<OUT>

Throws:

Exception

processElement1

public void processElement1(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN1> element)
                     throws Exception

Specified by:

processElement1 in interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator<IN1,IN2,OUT>

Throws:

Exception

processElement2

public void processElement2(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN2> element)
                     throws Exception

Specified by:

processElement2 in interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator<IN1,IN2,OUT>

Throws:

Exception

createUserDefinedFunctionsProto

public abstract List<FlinkFnApi.UserDefinedDataStreamFunction> createUserDefinedFunctionsProto()

Gets the proto representation of the Python user-defined functions to be executed.

getFunctionContext

public abstract Object getFunctionContext()

Gets The function context.

getTimerContext

public abstract Object getTimerContext()

Gets The Timer context.