Package org.apache.flink.streaming.api.operators.python.process

Class AbstractExternalPythonFunctionOperator<OUT>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.process.AbstractExternalPythonFunctionOperator<OUT>

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.KeyContextHandler, org.apache.flink.streaming.api.operators.StreamOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.streaming.api.operators.YieldingOperator<OUT>

Direct Known Subclasses:

AbstractExternalDataStreamPythonFunctionOperator, AbstractOneInputPythonFunctionOperator

@Internal
public abstract class AbstractExternalPythonFunctionOperator<OUT>
extends AbstractPythonFunctionOperator<OUT>

Abstract class for all stream operators to execute Python functions in external environment.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field	Description
protected PythonFunctionRunner	pythonFunctionRunner	The PythonFunctionRunner which is responsible for Python user-defined function execution.

Fields inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

bundleFinishedCallback, config, elementCount, lastFinishBundleTime, maxBundleSize, systemEnvEnabled

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

combinedWatermark, lastRecordAttributes1, lastRecordAttributes2, latencyStats, LOG, metrics, output, processingTimeService, stateHandler, stateKeySelector1, stateKeySelector2, timeServiceManager

Constructor Summary

Constructors

Constructor	Description
AbstractExternalPythonFunctionOperator(org.apache.flink.configuration.Configuration config)

Method Summary

Modifier and Type	Method	Description
void	close()
protected ProcessPythonEnvironmentManager	createPythonEnvironmentManager()
abstract PythonFunctionRunner	createPythonFunctionRunner()	Creates the PythonFunctionRunner which is responsible for Python user-defined function execution.
protected void	drainUnregisteredTimers()
abstract void	emitResult(org.apache.flink.api.java.tuple.Tuple3<String,byte[],Integer> resultTuple)	Sends the execution result to the downstream operator.
protected void	emitResults()
abstract org.apache.flink.table.functions.python.PythonEnv	getPythonEnv()	Returns the PythonEnv used to create PythonEnvironmentManager..
protected void	invokeFinishBundle()
void	open()

Methods inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

checkInvokeFinishBundleByCount, finish, getConfiguration, getFlinkMetricContainer, isBundleFinished, prepareSnapshotPreBarrier, processWatermark, setCurrentKey

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

beforeInitializeStateHandler, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getStateKeySelector1, getStateKeySelector2, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isAsyncStateProcessingEnabled, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes, processRecordAttributes1, processRecordAttributes2, processWatermark, processWatermark1, processWatermark1, processWatermark2, processWatermark2, processWatermarkStatus, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setKeyContextElement1, setKeyContextElement2, setMailboxExecutor, setProcessingTimeService, setup, snapshotState, snapshotState, useSplittableTimers

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

hasKeyContext

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

getOperatorAttributes

Field Detail

pythonFunctionRunner

protected transient PythonFunctionRunner pythonFunctionRunner

The PythonFunctionRunner which is responsible for Python user-defined function execution.

Constructor Detail

AbstractExternalPythonFunctionOperator

public AbstractExternalPythonFunctionOperator(org.apache.flink.configuration.Configuration config)

Method Detail

open

public void open()
          throws Exception

Specified by:

open in interface org.apache.flink.streaming.api.operators.StreamOperator<OUT>

Overrides:

open in class AbstractPythonFunctionOperator<OUT>

Throws:

Exception

close

public void close()
           throws Exception

Specified by:

close in interface org.apache.flink.streaming.api.operators.StreamOperator<OUT>

Overrides:

close in class AbstractPythonFunctionOperator<OUT>

Throws:

Exception

invokeFinishBundle

protected void invokeFinishBundle()
                           throws Exception

Specified by:

invokeFinishBundle in class AbstractPythonFunctionOperator<OUT>

Throws:

Exception

createPythonEnvironmentManager

protected ProcessPythonEnvironmentManager createPythonEnvironmentManager()

Specified by:

createPythonEnvironmentManager in class AbstractPythonFunctionOperator<OUT>

drainUnregisteredTimers

protected void drainUnregisteredTimers()

Overrides:

drainUnregisteredTimers in class AbstractPythonFunctionOperator<OUT>

emitResults

protected void emitResults()
                    throws Exception

Throws:

Exception

getPythonEnv

public abstract org.apache.flink.table.functions.python.PythonEnv getPythonEnv()

Returns the PythonEnv used to create PythonEnvironmentManager..

emitResult

public abstract void emitResult(org.apache.flink.api.java.tuple.Tuple3<String,byte[],Integer> resultTuple)
                         throws Exception

Sends the execution result to the downstream operator.

Throws:

Exception

createPythonFunctionRunner

public abstract PythonFunctionRunner createPythonFunctionRunner()
                                                         throws Exception

Creates the PythonFunctionRunner which is responsible for Python user-defined function execution.

Throws:

Exception