Package org.apache.flink.streaming.api.operators.python.process

Class ExternalPythonBatchKeyedCoBroadcastProcessOperator<OUT>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.process.AbstractExternalPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.process.AbstractExternalDataStreamPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.process.AbstractExternalTwoInputPythonFunctionOperator<org.apache.flink.types.Row,org.apache.flink.types.Row,OUT>

org.apache.flink.streaming.api.operators.python.process.ExternalPythonKeyedCoProcessOperator<OUT>

org.apache.flink.streaming.api.operators.python.process.ExternalPythonBatchKeyedCoBroadcastProcessOperator<OUT>

Type Parameters:

OUT - The output type of the CoBroadcastProcess function

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.api.java.typeutils.ResultTypeQueryable<OUT>, org.apache.flink.streaming.api.operators.BoundedMultiInput, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.KeyContextHandler, DataStreamPythonFunctionOperator<OUT>, org.apache.flink.streaming.api.operators.SetupableStreamOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.streaming.api.operators.Triggerable<org.apache.flink.types.Row,org.apache.flink.runtime.state.VoidNamespace>, org.apache.flink.streaming.api.operators.TwoInputStreamOperator<org.apache.flink.types.Row,org.apache.flink.types.Row,OUT>

@Internal
public class ExternalPythonBatchKeyedCoBroadcastProcessOperator<OUT>
extends ExternalPythonKeyedCoProcessOperator<OUT>
implements org.apache.flink.streaming.api.operators.BoundedMultiInput

The ExternalPythonBatchKeyedCoBroadcastProcessOperator is responsible for executing the Python CoBroadcastProcess function under BATCH mode, ExternalPythonKeyedCoProcessOperator is used under STREAMING mode. This operator forces to run out data from broadcast side first, and then process data from regular side.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.flink.streaming.api.operators.python.process.AbstractExternalTwoInputPythonFunctionOperator

baos, baosWrapper

Fields inherited from class org.apache.flink.streaming.api.operators.python.process.AbstractExternalPythonFunctionOperator

pythonFunctionRunner

Fields inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

bundleFinishedCallback, config, elementCount, lastFinishBundleTime, maxBundleSize, systemEnvEnabled

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, lastRecordAttributes1, lastRecordAttributes2, latencyStats, LOG, metrics, output, processingTimeService

Constructor Summary

Constructors

Constructor

Description

ExternalPythonBatchKeyedCoBroadcastProcessOperator(org.apache.flink.configuration.Configuration config, DataStreamPythonFunctionInfo pythonFunctionInfo, org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row> inputTypeInfo1, org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row> inputTypeInfo2, org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outputTypeInfo)

Method Summary

Modifier and Type	Method	Description
void	endInput(int inputId)
void	processElement1(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.types.Row> element)

Methods inherited from class org.apache.flink.streaming.api.operators.python.process.ExternalPythonKeyedCoProcessOperator

copy, createPythonFunctionRunner, getCurrentKey, onEventTime, onProcessingTime, open, processElement2, setCurrentKey

Methods inherited from class org.apache.flink.streaming.api.operators.python.process.AbstractExternalTwoInputPythonFunctionOperator

createInputCoderInfoDescriptor, createOutputCoderInfoDescriptor, emitResult, getLeftInputType, getRightInputType, processElement

Methods inherited from class org.apache.flink.streaming.api.operators.python.process.AbstractExternalDataStreamPythonFunctionOperator

addSideOutputTags, createSideOutputCoderDescriptors, getInternalParameters, getOutputTagById, getProducedType, getPythonEnv, getPythonFunctionInfo, getSideOutputTags, getSideOutputTypeSerializerById, setNumPartitions

Methods inherited from class org.apache.flink.streaming.api.operators.python.process.AbstractExternalPythonFunctionOperator

close, createPythonEnvironmentManager, emitResults, invokeFinishBundle

Methods inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

checkInvokeFinishBundleByCount, finish, getConfiguration, getFlinkMetricContainer, isBundleFinished, prepareSnapshotPreBarrier, processWatermark

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

getChainingStrategy, getContainingTask, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes, processRecordAttributes1, processRecordAttributes2, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setChainingStrategy, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted, notifyCheckpointComplete

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

hasKeyContext

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

close, finish, getMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState

Methods inherited from interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator

processLatencyMarker1, processLatencyMarker2, processRecordAttributes1, processRecordAttributes2, processWatermark1, processWatermark2, processWatermarkStatus1, processWatermarkStatus2

Constructor Detail

ExternalPythonBatchKeyedCoBroadcastProcessOperator

public ExternalPythonBatchKeyedCoBroadcastProcessOperator(org.apache.flink.configuration.Configuration config,
                                                          DataStreamPythonFunctionInfo pythonFunctionInfo,
                                                          org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row> inputTypeInfo1,
                                                          org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row> inputTypeInfo2,
                                                          org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outputTypeInfo)

Method Detail

endInput

public void endInput(int inputId)
              throws Exception

Specified by:

endInput in interface org.apache.flink.streaming.api.operators.BoundedMultiInput

Overrides:

endInput in class AbstractExternalTwoInputPythonFunctionOperator<org.apache.flink.types.Row,org.apache.flink.types.Row,OUT>

Throws:

Exception

processElement1

public void processElement1(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.types.Row> element)
                     throws Exception

Specified by:

processElement1 in interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator<org.apache.flink.types.Row,org.apache.flink.types.Row,OUT>

Overrides:

processElement1 in class ExternalPythonKeyedCoProcessOperator<OUT>

Throws:

Exception