Package org.apache.flink.streaming.api.runners.python.beam

Class BeamDataStreamPythonFunctionRunner

java.lang.Object

org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner

org.apache.flink.streaming.api.runners.python.beam.BeamDataStreamPythonFunctionRunner

All Implemented Interfaces:

AutoCloseable, PythonFunctionRunner

@Internal
public class BeamDataStreamPythonFunctionRunner
extends BeamPythonFunctionRunner

BeamDataStreamPythonFunctionRunner is responsible for starting a beam python harness to execute user defined python function.

Field Summary

Fields inherited from class org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner

inputCoderDescriptor, LOG, mainInputReceiver, outputCoderDescriptor, resultBuffer, sideOutputCoderDescriptors

Constructor Summary

Constructors

Constructor

Description

BeamDataStreamPythonFunctionRunner(org.apache.flink.runtime.execution.Environment environment, String taskName, ProcessPythonEnvironmentManager environmentManager, String headOperatorFunctionUrn, List<FlinkFnApi.UserDefinedDataStreamFunction> userDefinedDataStreamFunctions, FlinkMetricContainer flinkMetricContainer, org.apache.flink.runtime.state.KeyedStateBackend<?> keyedStateBackend, org.apache.flink.runtime.state.OperatorStateBackend operatorStateBackend, org.apache.flink.api.common.typeutils.TypeSerializer<?> keySerializer, org.apache.flink.api.common.typeutils.TypeSerializer<?> namespaceSerializer, TimerRegistration timerRegistration, org.apache.flink.runtime.memory.MemoryManager memoryManager, double managedMemoryFraction, FlinkFnApi.CoderInfoDescriptor inputCoderDescriptor, FlinkFnApi.CoderInfoDescriptor outputCoderDescriptor, FlinkFnApi.CoderInfoDescriptor timerCoderDescriptor, Map<String,FlinkFnApi.CoderInfoDescriptor> sideOutputCoderDescriptors)

Method Summary

Modifier and Type	Method	Description
protected void	buildTransforms(org.apache.beam.model.pipeline.v1.RunnerApi.Components.Builder componentsBuilder)
protected Optional<org.apache.beam.model.pipeline.v1.RunnerApi.Coder>	getOptionalTimerCoderProto()
protected List<org.apache.beam.runners.core.construction.graph.TimerReference>	getTimers(org.apache.beam.model.pipeline.v1.RunnerApi.Components components)

Methods inherited from class org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner

close, createJobBundleFactory, drainUnregisteredTimers, flush, notifyNoMoreResults, open, pollResult, process, processTimer, startBundle, takeResult

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BeamDataStreamPythonFunctionRunner

public BeamDataStreamPythonFunctionRunner(org.apache.flink.runtime.execution.Environment environment,
                                          String taskName,
                                          ProcessPythonEnvironmentManager environmentManager,
                                          String headOperatorFunctionUrn,
                                          List<FlinkFnApi.UserDefinedDataStreamFunction> userDefinedDataStreamFunctions,
                                          @Nullable
                                          FlinkMetricContainer flinkMetricContainer,
                                          @Nullable
                                          org.apache.flink.runtime.state.KeyedStateBackend<?> keyedStateBackend,
                                          @Nullable
                                          org.apache.flink.runtime.state.OperatorStateBackend operatorStateBackend,
                                          @Nullable
                                          org.apache.flink.api.common.typeutils.TypeSerializer<?> keySerializer,
                                          @Nullable
                                          org.apache.flink.api.common.typeutils.TypeSerializer<?> namespaceSerializer,
                                          @Nullable
                                          TimerRegistration timerRegistration,
                                          org.apache.flink.runtime.memory.MemoryManager memoryManager,
                                          double managedMemoryFraction,
                                          FlinkFnApi.CoderInfoDescriptor inputCoderDescriptor,
                                          FlinkFnApi.CoderInfoDescriptor outputCoderDescriptor,
                                          @Nullable
                                          FlinkFnApi.CoderInfoDescriptor timerCoderDescriptor,
                                          Map<String,FlinkFnApi.CoderInfoDescriptor> sideOutputCoderDescriptors)

Method Detail

buildTransforms

protected void buildTransforms(org.apache.beam.model.pipeline.v1.RunnerApi.Components.Builder componentsBuilder)

Specified by:

buildTransforms in class BeamPythonFunctionRunner

getTimers

protected List<org.apache.beam.runners.core.construction.graph.TimerReference> getTimers(org.apache.beam.model.pipeline.v1.RunnerApi.Components components)

Specified by:

getTimers in class BeamPythonFunctionRunner

getOptionalTimerCoderProto

protected Optional<org.apache.beam.model.pipeline.v1.RunnerApi.Coder> getOptionalTimerCoderProto()

Specified by:

getOptionalTimerCoderProto in class BeamPythonFunctionRunner