Package org.apache.flink.table.runtime.operators.wmassigners

Class ProcTimeMiniBatchAssignerOperator

  • All Implemented Interfaces:
    Serializable, org.apache.flink.api.common.operators.ProcessingTimeService.ProcessingTimeCallback, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.Input<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.KeyContextHandler, org.apache.flink.streaming.api.operators.OneInputStreamOperator<org.apache.flink.table.data.RowData,​org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.SetupableStreamOperator<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator
    public class ProcTimeMiniBatchAssignerOperator
extends org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>
implements org.apache.flink.streaming.api.operators.OneInputStreamOperator<org.apache.flink.table.data.RowData,​org.apache.flink.table.data.RowData>, org.apache.flink.api.common.operators.ProcessingTimeService.ProcessingTimeCallback
    A stream operator that emits mini-batch marker in a given period. This mini-batch assigner works in processing time, which means the mini-batch marker is generated in the given period using the processing time. The downstream operators will trigger mini-batch once the received mini-batch id advanced.

    NOTE: currently, we use Watermark to represents the mini-batch marker.

    The difference between this operator and RowTimeMiniBatchAssginerOperator is that, this operator generates watermarks by itself using processing time, but the other forwards watermarks from upstream.

    See Also:
    Serialized Form

    • Field Summary

      • Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

        chainingStrategy, config, latencyStats, LOG, metrics, output, processingTimeService

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void onProcessingTime​(long timestamp)  
      void open()  
      void processElement​(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)  
      void processWatermark​(org.apache.flink.streaming.api.watermark.Watermark mark)
      Override the base implementation to completely ignore watermarks propagated from upstream (we rely only on the AssignerWithPeriodicWatermarks to emit watermarks from here).

      • Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

        close, finish, getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, prepareSnapshotPreBarrier, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotState

      • Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

        notifyCheckpointAborted, notifyCheckpointComplete

      • Methods inherited from interface org.apache.flink.streaming.api.operators.Input

        processLatencyMarker, processWatermarkStatus

      • Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

        getCurrentKey, setCurrentKey

      • Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

        hasKeyContext

      • Methods inherited from interface org.apache.flink.streaming.api.operators.OneInputStreamOperator

        setKeyContextElement

      • Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

        close, finish, getMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState

    • Constructor Detail

      • ProcTimeMiniBatchAssignerOperator

        public ProcTimeMiniBatchAssignerOperator​(long intervalMs)

    • Method Detail

      • open

        public void open()
          throws Exception
        Specified by:
        open in interface org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>
        Overrides:
        open in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>
        Throws:
        Exception

      • processElement

        public void processElement​(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)
                    throws Exception
        Specified by:
        processElement in interface org.apache.flink.streaming.api.operators.Input<org.apache.flink.table.data.RowData>
        Throws:
        Exception

      • onProcessingTime

        public void onProcessingTime​(long timestamp)
                      throws Exception
        Specified by:
        onProcessingTime in interface org.apache.flink.api.common.operators.ProcessingTimeService.ProcessingTimeCallback
        Throws:
        Exception

      • processWatermark

        public void processWatermark​(org.apache.flink.streaming.api.watermark.Watermark mark)
                      throws Exception
        Override the base implementation to completely ignore watermarks propagated from upstream (we rely only on the AssignerWithPeriodicWatermarks to emit watermarks from here).
        Specified by:
        processWatermark in interface org.apache.flink.streaming.api.operators.Input<org.apache.flink.table.data.RowData>
        Overrides:
        processWatermark in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>
        Throws:
        Exception