Package org.apache.flink.table.runtime.operators.bundle

Class AbstractMapBundleOperator<K,​V,​IN,​OUT>

  • Type Parameters:
    K - The type of the key in the bundle map
    V - The type of the value in the bundle map
    IN - Input type for the operator.
    OUT - Output type for the operator.
    All Implemented Interfaces:
    Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.Input<IN>, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.KeyContextHandler, org.apache.flink.streaming.api.operators.OneInputStreamOperator<IN,​OUT>, org.apache.flink.streaming.api.operators.SetupableStreamOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, BundleTriggerCallback
    Direct Known Subclasses:
    KeyedMapBundleOperator, MapBundleOperator
    public abstract class AbstractMapBundleOperator<K,​V,​IN,​OUT>
extends org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>
implements org.apache.flink.streaming.api.operators.OneInputStreamOperator<IN,​OUT>, BundleTriggerCallback
    The AbstractMapBundleOperator simply used a java Map to store the input elements in key-value form. The map key is typically the same with the state key, so we can do some optimizations before accessing states, like pre aggregate values for each key. And we will only need to access state every key we have, but not every element we processed.

    NOTES: if all elements we processed have different keys, such operator will only increase memory footprint, and will not have any performance improvement.

    See Also:
    Serialized Form

    • Field Summary

      • Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

        chainingStrategy, config, latencyStats, LOG, metrics, output, processingTimeService

    • Method Summary

      All Methods Instance Methods Abstract Methods Concrete Methods 
      Modifier and Type Method Description
      void close()  
      void finish()  
      void finishBundle()
      This method is invoked to finish current bundle and start a new one when the trigger was fired.
      protected abstract K getKey​(IN input)
      Get the key for current processing element, which will be used as the map bundle's key.
      void open()  
      void prepareSnapshotPreBarrier​(long checkpointId)  
      void processElement​(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN> element)  
      void processWatermark​(org.apache.flink.streaming.api.watermark.Watermark mark)  

      • Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

        getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotState

      • Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

        notifyCheckpointAborted, notifyCheckpointComplete

      • Methods inherited from interface org.apache.flink.streaming.api.operators.Input

        processLatencyMarker, processWatermarkStatus

      • Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

        getCurrentKey, setCurrentKey

      • Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

        hasKeyContext

      • Methods inherited from interface org.apache.flink.streaming.api.operators.OneInputStreamOperator

        setKeyContextElement

      • Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

        getMetricGroup, getOperatorID, initializeState, setKeyContextElement1, setKeyContextElement2, snapshotState

    • Method Detail

      • open

        public void open()
          throws Exception
        Specified by:
        open in interface org.apache.flink.streaming.api.operators.StreamOperator<K>
        Overrides:
        open in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>
        Throws:
        Exception

      • processElement

        public void processElement​(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN> element)
                    throws Exception
        Specified by:
        processElement in interface org.apache.flink.streaming.api.operators.Input<K>
        Throws:
        Exception

      • getKey

        protected abstract K getKey​(IN input)
                     throws Exception
        Get the key for current processing element, which will be used as the map bundle's key.
        Throws:
        Exception

      • finishBundle

        public void finishBundle()
                  throws Exception
        Description copied from interface: BundleTriggerCallback
        This method is invoked to finish current bundle and start a new one when the trigger was fired.
        Specified by:
        finishBundle in interface BundleTriggerCallback
        Throws:
        Exception - This method may throw exceptions. Throwing an exception will cause the operation to fail and may trigger recovery.

      • processWatermark

        public void processWatermark​(org.apache.flink.streaming.api.watermark.Watermark mark)
                      throws Exception
        Specified by:
        processWatermark in interface org.apache.flink.streaming.api.operators.Input<K>
        Overrides:
        processWatermark in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>
        Throws:
        Exception

      • prepareSnapshotPreBarrier

        public void prepareSnapshotPreBarrier​(long checkpointId)
                               throws Exception
        Specified by:
        prepareSnapshotPreBarrier in interface org.apache.flink.streaming.api.operators.StreamOperator<K>
        Overrides:
        prepareSnapshotPreBarrier in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>
        Throws:
        Exception

      • finish

        public void finish()
            throws Exception
        Specified by:
        finish in interface org.apache.flink.streaming.api.operators.StreamOperator<K>
        Overrides:
        finish in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>
        Throws:
        Exception

      • close

        public void close()
           throws Exception
        Specified by:
        close in interface org.apache.flink.streaming.api.operators.StreamOperator<K>
        Overrides:
        close in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>
        Throws:
        Exception