Package org.apache.flink.table.runtime.operators.window.tvf.common

Class WindowAggOperator<K,​W>

  • All Implemented Interfaces:
    Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.Input<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.KeyContextHandler, org.apache.flink.streaming.api.operators.OneInputStreamOperator<org.apache.flink.table.data.RowData,​org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.streaming.api.operators.Triggerable<K,​W>, org.apache.flink.streaming.api.operators.YieldingOperator<org.apache.flink.table.data.RowData>
    @Internal
public final class WindowAggOperator<K,​W>
extends TableStreamOperator<org.apache.flink.table.data.RowData>
implements org.apache.flink.streaming.api.operators.OneInputStreamOperator<org.apache.flink.table.data.RowData,​org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.Triggerable<K,​W>, org.apache.flink.streaming.api.operators.KeyContext
    We divide windows into 2 categories: Aligned Windows and Unaligned Windows.

    Concept of Aligned Window and Unaligned Window

    Aligned Windows are windows have predetermined window boundaries and windows can be divided into finite number of non-overlapping chunks. The boundary of an aligned window is determined independently from the time characteristic of the data stream, or messages it receives. For example, hopping (sliding) window is an aligned window as the window boundaries are predetermined based on the window size and slide. Aligned windows include tumbling, hopping, cumulative windows.

    Unaligned Windows are windows determined dynamically based on elements. For example, session window is an unaligned window as the window boundaries are determined based on the messages timestamps and their correlations. Currently, unaligned windows include session window only.

    Because aligned windows can be divided into finite number of non-overlapping chunks (a.k.a. slices), which can apply efficient processing to share intermediate results. 

     Window
 |
 +-- Aligned Window (Slicing Window)
 |    |
 |    +-- Tumble (Slice Unshared Window)
 |    |
 |    +-- Hop (Slice Shared Window)
 |    |
 |    +-- Cumulate (Slice Shared Window)
 |
 +-- Unaligned Window (Unslice Window)
      |
      +-- Session

    Note: currently, WindowAggOperator doesn't support early-fire and late-arrival. Thus late elements (elements belong to emitted windows) will be simply dropped.

    See more in SlicingSyncStateWindowProcessor and UnslicingSyncStateWindowProcessor.

    See Also:
    Serialized Form

    • Field Summary

      • Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

        combinedWatermark, config, lastRecordAttributes1, lastRecordAttributes2, latencyStats, LOG, metrics, output, processingTimeService, stateHandler, stateKeySelector1, stateKeySelector2, timeServiceManager

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()  
      org.apache.flink.metrics.Counter getNumLateRecordsDropped()  
      org.apache.flink.metrics.Gauge<Long> getWatermarkLatency()  
      void initializeState​(org.apache.flink.runtime.state.StateInitializationContext context)  
      void onEventTime​(org.apache.flink.streaming.api.operators.InternalTimer<K,​W> timer)  
      void onProcessingTime​(org.apache.flink.streaming.api.operators.InternalTimer<K,​W> timer)  
      void open()  
      void prepareSnapshotPreBarrier​(long checkpointId)  
      void processElement​(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)  
      void processWatermark​(org.apache.flink.streaming.api.watermark.Watermark mark)  
      void snapshotState​(org.apache.flink.runtime.state.StateSnapshotContext context)  

      • Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

        beforeInitializeStateHandler, finish, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getStateKeySelector1, getStateKeySelector2, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, isAsyncStateProcessingEnabled, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes, processRecordAttributes1, processRecordAttributes2, processWatermark, processWatermark1, processWatermark1, processWatermark2, processWatermark2, processWatermarkStatus, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setMailboxExecutor, setProcessingTimeService, setup, snapshotState

      • Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

        notifyCheckpointAborted, notifyCheckpointComplete

      • Methods inherited from interface org.apache.flink.streaming.api.operators.Input

        processLatencyMarker, processRecordAttributes, processWatermark, processWatermarkStatus

      • Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

        getCurrentKey, setCurrentKey

      • Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

        hasKeyContext

      • Methods inherited from interface org.apache.flink.streaming.api.operators.OneInputStreamOperator

        setKeyContextElement

      • Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

        finish, getMetricGroup, getOperatorAttributes, getOperatorID, initializeState, setKeyContextElement1, setKeyContextElement2, snapshotState

    • Constructor Detail

    • Method Detail

      • open

        public void open()
          throws Exception
        Specified by:
        open in interface org.apache.flink.streaming.api.operators.StreamOperator<K>
        Overrides:
        open in class TableStreamOperator<org.apache.flink.table.data.RowData>
        Throws:
        Exception

      • initializeState

        public void initializeState​(org.apache.flink.runtime.state.StateInitializationContext context)
                     throws Exception
        Specified by:
        initializeState in interface org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator
        Overrides:
        initializeState in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>
        Throws:
        Exception

      • snapshotState

        public void snapshotState​(org.apache.flink.runtime.state.StateSnapshotContext context)
                   throws Exception
        Specified by:
        snapshotState in interface org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator
        Overrides:
        snapshotState in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>
        Throws:
        Exception

      • close

        public void close()
           throws Exception
        Specified by:
        close in interface org.apache.flink.streaming.api.operators.StreamOperator<K>
        Overrides:
        close in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>
        Throws:
        Exception

      • processElement

        public void processElement​(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)
                    throws Exception
        Specified by:
        processElement in interface org.apache.flink.streaming.api.operators.Input<K>
        Throws:
        Exception

      • processWatermark

        public void processWatermark​(org.apache.flink.streaming.api.watermark.Watermark mark)
                      throws Exception
        Specified by:
        processWatermark in interface org.apache.flink.streaming.api.operators.Input<K>
        Overrides:
        processWatermark in class TableStreamOperator<org.apache.flink.table.data.RowData>
        Throws:
        Exception

      • onEventTime

        public void onEventTime​(org.apache.flink.streaming.api.operators.InternalTimer<K,​W> timer)
                 throws Exception
        Specified by:
        onEventTime in interface org.apache.flink.streaming.api.operators.Triggerable<K,​W>
        Throws:
        Exception

      • onProcessingTime

        public void onProcessingTime​(org.apache.flink.streaming.api.operators.InternalTimer<K,​W> timer)
                      throws Exception
        Specified by:
        onProcessingTime in interface org.apache.flink.streaming.api.operators.Triggerable<K,​W>
        Throws:
        Exception

      • prepareSnapshotPreBarrier

        public void prepareSnapshotPreBarrier​(long checkpointId)
                               throws Exception
        Specified by:
        prepareSnapshotPreBarrier in interface org.apache.flink.streaming.api.operators.StreamOperator<K>
        Overrides:
        prepareSnapshotPreBarrier in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>
        Throws:
        Exception

      • getNumLateRecordsDropped

        @VisibleForTesting
public org.apache.flink.metrics.Counter getNumLateRecordsDropped()

      • getWatermarkLatency

        @VisibleForTesting
public org.apache.flink.metrics.Gauge<Long> getWatermarkLatency()