Package org.apache.flink.table.runtime.operators.deduplicate

Class RowTimeDeduplicateKeepFirstRowFunction

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.streaming.api.functions.KeyedProcessFunction<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>

org.apache.flink.table.runtime.operators.deduplicate.RowTimeDeduplicateKeepFirstRowFunction

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction

public class RowTimeDeduplicateKeepFirstRowFunction
extends org.apache.flink.streaming.api.functions.KeyedProcessFunction<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>

This function is used to deduplicate on keys and keeps only first row on row time. It produces append only stream thanks to emitting results only via firing the timers.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.streaming.api.functions.KeyedProcessFunction

org.apache.flink.streaming.api.functions.KeyedProcessFunction.Context, org.apache.flink.streaming.api.functions.KeyedProcessFunction.OnTimerContext

Field Summary

Fields

Modifier and Type	Field	Description
protected org.apache.flink.api.common.state.ValueState<Boolean>	alreadyEmittedState
protected org.apache.flink.api.common.state.ValueState<org.apache.flink.table.data.RowData>	waitingToEmitOnTimerState

Constructor Summary

Constructors

Constructor	Description
RowTimeDeduplicateKeepFirstRowFunction(InternalTypeInfo<org.apache.flink.table.data.RowData> typeInfo, long minRetentionTime, int rowtimeIndex)

Method Summary

Modifier and Type	Method	Description
protected org.apache.flink.metrics.Counter	getNumLateRecordsDropped()
void	onTimer(long timestamp, org.apache.flink.streaming.api.functions.KeyedProcessFunction.OnTimerContext ctx, org.apache.flink.util.Collector<org.apache.flink.table.data.RowData> out)
void	open(org.apache.flink.api.common.functions.OpenContext openContext)
void	processElement(org.apache.flink.table.data.RowData input, org.apache.flink.streaming.api.functions.KeyedProcessFunction.Context ctx, org.apache.flink.util.Collector<org.apache.flink.table.data.RowData> out)

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

close, getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

waitingToEmitOnTimerState

protected org.apache.flink.api.common.state.ValueState<org.apache.flink.table.data.RowData> waitingToEmitOnTimerState

alreadyEmittedState

protected org.apache.flink.api.common.state.ValueState<Boolean> alreadyEmittedState

Constructor Detail

RowTimeDeduplicateKeepFirstRowFunction

public RowTimeDeduplicateKeepFirstRowFunction(InternalTypeInfo<org.apache.flink.table.data.RowData> typeInfo,
                                              long minRetentionTime,
                                              int rowtimeIndex)

Method Detail

open

public void open(org.apache.flink.api.common.functions.OpenContext openContext)
          throws Exception

Specified by:

open in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

open in class org.apache.flink.api.common.functions.AbstractRichFunction

Throws:

Exception

processElement

public void processElement(org.apache.flink.table.data.RowData input,
                           org.apache.flink.streaming.api.functions.KeyedProcessFunction.Context ctx,
                           org.apache.flink.util.Collector<org.apache.flink.table.data.RowData> out)
                    throws Exception

Specified by:

processElement in class org.apache.flink.streaming.api.functions.KeyedProcessFunction<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>

Throws:

Exception

onTimer

public void onTimer(long timestamp,
                    org.apache.flink.streaming.api.functions.KeyedProcessFunction.OnTimerContext ctx,
                    org.apache.flink.util.Collector<org.apache.flink.table.data.RowData> out)
             throws Exception

Overrides:

onTimer in class org.apache.flink.streaming.api.functions.KeyedProcessFunction<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>

Throws:

Exception

getNumLateRecordsDropped

@VisibleForTesting
protected org.apache.flink.metrics.Counter getNumLateRecordsDropped()