Interface WithPostCommitTopology<InputT,​CommT>

  • All Superinterfaces:
    Serializable, org.apache.flink.api.connector.sink2.Sink<InputT>, org.apache.flink.api.connector.sink2.TwoPhaseCommittingSink<InputT,​CommT>

    @Experimental
    public interface WithPostCommitTopology<InputT,​CommT>
    extends org.apache.flink.api.connector.sink2.TwoPhaseCommittingSink<InputT,​CommT>
    Allows expert users to implement a custom topology after Committer.

    It is recommended to use immutable committables because mutating committables can have unexpected side-effects.

    • Nested Class Summary

      • Nested classes/interfaces inherited from interface org.apache.flink.api.connector.sink2.Sink

        org.apache.flink.api.connector.sink2.Sink.InitContext
      • Nested classes/interfaces inherited from interface org.apache.flink.api.connector.sink2.TwoPhaseCommittingSink

        org.apache.flink.api.connector.sink2.TwoPhaseCommittingSink.PrecommittingSinkWriter<InputT extends Object,​CommT extends Object>
    • Method Detail

      • addPostCommitTopology

        void addPostCommitTopology​(DataStream<CommittableMessage<CommT>> committables)
        Adds a custom post-commit topology where all committables can be processed.

        It is strongly recommended to keep this pipeline stateless such that batch and streaming modes do not require special cases.

        All operations need to be idempotent: on recovery, any number of committables may be replayed that have already been committed. It's mandatory that these committables have no effect on the external system.

        Parameters:
        committables - the stream of committables.