Package org.apache.flink.connector.base.sink

Class AsyncSinkBaseBuilder<InputT,​RequestEntryT extends Serializable,​ConcreteBuilderT extends AsyncSinkBaseBuilder<?,​?,​?>>

  • Type Parameters:
    InputT - type of elements that should be persisted in the destination
    RequestEntryT - type of payload that contains the element and additional metadata that is required to submit a single element to the destination
    ConcreteBuilderT - type of concrete implementation of this builder class
    @PublicEvolving
public abstract class AsyncSinkBaseBuilder<InputT,​RequestEntryT extends Serializable,​ConcreteBuilderT extends AsyncSinkBaseBuilder<?,​?,​?>>
extends Object
    Abstract builder for constructing a concrete implementation of AsyncSinkBase.

    • Constructor Detail

      • AsyncSinkBaseBuilder

        public AsyncSinkBaseBuilder()

    • Method Detail

      • setMaxBatchSize

        public ConcreteBuilderT setMaxBatchSize​(int maxBatchSize)
        Parameters:
        maxBatchSize - maximum number of elements that may be passed in a list to be written downstream.
        Returns:
        AsyncSinkBaseBuilder itself

      • setMaxInFlightRequests

        public ConcreteBuilderT setMaxInFlightRequests​(int maxInFlightRequests)
        Parameters:
        maxInFlightRequests - maximum number of uncompleted calls to submitRequestEntries that the SinkWriter will allow at any given point. Once this point has reached, writes and callbacks to add elements to the buffer may block until one or more requests to submitRequestEntries completes.
        Returns:
        AsyncSinkBaseBuilder itself

      • setMaxBufferedRequests

        public ConcreteBuilderT setMaxBufferedRequests​(int maxBufferedRequests)
        Parameters:
        maxBufferedRequests - the maximum buffer length. Callbacks to add elements to the buffer and calls to write will block if this length has been reached and will only unblock if elements from the buffer have been removed for flushing.
        Returns:
        AsyncSinkBaseBuilder itself

      • setMaxBatchSizeInBytes

        public ConcreteBuilderT setMaxBatchSizeInBytes​(long maxBatchSizeInBytes)
        Parameters:
        maxBatchSizeInBytes - a flush will be attempted if the most recent call to write introduces an element to the buffer such that the total size of the buffer is greater than or equal to this threshold value. If this happens, the maximum number of elements from the head of the buffer will be selected, that is smaller than maxBatchSizeInBytes in size will be flushed.
        Returns:
        AsyncSinkBaseBuilder itself

      • setMaxTimeInBufferMS

        public ConcreteBuilderT setMaxTimeInBufferMS​(long maxTimeInBufferMS)
        Parameters:
        maxTimeInBufferMS - the maximum amount of time an element may remain in the buffer. In most cases elements are flushed as a result of the batch size (in bytes or number) being reached or during a snapshot. However, there are scenarios where an element may remain in the buffer forever or a long period of time. To mitigate this, a timer is constantly active in the buffer such that: while the buffer is not empty, it will flush every maxTimeInBufferMS milliseconds.
        Returns:
        AsyncSinkBaseBuilder itself

      • setMaxRecordSizeInBytes

        public ConcreteBuilderT setMaxRecordSizeInBytes​(long maxRecordSizeInBytes)
        Parameters:
        maxRecordSizeInBytes - the maximum size of each records in bytes. If a record larger than this is passed to the sink, it will throw an IllegalArgumentException.
        Returns:
        AsyncSinkBaseBuilder itself

      • getMaxBatchSize

        protected Integer getMaxBatchSize()

      • getMaxInFlightRequests

        protected Integer getMaxInFlightRequests()

      • getMaxBufferedRequests

        protected Integer getMaxBufferedRequests()

      • getMaxBatchSizeInBytes

        protected Long getMaxBatchSizeInBytes()

      • getMaxTimeInBufferMS

        protected Long getMaxTimeInBufferMS()

      • getMaxRecordSizeInBytes

        protected Long getMaxRecordSizeInBytes()