Interface RateLimiterStrategy<SplitT extends SourceSplit>

  • Type Parameters:
    SplitT - The type of the source splits.
    All Superinterfaces:
    Serializable

    @Experimental
    public interface RateLimiterStrategy<SplitT extends SourceSplit>
    extends Serializable
    A factory for RateLimiters which apply rate-limiting to a source sub-task.
    • Method Detail

      • createRateLimiter

        RateLimiter<SplitT> createRateLimiter​(int parallelism)
        Creates a RateLimiter that limits the rate of records going through. When there is parallelism, the limiting rate is evenly reduced per subtask, such that all the sub-tasks limiting rates equals the cumulative limiting rate.
      • perSecond

        static RateLimiterStrategy perSecond​(double recordsPerSecond)
        Creates a RateLimiterStrategy that is limiting the number of records per second.
        Parameters:
        recordsPerSecond - The number of records produced per second. The actual number of produced records is subject to rounding due to dividing the number of produced records among the parallel instances.
      • perCheckpoint

        static RateLimiterStrategy perCheckpoint​(int recordsPerCheckpoint)
        Creates a RateLimiterStrategy that is limiting the number of records per checkpoint.
        Parameters:
        recordsPerCheckpoint - The number of records produced per checkpoint. This value has to be greater or equal to parallelism. The actual number of produced records is subject to rounding due to dividing the number of produced records among the parallel instances.
      • noOp

        static RateLimiterStrategy noOp()
        Creates a convenience RateLimiterStrategy that is not limiting the records rate.