Interface RateLimiterStrategy<SplitT extends SourceSplit>
-
- Type Parameters:
SplitT- The type of the source splits.
- All Superinterfaces:
Serializable
@Experimental public interface RateLimiterStrategy<SplitT extends SourceSplit> extends Serializable
A factory forRateLimiterswhich apply rate-limiting to a source sub-task.
-
-
Method Summary
All Methods Static Methods Instance Methods Abstract Methods Modifier and Type Method Description RateLimiter<SplitT>createRateLimiter(int parallelism)Creates aRateLimiterthat limits the rate of records going through.static RateLimiterStrategynoOp()Creates a convenienceRateLimiterStrategythat is not limiting the records rate.static RateLimiterStrategyperCheckpoint(int recordsPerCheckpoint)Creates aRateLimiterStrategythat is limiting the number of records per checkpoint.static RateLimiterStrategyperSecond(double recordsPerSecond)Creates aRateLimiterStrategythat is limiting the number of records per second.
-
-
-
Method Detail
-
createRateLimiter
RateLimiter<SplitT> createRateLimiter(int parallelism)
Creates aRateLimiterthat limits the rate of records going through. When there is parallelism, the limiting rate is evenly reduced per subtask, such that all the sub-tasks limiting rates equals the cumulative limiting rate.
-
perSecond
static RateLimiterStrategy perSecond(double recordsPerSecond)
Creates aRateLimiterStrategythat is limiting the number of records per second.- Parameters:
recordsPerSecond- The number of records produced per second. The actual number of produced records is subject to rounding due to dividing the number of produced records among the parallel instances.
-
perCheckpoint
static RateLimiterStrategy perCheckpoint(int recordsPerCheckpoint)
Creates aRateLimiterStrategythat is limiting the number of records per checkpoint.- Parameters:
recordsPerCheckpoint- The number of records produced per checkpoint. This value has to be greater or equal to parallelism. The actual number of produced records is subject to rounding due to dividing the number of produced records among the parallel instances.
-
noOp
static RateLimiterStrategy noOp()
Creates a convenienceRateLimiterStrategythat is not limiting the records rate.
-
-