Class RandomBrokerPartitionSubsetPartitioner

java.lang.Object

io.confluent.telemetry.events.exporter.kafka.RandomBrokerPartitionSubsetPartitioner

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.kafka.clients.producer.Partitioner, org.apache.kafka.common.Configurable

public class RandomBrokerPartitionSubsetPartitioner
extends Object
implements org.apache.kafka.clients.producer.Partitioner

A partitioner that reduces the number of connections a producer needs to make by

1. Specifying a subset percentage of total partitions to produce to
2. Randomly connecting to brokers and taking all partitions on that broker until it has connected to the subset of partitions
3. Randomly produce to the recorded partitions

NOTE: This partitioner will not respect keys when determining the partition to be used. This is done so that keyed messages do not result in connections to every broker.

Field Summary

Fields

Modifier and Type	Field	Description
static final double	DEFAULT_SUBSET_PARTITIONER_PARTITION_PERCENTAGE
static final String	SUBSET_PARTITIONER_PARTITION_PERCENTAGE_CONFIG
static final String	SUBSET_PARTITIONER_RECALCULATION_BACKOFF_CONFIG

Constructor Summary

Constructors

Constructor	Description
RandomBrokerPartitionSubsetPartitioner()

Method Summary

Modifier and Type	Method	Description
void	close()
void	configure(Map<String,?> configs)
int	partition(String topic, Object key, byte[] keyBytes, Object value, byte[] valueBytes, org.apache.kafka.common.Cluster cluster)	Choose the partition for a record from partitions found on a limited number of brokers according to `subset.partitioner.partition.percentage`.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

SUBSET_PARTITIONER_PARTITION_PERCENTAGE_CONFIG

public static final String SUBSET_PARTITIONER_PARTITION_PERCENTAGE_CONFIG

See Also:

• Constant Field Values

DEFAULT_SUBSET_PARTITIONER_PARTITION_PERCENTAGE

public static final double DEFAULT_SUBSET_PARTITIONER_PARTITION_PERCENTAGE

See Also:

• Constant Field Values

SUBSET_PARTITIONER_RECALCULATION_BACKOFF_CONFIG

public static final String SUBSET_PARTITIONER_RECALCULATION_BACKOFF_CONFIG

See Also:

• Constant Field Values

Constructor Details

RandomBrokerPartitionSubsetPartitioner

public RandomBrokerPartitionSubsetPartitioner()

Method Details

partition

public int partition(String topic,
 Object key,
 byte[] keyBytes,
 Object value,
 byte[] valueBytes,
 org.apache.kafka.common.Cluster cluster)

Choose the partition for a record from partitions found on a limited number of brokers according to `subset.partitioner.partition.percentage`. NOTE: this partitioner will not respect the record key when choosing a partition for the record.

Specified by:

partition in interface org.apache.kafka.clients.producer.Partitioner

configure

public void configure(Map<String,?> configs)

Specified by:

configure in interface org.apache.kafka.common.Configurable

close

public void close()

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Specified by:

close in interface org.apache.kafka.clients.producer.Partitioner