org.apache.kafka.connect.mirror

Class RemoteClusterUtils

java.lang.Object

org.apache.kafka.connect.mirror.RemoteClusterUtils

public final class RemoteClusterUtils
extends Object

Convenience tool for multi-cluster environments. Wraps MirrorClient

Properties passed to these methods are used to construct internal Admin and Consumer clients. Sub-configs like "admin.xyz" are also supported. For example:

     bootstrap.servers = host1:9092
     consumer.client.id = mm2-client
 

See Also:

for additional properties used by the internal MirrorClient.

Method Summary

Modifier and Type	Method and Description
static Set<String>	checkpointTopics(Map<String,Object> properties) Finds all checkpoints topics
static Set<String>	heartbeatTopics(Map<String,Object> properties) Finds all heartbeats topics
static int	replicationHops(Map<String,Object> properties, String upstreamClusterAlias) Finds the shortest number of hops from an upstream cluster.
static Map<org.apache.kafka.common.TopicPartition,org.apache.kafka.clients.consumer.OffsetAndMetadata>	translateOffsets(Map<String,Object> properties, String remoteClusterAlias, String consumerGroupId, Duration timeout) Translates a remote consumer group's offsets into corresponding local offsets.
static Set<String>	upstreamClusters(Map<String,Object> properties) Finds all upstream clusters

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

replicationHops

public static int replicationHops(Map<String,Object> properties,
                                  String upstreamClusterAlias)
                           throws InterruptedException,
                                  TimeoutException

Finds the shortest number of hops from an upstream cluster. Returns -1 if the cluster is unreachable.

Throws:

InterruptedException

TimeoutException

heartbeatTopics

public static Set<String> heartbeatTopics(Map<String,Object> properties)
                                   throws InterruptedException,
                                          TimeoutException

Finds all heartbeats topics

Throws:

InterruptedException

TimeoutException

checkpointTopics

public static Set<String> checkpointTopics(Map<String,Object> properties)
                                    throws InterruptedException,
                                           TimeoutException

Finds all checkpoints topics

Throws:

InterruptedException

TimeoutException

upstreamClusters

public static Set<String> upstreamClusters(Map<String,Object> properties)
                                    throws InterruptedException,
                                           TimeoutException

Finds all upstream clusters

Throws:

InterruptedException

TimeoutException

translateOffsets

public static Map<org.apache.kafka.common.TopicPartition,org.apache.kafka.clients.consumer.OffsetAndMetadata> translateOffsets(Map<String,Object> properties,
                                                                                                                               String remoteClusterAlias,
                                                                                                                               String consumerGroupId,
                                                                                                                               Duration timeout)
                                                                                                                        throws InterruptedException,
                                                                                                                               TimeoutException

Translates a remote consumer group's offsets into corresponding local offsets. Topics are automatically renamed according to the configured ReplicationPolicy.

Parameters:

properties - Map of properties to instantiate a MirrorClient

remoteClusterAlias - The alias of the remote cluster

consumerGroupId - The group ID of remote consumer group

timeout - The maximum time to block when consuming from the checkpoints topic

Throws:

InterruptedException

TimeoutException