4.0.0io.confluentcommon5.2.0io.confluentkafka-connect-elasticsearchjarkafka-connect-elasticsearchConfluent, Inc.http://confluent.iohttp://confluent.io
Elasticsearch Sink Connector for Kafka Connect
Confluent Community Licensehttp://www.confluent.io/confluent-community-licensereposcm:git:git://github.com/confluentinc/kafka-connect-elasticsearch.gitscm:git:git@github.com:confluentinc/kafka-connect-elasticsearch.githttps://github.com/confluentinc/kafka-connect-elasticsearchHEAD2.4.15.5.22.4.00.11.1http://packages.confluent.io/maven/0.28.0confluentConfluent${confluent.maven.repo}org.apache.kafkaconnect-apiprovidedorg.apache.kafkaconnect-jsonprovidedio.searchboxjest${jest.version}org.hamcresthamcrest-all${hamcrest.version}testorg.mockitomockito-core${mockito.version}testjunitjunit${junit.version}org.hamcresthamcrest-coretestorg.apache.lucenelucene-test-framework${lucene.version}testorg.elasticsearchelasticsearch${es.version}testtest-jarorg.elasticsearchelasticsearch${es.version}testorg.apache.lucenelucene-expressions${lucene.version}testcom.google.guavaguava27.0.1-jretestorg.apache.kafkaconnect-runtime${kafka.version}testtest-jartestorg.apache.kafkaconnect-runtime${kafka.version}org.apache.kafkakafka-clients${kafka.version}testtest-jartestorg.apache.kafkakafka_${kafka.scala.version}${kafka.version}testorg.apache.kafkakafka_${kafka.scala.version}test-jartesttest${kafka.version}org.apache.maven.pluginsmaven-failsafe-plugin3.0.0-M3trueio.confluent${kafka.connect.maven.plugin.version}kafka-connect-maven-pluginkafka-connectKafka Connect Elasticsearchhttps://docs.confluent.io/${project.version}/connect/connect-elasticsearch/docs/index.html
The Elasticsearch connector allows moving data from Kafka to Elasticsearch. It writes data from a topic in Kafka to an index in Elasticsearch and all data for a topic have the same type.
Elasticsearch is often used for text queries, analytics and as an key-value store (use cases). The connector covers both the analytics and key-value store use cases. For the analytics use case, each message is in Kafka is treated as an event and the connector uses topic+partition+offset as a unique identifier for events, which then converted to unique documents in Elasticsearch. For the key-value store use case, it supports using keys from Kafka messages as document ids in Elasticsearch and provides configurations ensuring that updates to a key are written to Elasticsearch in order. For both use cases, Elasticsearch’s idempotent write semantics guarantees exactly once delivery.
Mapping is the process of defining how a document, and the fields it contains, are stored and indexed. Users can explicitly define mappings for types in indices. When a mapping is not explicitly defined, Elasticsearch can determine field names and types from data, however, some types such as timestamp and decimal, may not be correctly inferred. To ensure that the types are correctly inferred, the connector provides a feature to infer a mapping from the schemas of Kafka messages.
logos/elasticsearch.jpgConfluent, Inc.Confluent supports the Elasticsearch sink connector alongside community members as part of its Confluent Platform offering.https://docs.confluent.io/current/logos/confluent.pngconfluentincorganizationConfluent, Inc.https://confluent.io/logos/confluent.pngconfluentinccp-kafka-connect${project.version}sinkElasticelasticsearchsearchanalyticslogElasticsearch 2.x, 5.x, or 6.xexactlyOncetrueorg.apache.maven.pluginsmaven-compiler-plugintrue-Xlint:all-Werrororg.apache.maven.pluginsmaven-assembly-pluginsrc/assembly/development.xmlsrc/assembly/package.xmlmake-assemblypackagesingleorg.apache.maven.pluginsmaven-surefire-plugin@{argLine} -Djava.awt.headless=true -Dtests.security.manager=false -Dtests.jarhell.check=falseorg.apache.maven.pluginsmaven-checkstyle-pluginvalidatevalidatecheckstyle/suppressions.xmlchecksrc/main/resourcestruestandalonemaven-assembly-pluginsrc/assembly/standalone.xmljenkinsorg.apache.maven.pluginsmaven-failsafe-plugin3.0.0-M3falseio.fabric8docker-maven-plugin${version.docker.maven.plugin}500defaulttrueelastic-integration-testconfluentinc/elastic-integration-%t${project.version}
${project.basedir}/src/test/resources/Dockerfile
elastichost9200:9200localhost9200startpre-integration-testbuildstartstoppost-integration-teststoplicenses-packageorg.codehaus.mojoexec-maven-plugin1.2.1create-licensesio.confluent.licenses.LicenseFinder-i ${project.build.directory}/${project.build.finalName}-package/share/java/kafka-connect-elasticsearch-o ${project.basedir}/licenses-f-h ${project.build.directory}/${project.build.finalName}-package/share/doc/kafka-connect-elasticsearch/licenses.html-l ${project.build.directory}/${project.build.finalName}-package/share/doc/kafka-connect-elasticsearch/licenses-n ${project.build.directory}/${project.build.finalName}-package/share/doc/kafka-connect-elasticsearch/notices-t ${project.name}-x licenses-${project.version}.jarpackagejavatruetrueio.confluentlicensesio.confluentlicenses${project.version}licenses-sourceorg.codehaus.mojoexec-maven-plugin1.2.1create-licensesio.confluent.licenses.LicenseFinder-i ${project.build.directory}/${project.build.finalName}-development/share/java/kafka-connect-elasticsearch-o ${project.basedir}/licenses-f-h ${project.basedir}/licenses.html-l ${project.basedir}/licenses-n ${project.basedir}/notices-t ${project.name}-x licenses-${project.version}.jarpackagejavatruetrueio.confluentlicensesio.confluentlicenses${project.version}maven-clean-plugin3.0.0.licenses.htmllicenses/*.*notices/*.*