4.0.0io.confluentkafka-connect-storage-cloud5.3.3kafka-connect-s3jarkafka-connect-s3
Kafka Connect cloud storage connector for Amazon Simple Storage Service (S3).
1.11.7250.2.54.5.90.11.1com.amazonawsaws-java-sdk-bom${aws.version}pomimportcom.amazonawsaws-java-sdk-sts${aws.version}com.amazonawsaws-java-sdk-s3com.amazonawsaws-java-sdk-stsorg.apache.httpcomponentshttpclient${apache.httpclient.version}io.findifys3mock_2.12${s3mock.version}testio.confluentkafka-connect-maven-plugin${kafka.connect.maven.plugin.version}kafka-connectKafka Connect S3https://github.com/confluentinc/kafka-connect-storage-cloudhttps://docs.confluent.io/${project.version}/connect/connect-storage-cloud/kafka-connect-s3/docs/index.html
The S3 connector, currently available as a sink, allows you to export data from Kafka topics to S3 objects in either Avro or JSON formats. In addition, for certain data layouts, S3 connector exports data by guaranteeing exactly-once delivery semantics to consumers of the S3 objects it produces.
Being a sink, the S3 connector periodically polls data from Kafka and in turn uploads it to S3. A partitioner is used to split the data of every Kafka partition into chunks. Each chunk of data is represented as an S3 object, whose key name encodes the topic, the Kafka partition and the start offset of this data chunk. If no partitioner is specified in the configuration, the default partitioner which preserves Kafka partitioning is used. The size of each data chunk is determined by the number of records written to S3 and by schema compatibility.
logos/s3.jpgConfluent, Inc.Confluent supports the S3 sink connector alongside community members as part of its Confluent Platform offering.https://docs.confluent.io/current/logos/confluent.pngconfluentincorganizationConfluent, Inc.https://confluent.io/logos/confluent.pngconfluentinccp-kafka-connect${project.version}sinks3awstrueAWS S3 bucket with write permissionsorg.apache.maven.pluginsmaven-compiler-plugin2.5.1truetest-compileprocess-test-sourcestestCompile1.8maven-assembly-pluginsrc/assembly/development.xmlsrc/assembly/package.xmlfalsemake-assemblypackagesinglesrc/main/resourcestruestandalonemaven-assembly-pluginsrc/assembly/standalone.xmllicenses-packagefalseorg.codehaus.mojoexec-maven-plugin1.2.1create-licensesio.confluent.licenses.LicenseFinder-i
${project.build.directory}/${project.build.finalName}-package/share/java/${project.name}
-o ${project.basedir}/licenses-f-h
${project.build.directory}/${project.build.finalName}-package/share/doc/${project.name}/licenses.html
-l
${project.build.directory}/${project.build.finalName}-package/share/doc/${project.name}/licenses
-n
${project.build.directory}/${project.build.finalName}-package/share/doc/${project.name}/notices
-t ${project.name}-x licenses-${licenses.version}.jarpackagejavatruetrueio.confluentlicensesio.confluentlicenses${licenses.version}licenses-sourcefalseorg.codehaus.mojoexec-maven-plugin1.2.1create-licensesio.confluent.licenses.LicenseFinder-i
${project.build.directory}/${project.build.finalName}-development/share/java/${project.name}
-o ${project.basedir}/licenses-f-h ${project.basedir}/licenses.html-l ${project.basedir}/licenses-n ${project.basedir}/notices-t ${project.name}-x licenses-${licenses.version}.jarpackagejavatruetrueio.confluentlicensesio.confluentlicenses${licenses.version}maven-clean-plugin3.0.0.licenses.htmllicenses/*.*notices/*.*