Package org.apache.flink.formats.compress

Class CompressWriterFactory<IN>

java.lang.Object

org.apache.flink.formats.compress.CompressWriterFactory<IN>

Type Parameters:

IN - The type of element to write.

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.serialization.BulkWriter.Factory<IN>

@PublicEvolving
public class CompressWriterFactory<IN>
extends Object
implements org.apache.flink.api.common.serialization.BulkWriter.Factory<IN>

A factory that creates for bulk writers that, when provided with a CompressionCodec, they compress the data they write. If no codec is provided, the data is written in bulk but uncompressed.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor	Description
CompressWriterFactory(Extractor<IN> extractor)	Creates a new CompressWriterFactory using the given Extractor to assemble either HadoopCompressionBulkWriter or NoCompressionBulkWriter based on whether a Hadoop CompressionCodec name is specified.

Method Summary

Modifier and Type	Method	Description
org.apache.flink.api.common.serialization.BulkWriter<IN>	create(org.apache.flink.core.fs.FSDataOutputStream out)
String	getExtension()
CompressWriterFactory<IN>	withHadoopCompression(String codecName)	Compresses the data using the provided Hadoop CompressionCodec.
CompressWriterFactory<IN>	withHadoopCompression(String codecName, org.apache.hadoop.conf.Configuration hadoopConfig)	Compresses the data using the provided Hadoop CompressionCodec and Configuration.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CompressWriterFactory

public CompressWriterFactory(Extractor<IN> extractor)

Creates a new CompressWriterFactory using the given Extractor to assemble either HadoopCompressionBulkWriter or NoCompressionBulkWriter based on whether a Hadoop CompressionCodec name is specified.

Parameters:

extractor - Extractor to extract the element

Method Detail

withHadoopCompression

public CompressWriterFactory<IN> withHadoopCompression(String codecName)
                                                throws IOException

Compresses the data using the provided Hadoop CompressionCodec.

Parameters:

codecName - Simple/complete name or alias of the CompressionCodec

Returns:

the instance of CompressionWriterFactory

Throws:

IOException

withHadoopCompression

public CompressWriterFactory<IN> withHadoopCompression(String codecName,
                                                       org.apache.hadoop.conf.Configuration hadoopConfig)
                                                throws IOException

Compresses the data using the provided Hadoop CompressionCodec and Configuration.

Parameters:

codecName - Simple/complete name or alias of the CompressionCodec

hadoopConfig - Hadoop Configuration

Returns:

the instance of CompressionWriterFactory

Throws:

IOException

create

public org.apache.flink.api.common.serialization.BulkWriter<IN> create(org.apache.flink.core.fs.FSDataOutputStream out)
                                                                throws IOException

Specified by:

create in interface org.apache.flink.api.common.serialization.BulkWriter.Factory<IN>

Throws:

IOException

getExtension

public String getExtension()