Package org.apache.flink.formats.parquet

Class ParquetVectorizedInputFormat<T,SplitT extends org.apache.flink.connector.file.src.FileSourceSplit>

java.lang.Object

org.apache.flink.formats.parquet.ParquetVectorizedInputFormat<T,SplitT>

All Implemented Interfaces:

Serializable, org.apache.flink.api.java.typeutils.ResultTypeQueryable<T>, org.apache.flink.connector.file.src.reader.BulkFormat<T,SplitT>

Direct Known Subclasses:

ParquetColumnarRowInputFormat

public abstract class ParquetVectorizedInputFormat<T,SplitT extends org.apache.flink.connector.file.src.FileSourceSplit>
extends Object
implements org.apache.flink.connector.file.src.reader.BulkFormat<T,SplitT>

Parquet BulkFormat that reads data from the file to VectorizedColumnBatch in vectorized mode.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected static class	ParquetVectorizedInputFormat.ParquetReaderBatch<T>	Reader batch that provides writing and reading capabilities.

Nested classes/interfaces inherited from interface org.apache.flink.connector.file.src.reader.BulkFormat

org.apache.flink.connector.file.src.reader.BulkFormat.Reader<T extends Object>, org.apache.flink.connector.file.src.reader.BulkFormat.RecordIterator<T extends Object>

Field Summary

Fields

Modifier and Type	Field	Description
protected SerializableConfiguration	hadoopConfig
protected boolean	isUtcTimestamp

Constructor Summary

Constructors

Constructor	Description
ParquetVectorizedInputFormat(SerializableConfiguration hadoopConfig, org.apache.flink.table.types.logical.RowType projectedType, ColumnBatchFactory<SplitT> batchFactory, int batchSize, boolean isUtcTimestamp, boolean isCaseSensitive)

Method Summary

Modifier and Type	Method	Description
org.apache.flink.formats.parquet.ParquetVectorizedInputFormat.ParquetReader	createReader(org.apache.flink.configuration.Configuration config, SplitT split)
protected abstract ParquetVectorizedInputFormat.ParquetReaderBatch<T>	createReaderBatch(org.apache.flink.table.data.columnar.vector.writable.WritableColumnVector[] writableVectors, org.apache.flink.table.data.columnar.vector.VectorizedColumnBatch columnarBatch, org.apache.flink.connector.file.src.util.Pool.Recycler<ParquetVectorizedInputFormat.ParquetReaderBatch<T>> recycler)
boolean	isSplittable()
protected int	numBatchesToCirculate(org.apache.flink.configuration.Configuration config)
org.apache.flink.formats.parquet.ParquetVectorizedInputFormat.ParquetReader	restoreReader(org.apache.flink.configuration.Configuration config, SplitT split)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.connector.file.src.reader.BulkFormat

getProducedType

Field Detail

hadoopConfig

protected final SerializableConfiguration hadoopConfig

isUtcTimestamp

protected final boolean isUtcTimestamp

Constructor Detail

ParquetVectorizedInputFormat

public ParquetVectorizedInputFormat(SerializableConfiguration hadoopConfig,
                                    org.apache.flink.table.types.logical.RowType projectedType,
                                    ColumnBatchFactory<SplitT> batchFactory,
                                    int batchSize,
                                    boolean isUtcTimestamp,
                                    boolean isCaseSensitive)

Method Detail

createReader

public org.apache.flink.formats.parquet.ParquetVectorizedInputFormat.ParquetReader createReader(org.apache.flink.configuration.Configuration config,
                                                                                                SplitT split)
                                                                                         throws IOException

Specified by:

createReader in interface org.apache.flink.connector.file.src.reader.BulkFormat<T,SplitT extends org.apache.flink.connector.file.src.FileSourceSplit>

Throws:

IOException

numBatchesToCirculate

protected int numBatchesToCirculate(org.apache.flink.configuration.Configuration config)

restoreReader

public org.apache.flink.formats.parquet.ParquetVectorizedInputFormat.ParquetReader restoreReader(org.apache.flink.configuration.Configuration config,
                                                                                                 SplitT split)
                                                                                          throws IOException

Specified by:

restoreReader in interface org.apache.flink.connector.file.src.reader.BulkFormat<T,SplitT extends org.apache.flink.connector.file.src.FileSourceSplit>

Throws:

IOException

isSplittable

public boolean isSplittable()

Specified by:

isSplittable in interface org.apache.flink.connector.file.src.reader.BulkFormat<T,SplitT extends org.apache.flink.connector.file.src.FileSourceSplit>

createReaderBatch

protected abstract ParquetVectorizedInputFormat.ParquetReaderBatch<T> createReaderBatch(org.apache.flink.table.data.columnar.vector.writable.WritableColumnVector[] writableVectors,
                                                                                        org.apache.flink.table.data.columnar.vector.VectorizedColumnBatch columnarBatch,
                                                                                        org.apache.flink.connector.file.src.util.Pool.Recycler<ParquetVectorizedInputFormat.ParquetReaderBatch<T>> recycler)

Parameters:

writableVectors - vectors to be write

columnarBatch - vectors to be read

recycler - batch recycler