Package org.apache.flink.formats.parquet.vector.reader

Class TimestampColumnReader

java.lang.Object

org.apache.flink.formats.parquet.vector.reader.AbstractColumnReader<org.apache.flink.table.data.columnar.vector.writable.WritableTimestampVector>

org.apache.flink.formats.parquet.vector.reader.TimestampColumnReader

All Implemented Interfaces:

ColumnReader<org.apache.flink.table.data.columnar.vector.writable.WritableTimestampVector>

public class TimestampColumnReader
extends AbstractColumnReader<org.apache.flink.table.data.columnar.vector.writable.WritableTimestampVector>

Timestamp ColumnReader. We support INT96 and INT64 now, julianDay(4) + nanosOfDay(8). See https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#timestamp TIMESTAMP_MILLIS and TIMESTAMP_MICROS are the deprecated ConvertedType.

Field Summary

Fields

Modifier and Type	Field	Description
static int	JULIAN_EPOCH_OFFSET_DAYS
static long	MICROS_PER_MILLISECOND
static long	MICROS_PER_SECOND
static long	MILLIS_IN_DAY
static long	MILLIS_PER_SECOND
static long	NANOS_PER_MICROSECONDS
static long	NANOS_PER_MILLISECOND
static long	NANOS_PER_SECOND

Fields inherited from class org.apache.flink.formats.parquet.vector.reader.AbstractColumnReader

descriptor, dictionary, maxDefLevel, runLenDecoder

Constructor Summary

Constructors

Constructor	Description
TimestampColumnReader(boolean utcTimestamp, org.apache.parquet.column.ColumnDescriptor descriptor, org.apache.parquet.column.page.PageReader pageReader)

Method Summary

Modifier and Type	Method	Description
static org.apache.flink.table.data.TimestampData	decodeInt64ToTimestamp(boolean utcTimestamp, org.apache.parquet.column.Dictionary dictionary, int id, org.apache.parquet.schema.LogicalTypeAnnotation.TimeUnit timeUnit)
static org.apache.flink.table.data.TimestampData	decodeInt96ToTimestamp(boolean utcTimestamp, org.apache.parquet.column.Dictionary dictionary, int id)
static org.apache.flink.table.data.TimestampData	int64ToTimestamp(boolean utcTimestamp, long value, org.apache.parquet.schema.LogicalTypeAnnotation.TimeUnit timeUnit)
static org.apache.flink.table.data.TimestampData	int96ToTimestamp(boolean utcTimestamp, long nanosOfDay, int julianDay)
protected void	readBatch(int rowId, int num, org.apache.flink.table.data.columnar.vector.writable.WritableTimestampVector column)	Read batch from AbstractColumnReader.runLenDecoder and AbstractColumnReader.dataInputStream.
protected void	readBatchFromDictionaryIds(int rowId, int num, org.apache.flink.table.data.columnar.vector.writable.WritableTimestampVector column, org.apache.flink.table.data.columnar.vector.writable.WritableIntVector dictionaryIds)	Decode dictionary ids to data.
protected boolean	supportLazyDecode()	Support lazy dictionary ids decode.

Methods inherited from class org.apache.flink.formats.parquet.vector.reader.AbstractColumnReader

afterReadPage, checkTypeName, readToVector

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

JULIAN_EPOCH_OFFSET_DAYS

public static final int JULIAN_EPOCH_OFFSET_DAYS

See Also:

Constant Field Values

MILLIS_IN_DAY

public static final long MILLIS_IN_DAY

NANOS_PER_MILLISECOND

public static final long NANOS_PER_MILLISECOND

NANOS_PER_SECOND

public static final long NANOS_PER_SECOND

MICROS_PER_MILLISECOND

public static final long MICROS_PER_MILLISECOND

NANOS_PER_MICROSECONDS

public static final long NANOS_PER_MICROSECONDS

MILLIS_PER_SECOND

public static final long MILLIS_PER_SECOND

MICROS_PER_SECOND

public static final long MICROS_PER_SECOND

Constructor Detail

TimestampColumnReader

public TimestampColumnReader(boolean utcTimestamp,
                             org.apache.parquet.column.ColumnDescriptor descriptor,
                             org.apache.parquet.column.page.PageReader pageReader)
                      throws IOException

Throws:

IOException

Method Detail

supportLazyDecode

protected boolean supportLazyDecode()

Description copied from class: AbstractColumnReader

Support lazy dictionary ids decode. See more in ParquetDictionary. If return false, we will decode all the data first.

Overrides:

supportLazyDecode in class AbstractColumnReader<org.apache.flink.table.data.columnar.vector.writable.WritableTimestampVector>

readBatch

protected void readBatch(int rowId,
                         int num,
                         org.apache.flink.table.data.columnar.vector.writable.WritableTimestampVector column)

Description copied from class: AbstractColumnReader

Read batch from AbstractColumnReader.runLenDecoder and AbstractColumnReader.dataInputStream.

Specified by:

readBatch in class AbstractColumnReader<org.apache.flink.table.data.columnar.vector.writable.WritableTimestampVector>

readBatchFromDictionaryIds

protected void readBatchFromDictionaryIds(int rowId,
                                          int num,
                                          org.apache.flink.table.data.columnar.vector.writable.WritableTimestampVector column,
                                          org.apache.flink.table.data.columnar.vector.writable.WritableIntVector dictionaryIds)

Description copied from class: AbstractColumnReader

Decode dictionary ids to data. From AbstractColumnReader.runLenDecoder and AbstractColumnReader.dictionaryIdsDecoder.

Specified by:

readBatchFromDictionaryIds in class AbstractColumnReader<org.apache.flink.table.data.columnar.vector.writable.WritableTimestampVector>

decodeInt64ToTimestamp

public static org.apache.flink.table.data.TimestampData decodeInt64ToTimestamp(boolean utcTimestamp,
                                                                               org.apache.parquet.column.Dictionary dictionary,
                                                                               int id,
                                                                               org.apache.parquet.schema.LogicalTypeAnnotation.TimeUnit timeUnit)

decodeInt96ToTimestamp

public static org.apache.flink.table.data.TimestampData decodeInt96ToTimestamp(boolean utcTimestamp,
                                                                               org.apache.parquet.column.Dictionary dictionary,
                                                                               int id)

int96ToTimestamp

public static org.apache.flink.table.data.TimestampData int96ToTimestamp(boolean utcTimestamp,
                                                                         long nanosOfDay,
                                                                         int julianDay)

int64ToTimestamp

public static org.apache.flink.table.data.TimestampData int64ToTimestamp(boolean utcTimestamp,
                                                                         long value,
                                                                         org.apache.parquet.schema.LogicalTypeAnnotation.TimeUnit timeUnit)