Package org.apache.flink.state.forst

Class ForStOperationUtils

java.lang.Object

org.apache.flink.state.forst.ForStOperationUtils

public class ForStOperationUtils
extends Object

Utils for ForSt Operations.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	ForStOperationUtils.ForStKvStateInfo	ForSt specific information about the k/v states.

Field Summary

Fields

Modifier and Type	Field	Description
static String	MERGE_OPERATOR_NAME	The name of the merge operator in ForSt.

Constructor Summary

Constructors

Constructor	Description
ForStOperationUtils()

Method Summary

Modifier and Type	Method	Description
static void	addColumnFamilyOptionsToCloseLater(List<org.forstdb.ColumnFamilyOptions> columnFamilyOptions, org.forstdb.ColumnFamilyHandle columnFamilyHandle)
static org.apache.flink.runtime.memory.OpaqueMemoryResource<org.apache.flink.state.forst.ForStSharedResources>	allocateSharedCachesIfConfigured(ForStMemoryConfiguration jobMemoryConfig, org.apache.flink.runtime.execution.Environment env, double memoryFraction, org.slf4j.Logger logger, ForStMemoryControllerUtils.ForStMemoryFactory forStMemoryFactory)
static org.forstdb.ColumnFamilyDescriptor	createColumnFamilyDescriptor(org.apache.flink.runtime.state.RegisteredStateMetaInfoBase metaInfoBase, Function<String,org.forstdb.ColumnFamilyOptions> columnFamilyOptionsFactory, ForStDBTtlCompactFiltersManager ttlCompactFiltersManager, Long writeBufferManagerCapacity)	Creates a column descriptor for a state column family.
static org.forstdb.ColumnFamilyOptions	createColumnFamilyOptions(Function<String,org.forstdb.ColumnFamilyOptions> columnFamilyOptionsFactory, String stateName)
static ForStOperationUtils.ForStKvStateInfo	createStateInfo(org.apache.flink.runtime.state.RegisteredStateMetaInfoBase metaInfoBase, org.forstdb.RocksDB db, Function<String,org.forstdb.ColumnFamilyOptions> columnFamilyOptionsFactory, ForStDBTtlCompactFiltersManager ttlCompactFiltersManager, Long writeBufferManagerCapacity, List<org.forstdb.ExportImportFilesMetaData> importFilesMetaData, org.apache.flink.core.fs.ICloseableRegistry cancelStreamRegistryForRestore)	Creates a state info from a new meta info to use with a k/v state.
static ForStOperationUtils.ForStKvStateInfo	createStateInfo(org.apache.flink.runtime.state.RegisteredStateMetaInfoBase metaInfoBase, org.forstdb.RocksDB db, Function<String,org.forstdb.ColumnFamilyOptions> columnFamilyOptionsFactory, ForStDBTtlCompactFiltersManager ttlCompactFiltersManager, Long writeBufferManagerCapacity, org.apache.flink.core.fs.ICloseableRegistry cancelStreamRegistryForRestore)	Create RocksDB-backed KV-state, including RocksDB ColumnFamily.
static ForStIteratorWrapper	getForStIterator(org.forstdb.RocksDB db, org.forstdb.ColumnFamilyHandle columnFamilyHandle, org.forstdb.ReadOptions readOptions)
static org.forstdb.RocksDB	openDB(String path, List<org.forstdb.ColumnFamilyDescriptor> stateColumnFamilyDescriptors, List<org.forstdb.ColumnFamilyHandle> stateColumnFamilyHandles, org.forstdb.ColumnFamilyOptions columnFamilyOptions, org.forstdb.DBOptions dbOptions)
static void	registerKvStateInformation(Map<String,ForStOperationUtils.ForStKvStateInfo> kvStateInformation, ForStNativeMetricMonitor nativeMetricMonitor, String columnFamilyName, ForStOperationUtils.ForStKvStateInfo registeredColumn)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MERGE_OPERATOR_NAME

public static final String MERGE_OPERATOR_NAME

The name of the merge operator in ForSt. Do not change except you know exactly what you do.

See Also:

Constant Field Values

Constructor Detail

ForStOperationUtils

public ForStOperationUtils()

Method Detail

openDB

public static org.forstdb.RocksDB openDB(String path,
                                         List<org.forstdb.ColumnFamilyDescriptor> stateColumnFamilyDescriptors,
                                         List<org.forstdb.ColumnFamilyHandle> stateColumnFamilyHandles,
                                         org.forstdb.ColumnFamilyOptions columnFamilyOptions,
                                         org.forstdb.DBOptions dbOptions)
                                  throws IOException

Throws:

IOException

createColumnFamilyOptions

public static org.forstdb.ColumnFamilyOptions createColumnFamilyOptions(Function<String,org.forstdb.ColumnFamilyOptions> columnFamilyOptionsFactory,
                                                                        String stateName)

allocateSharedCachesIfConfigured

@Nullable
public static org.apache.flink.runtime.memory.OpaqueMemoryResource<org.apache.flink.state.forst.ForStSharedResources> allocateSharedCachesIfConfigured(ForStMemoryConfiguration jobMemoryConfig,
                                                                                                                                                       org.apache.flink.runtime.execution.Environment env,
                                                                                                                                                       double memoryFraction,
                                                                                                                                                       org.slf4j.Logger logger,
                                                                                                                                                       ForStMemoryControllerUtils.ForStMemoryFactory forStMemoryFactory)
                                                                                                                                                throws IOException

Throws:

IOException

getForStIterator

public static ForStIteratorWrapper getForStIterator(org.forstdb.RocksDB db,
                                                    org.forstdb.ColumnFamilyHandle columnFamilyHandle,
                                                    org.forstdb.ReadOptions readOptions)

addColumnFamilyOptionsToCloseLater

public static void addColumnFamilyOptionsToCloseLater(List<org.forstdb.ColumnFamilyOptions> columnFamilyOptions,
                                                      org.forstdb.ColumnFamilyHandle columnFamilyHandle)

createStateInfo

public static ForStOperationUtils.ForStKvStateInfo createStateInfo(org.apache.flink.runtime.state.RegisteredStateMetaInfoBase metaInfoBase,
                                                                   org.forstdb.RocksDB db,
                                                                   Function<String,org.forstdb.ColumnFamilyOptions> columnFamilyOptionsFactory,
                                                                   @Nullable
                                                                   ForStDBTtlCompactFiltersManager ttlCompactFiltersManager,
                                                                   @Nullable
                                                                   Long writeBufferManagerCapacity,
                                                                   List<org.forstdb.ExportImportFilesMetaData> importFilesMetaData,
                                                                   org.apache.flink.core.fs.ICloseableRegistry cancelStreamRegistryForRestore)

Creates a state info from a new meta info to use with a k/v state.

Creates the column family for the state. Sets TTL compaction filter if ttlCompactFiltersManager is not null.

Parameters:

importFilesMetaData - if not empty, we import the files specified in the metadata to the column family.

createStateInfo

public static ForStOperationUtils.ForStKvStateInfo createStateInfo(org.apache.flink.runtime.state.RegisteredStateMetaInfoBase metaInfoBase,
                                                                   org.forstdb.RocksDB db,
                                                                   Function<String,org.forstdb.ColumnFamilyOptions> columnFamilyOptionsFactory,
                                                                   @Nullable
                                                                   ForStDBTtlCompactFiltersManager ttlCompactFiltersManager,
                                                                   @Nullable
                                                                   Long writeBufferManagerCapacity,
                                                                   org.apache.flink.core.fs.ICloseableRegistry cancelStreamRegistryForRestore)

Create RocksDB-backed KV-state, including RocksDB ColumnFamily.

Parameters:

cancelStreamRegistryForRestore - closing it interrupts KV state creation

createColumnFamilyDescriptor

public static org.forstdb.ColumnFamilyDescriptor createColumnFamilyDescriptor(org.apache.flink.runtime.state.RegisteredStateMetaInfoBase metaInfoBase,
                                                                              Function<String,org.forstdb.ColumnFamilyOptions> columnFamilyOptionsFactory,
                                                                              @Nullable
                                                                              ForStDBTtlCompactFiltersManager ttlCompactFiltersManager,
                                                                              @Nullable
                                                                              Long writeBufferManagerCapacity)

Creates a column descriptor for a state column family.

Sets TTL compaction filter if ttlCompactFiltersManager is not null.

registerKvStateInformation

public static void registerKvStateInformation(Map<String,ForStOperationUtils.ForStKvStateInfo> kvStateInformation,
                                              ForStNativeMetricMonitor nativeMetricMonitor,
                                              String columnFamilyName,
                                              ForStOperationUtils.ForStKvStateInfo registeredColumn)