Package org.apache.flink.state.forst.datatransfer

Class ForStStateDataTransfer

java.lang.Object

org.apache.flink.state.forst.datatransfer.ForStStateDataTransfer

All Implemented Interfaces:

Closeable, AutoCloseable

public class ForStStateDataTransfer
extends Object
implements Closeable

Data transfer util class for ForStKeyedStateBackend.

Field Summary

Fields

Modifier and Type	Field	Description
static int	DEFAULT_THREAD_NUM
protected ExecutorService	executorService

Constructor Summary

Constructors

Constructor	Description
ForStStateDataTransfer(int threadNum)
ForStStateDataTransfer(int threadNum, ForStFlinkFileSystem forStFs)

Method Summary

Modifier and Type	Method	Description
void	close()
void	transferAllStateDataToDirectory(ForStPathContainer forStPathContainer, Collection<StateHandleTransferSpec> transferSpecs, org.apache.flink.core.fs.CloseableRegistry closeableRegistry, org.apache.flink.core.execution.RecoveryClaimMode recoveryClaimMode)	Transfer all state data to the target directory, as specified in the transfer requests.
List<org.apache.flink.runtime.state.IncrementalKeyedStateHandle.HandleAndLocalPath>	transferFilesToCheckpointFs(org.apache.flink.runtime.checkpoint.SnapshotType.SharingFilesStrategy sharingFilesStrategy, List<org.apache.flink.core.fs.Path> files, org.apache.flink.runtime.state.CheckpointStreamFactory checkpointStreamFactory, org.apache.flink.runtime.state.CheckpointedStateScope stateScope, org.apache.flink.core.fs.CloseableRegistry closeableRegistry, org.apache.flink.core.fs.CloseableRegistry tmpResourcesRegistry, boolean forceCopy)	Transfer a batch of files to checkpoint filesystem.
org.apache.flink.runtime.state.IncrementalKeyedStateHandle.HandleAndLocalPath	transferFileToCheckpointFs(org.apache.flink.runtime.checkpoint.SnapshotType.SharingFilesStrategy sharingFilesStrategy, org.apache.flink.core.fs.Path file, long transferBytes, org.apache.flink.runtime.state.CheckpointStreamFactory checkpointStreamFactory, org.apache.flink.runtime.state.CheckpointedStateScope stateScope, org.apache.flink.core.fs.CloseableRegistry snapshotCloseableRegistry, org.apache.flink.core.fs.CloseableRegistry tmpResourcesRegistry, boolean forceCopy)	Transfer a single file to checkpoint filesystem.
org.apache.flink.runtime.state.IncrementalKeyedStateHandle.HandleAndLocalPath	writeFileToCheckpointFs(String filename, String fileContent, org.apache.flink.runtime.state.CheckpointStreamFactory checkpointStreamFactory, org.apache.flink.runtime.state.CheckpointedStateScope stateScope, org.apache.flink.core.fs.CloseableRegistry closeableRegistry, org.apache.flink.core.fs.CloseableRegistry tmpResourcesRegistry)	Write a file to checkpoint filesystem.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_THREAD_NUM

public static final int DEFAULT_THREAD_NUM

See Also:

Constant Field Values

executorService

protected final ExecutorService executorService

Constructor Detail

ForStStateDataTransfer

public ForStStateDataTransfer(int threadNum)

ForStStateDataTransfer

public ForStStateDataTransfer(int threadNum,
                              ForStFlinkFileSystem forStFs)

Method Detail

transferFileToCheckpointFs

public org.apache.flink.runtime.state.IncrementalKeyedStateHandle.HandleAndLocalPath transferFileToCheckpointFs(org.apache.flink.runtime.checkpoint.SnapshotType.SharingFilesStrategy sharingFilesStrategy,
                                                                                                                org.apache.flink.core.fs.Path file,
                                                                                                                long transferBytes,
                                                                                                                org.apache.flink.runtime.state.CheckpointStreamFactory checkpointStreamFactory,
                                                                                                                org.apache.flink.runtime.state.CheckpointedStateScope stateScope,
                                                                                                                org.apache.flink.core.fs.CloseableRegistry snapshotCloseableRegistry,
                                                                                                                org.apache.flink.core.fs.CloseableRegistry tmpResourcesRegistry,
                                                                                                                boolean forceCopy)
                                                                                                         throws Exception

Transfer a single file to checkpoint filesystem.

Parameters:

transferBytes - Bytes will be transfer from the head of the file. If < 0, the whole file will be transferred.

Throws:

Exception

transferFilesToCheckpointFs

public List<org.apache.flink.runtime.state.IncrementalKeyedStateHandle.HandleAndLocalPath> transferFilesToCheckpointFs(org.apache.flink.runtime.checkpoint.SnapshotType.SharingFilesStrategy sharingFilesStrategy,
                                                                                                                       List<org.apache.flink.core.fs.Path> files,
                                                                                                                       org.apache.flink.runtime.state.CheckpointStreamFactory checkpointStreamFactory,
                                                                                                                       org.apache.flink.runtime.state.CheckpointedStateScope stateScope,
                                                                                                                       org.apache.flink.core.fs.CloseableRegistry closeableRegistry,
                                                                                                                       org.apache.flink.core.fs.CloseableRegistry tmpResourcesRegistry,
                                                                                                                       boolean forceCopy)
                                                                                                                throws Exception

Transfer a batch of files to checkpoint filesystem.

Throws:

Exception

writeFileToCheckpointFs

public org.apache.flink.runtime.state.IncrementalKeyedStateHandle.HandleAndLocalPath writeFileToCheckpointFs(String filename,
                                                                                                             String fileContent,
                                                                                                             org.apache.flink.runtime.state.CheckpointStreamFactory checkpointStreamFactory,
                                                                                                             org.apache.flink.runtime.state.CheckpointedStateScope stateScope,
                                                                                                             org.apache.flink.core.fs.CloseableRegistry closeableRegistry,
                                                                                                             org.apache.flink.core.fs.CloseableRegistry tmpResourcesRegistry)
                                                                                                      throws IOException

Write a file to checkpoint filesystem.

Throws:

IOException

transferAllStateDataToDirectory

public void transferAllStateDataToDirectory(ForStPathContainer forStPathContainer,
                                            Collection<StateHandleTransferSpec> transferSpecs,
                                            org.apache.flink.core.fs.CloseableRegistry closeableRegistry,
                                            org.apache.flink.core.execution.RecoveryClaimMode recoveryClaimMode)
                                     throws Exception

Transfer all state data to the target directory, as specified in the transfer requests.

Parameters:

transferSpecs - the list of transfers.

Throws:

Exception - If anything about the transfer goes wrong.

close

public void close()

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable