Package org.apache.flink.state.forst.fs

Class ForStFlinkFileSystem

java.lang.Object

org.apache.flink.core.fs.FileSystem

org.apache.flink.state.forst.fs.ForStFlinkFileSystem

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.flink.core.fs.IFileSystem

public class ForStFlinkFileSystem
extends org.apache.flink.core.fs.FileSystem
implements Closeable

A FileSystem delegates some requests to file system loaded by Flink FileSystem mechanism.

All methods in this class maybe used by ForSt, please start a discussion firstly if it has to be modified.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	ForStFlinkFileSystem.FileStatusWrapper

Nested classes/interfaces inherited from class org.apache.flink.core.fs.FileSystem

org.apache.flink.core.fs.FileSystem.FSKey, org.apache.flink.core.fs.FileSystem.WriteMode

Constructor Summary

Constructors

Constructor	Description
ForStFlinkFileSystem(org.apache.flink.core.fs.FileSystem delegateFS, String remoteBase, String localBase, FileBasedCache fileBasedCache)

Method Summary

Modifier and Type	Method	Description
void	close()
ByteBufferWritableFSDataOutputStream	create(org.apache.flink.core.fs.Path path)	Create ByteBufferWritableFSDataOutputStream from specific path which supports to write data to ByteBuffer with FileSystem.WriteMode.OVERWRITE mode.
ByteBufferWritableFSDataOutputStream	create(org.apache.flink.core.fs.Path dbFilePath, org.apache.flink.core.fs.FileSystem.WriteMode overwriteMode)
boolean	delete(org.apache.flink.core.fs.Path path, boolean recursive)
boolean	exists(org.apache.flink.core.fs.Path f)
static ForStFlinkFileSystem	get(URI uri)	Returns a reference to the FileSystem instance for accessing the file system identified by the given URI.
static ForStFlinkFileSystem	get(URI uri, org.apache.flink.core.fs.Path localBase, FileBasedCache fileBasedCache)
org.apache.flink.core.fs.FileSystem	getDelegateFS()
static FileBasedCache	getFileBasedCache(org.apache.flink.configuration.ReadableConfig config, org.apache.flink.core.fs.Path cacheBase, org.apache.flink.core.fs.Path remoteForStPath, long cacheCapacity, long cacheReservedSize, org.apache.flink.metrics.MetricGroup metricGroup)
org.apache.flink.core.fs.BlockLocation[]	getFileBlockLocations(org.apache.flink.core.fs.FileStatus file, long start, long len)
org.apache.flink.core.fs.FileStatus	getFileStatus(org.apache.flink.core.fs.Path path)
org.apache.flink.core.fs.Path	getHomeDirectory()
MappingEntry	getMappingEntry(org.apache.flink.core.fs.Path path)
String	getRemoteBase()
URI	getUri()
org.apache.flink.core.fs.Path	getWorkingDirectory()
void	giveUpOwnership(org.apache.flink.core.fs.Path path, org.apache.flink.runtime.state.StreamStateHandle stateHandle)
boolean	isDistributedFS()
int	link(String src, org.apache.flink.core.fs.Path dst)
int	link(org.apache.flink.core.fs.Path src, org.apache.flink.core.fs.Path dst)
org.apache.flink.core.fs.FileStatus[]	listStatus(org.apache.flink.core.fs.Path path)
boolean	mkdirs(org.apache.flink.core.fs.Path path)
ByteBufferReadableFSDataInputStream	open(org.apache.flink.core.fs.Path dbFilePath)
ByteBufferReadableFSDataInputStream	open(org.apache.flink.core.fs.Path dbFilePath, int bufferSize)
void	registerReusedRestoredFile(String key, org.apache.flink.runtime.state.StreamStateHandle stateHandle, org.apache.flink.core.fs.Path dbFilePath)
boolean	rename(org.apache.flink.core.fs.Path src, org.apache.flink.core.fs.Path dst)

Methods inherited from class org.apache.flink.core.fs.FileSystem

create, create, createRecoverableWriter, createRecoverableWriter, getDefaultBlockSize, getDefaultFsUri, getLocalFileSystem, getUnguardedFileSystem, initialize, initialize, initOutPathDistFS, initOutPathLocalFS

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.core.fs.IFileSystem

canCopyPaths

Constructor Detail

ForStFlinkFileSystem

public ForStFlinkFileSystem(org.apache.flink.core.fs.FileSystem delegateFS,
                            String remoteBase,
                            String localBase,
                            @Nullable
                            FileBasedCache fileBasedCache)

Method Detail

get

public static ForStFlinkFileSystem get(URI uri)
                                throws IOException

Returns a reference to the FileSystem instance for accessing the file system identified by the given URI.

Parameters:

uri - the URI identifying the file system.

Returns:

a reference to the FileSystem instance for accessing the file system identified by the given URI.

Throws:

IOException - thrown if a reference to the file system instance could not be obtained.

get

public static ForStFlinkFileSystem get(URI uri,
                                       org.apache.flink.core.fs.Path localBase,
                                       FileBasedCache fileBasedCache)
                                throws IOException

Throws:

IOException

getFileBasedCache

public static FileBasedCache getFileBasedCache(org.apache.flink.configuration.ReadableConfig config,
                                               org.apache.flink.core.fs.Path cacheBase,
                                               org.apache.flink.core.fs.Path remoteForStPath,
                                               long cacheCapacity,
                                               long cacheReservedSize,
                                               org.apache.flink.metrics.MetricGroup metricGroup)
                                        throws IOException

Throws:

IOException

getDelegateFS

public org.apache.flink.core.fs.FileSystem getDelegateFS()

getRemoteBase

public String getRemoteBase()

create

public ByteBufferWritableFSDataOutputStream create(org.apache.flink.core.fs.Path path)
                                            throws IOException

Create ByteBufferWritableFSDataOutputStream from specific path which supports to write data to ByteBuffer with FileSystem.WriteMode.OVERWRITE mode.

Parameters:

path - The file path to write to.

Returns:

The stream to the new file at the target path.

Throws:

IOException - Thrown, if the stream could not be opened because of an I/O, or because a file already exists at that path and the write mode indicates to not overwrite the file.

create

public ByteBufferWritableFSDataOutputStream create(org.apache.flink.core.fs.Path dbFilePath,
                                                   org.apache.flink.core.fs.FileSystem.WriteMode overwriteMode)
                                            throws IOException

Specified by:

create in interface org.apache.flink.core.fs.IFileSystem

Specified by:

create in class org.apache.flink.core.fs.FileSystem

Throws:

IOException

open

public ByteBufferReadableFSDataInputStream open(org.apache.flink.core.fs.Path dbFilePath,
                                                int bufferSize)
                                         throws IOException

Specified by:

open in interface org.apache.flink.core.fs.IFileSystem

Specified by:

open in class org.apache.flink.core.fs.FileSystem

Throws:

IOException

open

public ByteBufferReadableFSDataInputStream open(org.apache.flink.core.fs.Path dbFilePath)
                                         throws IOException

Specified by:

open in interface org.apache.flink.core.fs.IFileSystem

Specified by:

open in class org.apache.flink.core.fs.FileSystem

Throws:

IOException

rename

public boolean rename(org.apache.flink.core.fs.Path src,
                      org.apache.flink.core.fs.Path dst)
               throws IOException

Specified by:

rename in interface org.apache.flink.core.fs.IFileSystem

Specified by:

rename in class org.apache.flink.core.fs.FileSystem

Throws:

IOException

getWorkingDirectory

public org.apache.flink.core.fs.Path getWorkingDirectory()

Specified by:

getWorkingDirectory in interface org.apache.flink.core.fs.IFileSystem

Specified by:

getWorkingDirectory in class org.apache.flink.core.fs.FileSystem

getHomeDirectory

public org.apache.flink.core.fs.Path getHomeDirectory()

Specified by:

getHomeDirectory in interface org.apache.flink.core.fs.IFileSystem

Specified by:

getHomeDirectory in class org.apache.flink.core.fs.FileSystem

getUri

public URI getUri()

Specified by:

getUri in interface org.apache.flink.core.fs.IFileSystem

Specified by:

getUri in class org.apache.flink.core.fs.FileSystem

exists

public boolean exists(org.apache.flink.core.fs.Path f)
               throws IOException

Specified by:

exists in interface org.apache.flink.core.fs.IFileSystem

Overrides:

exists in class org.apache.flink.core.fs.FileSystem

Throws:

IOException

getFileStatus

public org.apache.flink.core.fs.FileStatus getFileStatus(org.apache.flink.core.fs.Path path)
                                                  throws IOException

Specified by:

getFileStatus in interface org.apache.flink.core.fs.IFileSystem

Specified by:

getFileStatus in class org.apache.flink.core.fs.FileSystem

Throws:

IOException

getFileBlockLocations

public org.apache.flink.core.fs.BlockLocation[] getFileBlockLocations(org.apache.flink.core.fs.FileStatus file,
                                                                      long start,
                                                                      long len)
                                                               throws IOException

Specified by:

getFileBlockLocations in interface org.apache.flink.core.fs.IFileSystem

Specified by:

getFileBlockLocations in class org.apache.flink.core.fs.FileSystem

Throws:

IOException

listStatus

public org.apache.flink.core.fs.FileStatus[] listStatus(org.apache.flink.core.fs.Path path)
                                                 throws IOException

Specified by:

listStatus in interface org.apache.flink.core.fs.IFileSystem

Specified by:

listStatus in class org.apache.flink.core.fs.FileSystem

Throws:

IOException

delete

public boolean delete(org.apache.flink.core.fs.Path path,
                      boolean recursive)
               throws IOException

Specified by:

delete in interface org.apache.flink.core.fs.IFileSystem

Specified by:

delete in class org.apache.flink.core.fs.FileSystem

Throws:

IOException

mkdirs

public boolean mkdirs(org.apache.flink.core.fs.Path path)
               throws IOException

Specified by:

mkdirs in interface org.apache.flink.core.fs.IFileSystem

Specified by:

mkdirs in class org.apache.flink.core.fs.FileSystem

Throws:

IOException

isDistributedFS

public boolean isDistributedFS()

Specified by:

isDistributedFS in interface org.apache.flink.core.fs.IFileSystem

Specified by:

isDistributedFS in class org.apache.flink.core.fs.FileSystem

link

public int link(org.apache.flink.core.fs.Path src,
                org.apache.flink.core.fs.Path dst)
         throws IOException

Throws:

IOException

link

public int link(String src,
                org.apache.flink.core.fs.Path dst)
         throws IOException

Throws:

IOException

registerReusedRestoredFile

public void registerReusedRestoredFile(String key,
                                       org.apache.flink.runtime.state.StreamStateHandle stateHandle,
                                       org.apache.flink.core.fs.Path dbFilePath)

getMappingEntry

@Nullable
public MappingEntry getMappingEntry(org.apache.flink.core.fs.Path path)

giveUpOwnership

public void giveUpOwnership(org.apache.flink.core.fs.Path path,
                            org.apache.flink.runtime.state.StreamStateHandle stateHandle)

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Throws:

IOException