Package org.apache.flink.core.fs

Interface PathsCopyingFileSystem

All Superinterfaces:

IFileSystem

All Known Implementing Classes:

SafetyNetWrapperFileSystem

@Experimental
public interface PathsCopyingFileSystem
extends IFileSystem

An interface marking that given FileSystem have an optimised path for copying paths instead of using FSDataOutputStream or FSDataInputStream.

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static interface	PathsCopyingFileSystem.CopyRequest	A pair of source and destination to duplicate a file.

Method Summary

Modifier and Type	Method	Description
default boolean	canCopyPaths(Path source, Path destination)	Tells if this FileSystem supports an optimised way to directly copy between given paths.
void	copyFiles(List<PathsCopyingFileSystem.CopyRequest> requests, ICloseableRegistry closeableRegistry)	List of PathsCopyingFileSystem.CopyRequest to copy in batch by this PathsCopyingFileSystem.

Methods inherited from interface org.apache.flink.core.fs.IFileSystem

create, createRecoverableWriter, createRecoverableWriter, delete, exists, getFileBlockLocations, getFileStatus, getHomeDirectory, getUri, getWorkingDirectory, initOutPathDistFS, initOutPathLocalFS, isDistributedFS, listStatus, mkdirs, open, open, rename

Method Detail

copyFiles

void copyFiles(List<PathsCopyingFileSystem.CopyRequest> requests,
               ICloseableRegistry closeableRegistry)
        throws IOException

List of PathsCopyingFileSystem.CopyRequest to copy in batch by this PathsCopyingFileSystem. In case of an exception some files might have been already copied fully or partially. Caller should clean this up. Copy can be interrupted by the CloseableRegistry.

Throws:

IOException

canCopyPaths

default boolean canCopyPaths(Path source,
                             Path destination)
                      throws IOException

Description copied from interface: IFileSystem

Tells if this FileSystem supports an optimised way to directly copy between given paths. In other words if it implements PathsCopyingFileSystem.

At least one of, either source or destination belongs to this IFileSystem. One of them can point to the local file system. In other words this request can correspond to either: downloading a file from the remote file system, uploading a file to the remote file system or duplicating a file in the remote file system.

Specified by:

canCopyPaths in interface IFileSystem

Parameters:

source - The path of the source file to duplicate

destination - The path where to duplicate the source file

Returns:

true, if this IFileSystem can perform this operation more quickly compared to the generic code path of using streams.

Throws:

IOException