Package org.apache.flink.fs.gs.storage

Interface GSBlobStorage

  • All Known Implementing Classes:
    GSBlobStorageImpl
    public interface GSBlobStorage
    Abstract blob storage, used to simplify interface to Google storage and make it mockable.

    • Method Detail

      • writeBlob

        GSBlobStorage.WriteChannel writeBlob​(GSBlobIdentifier blobIdentifier)
        Creates a write channel with the default chunk size.
        Parameters:
        blobIdentifier - The blob identifier to which to write
        Returns:
        The WriteChannel helper

      • writeBlob

        GSBlobStorage.WriteChannel writeBlob​(GSBlobIdentifier blobIdentifier,
                                     org.apache.flink.configuration.MemorySize chunkSize)
        Creates a write channel with the specified chunk size.
        Parameters:
        blobIdentifier - The blob identifier to which to write
        chunkSize - The chunk size, must be > 0
        Returns:
        The WriteChannel helper

      • createBlob

        void createBlob​(GSBlobIdentifier blobIdentifier)
        Create an empty blob.
        Parameters:
        blobIdentifier - The blob to create

      • getMetadata

        Optional<GSBlobStorage.BlobMetadata> getMetadata​(GSBlobIdentifier blobIdentifier)
        Gets blob metadata.
        Parameters:
        blobIdentifier - The blob identifier
        Returns:
        The blob metadata, if the blob exists. Empty if the blob doesn't exist.

      • list

        List<GSBlobIdentifier> list​(String bucketName,
                            String prefix)
        Lists all the blobs in a bucket matching a given prefix.
        Parameters:
        bucketName - The bucket name
        prefix - The object prefix
        Returns:
        The found blobs ids

      • copy

        void copy​(GSBlobIdentifier sourceBlobIdentifier,
          GSBlobIdentifier targetBlobIdentifier)
        Copies from a source blob id to a target blob id. Does not delete the source blob.
        Parameters:
        sourceBlobIdentifier - The source blob identifier
        targetBlobIdentifier - The target glob identifier

      • compose

        void compose​(List<GSBlobIdentifier> sourceBlobIdentifiers,
             GSBlobIdentifier targetBlobIdentifier)
        Composes multiple blobs into one. Does not delete any of the source blobs.
        Parameters:
        sourceBlobIdentifiers - The source blob identifiers to combine, max of 32
        targetBlobIdentifier - The target blob identifier

      • delete

        List<Boolean> delete​(Iterable<GSBlobIdentifier> blobIdentifiers)
        Deletes blobs. Note that this does not fail if blobs don't exist.
        Parameters:
        blobIdentifiers - The blob identifiers to delete
        Returns:
        The results of each delete operation.