Package org.apache.flink.formats.hadoop.bulk

Interface HadoopFileCommitter

  • All Known Implementing Classes:
    HadoopRenameFileCommitter
    @Internal
public interface HadoopFileCommitter
    The committer publishes an intermediate Hadoop file to the target path after it finishes writing.

    • Method Summary

      All Methods Instance Methods Abstract Methods 
      Modifier and Type Method Description
      void commit()
      Commits the in-progress file to the target path.
      void commitAfterRecovery()
      Re-commits the in-progress file to the target path after fail-over.
      org.apache.hadoop.fs.Path getTargetFilePath()
      Gets the target path to commit to.
      org.apache.hadoop.fs.Path getTempFilePath()
      Gets the path of the intermediate file to commit.
      void preCommit()
      Prepares the intermediates file for committing.

    • Method Detail

      • getTargetFilePath

        org.apache.hadoop.fs.Path getTargetFilePath()
        Gets the target path to commit to.
        Returns:
        The target path to commit to.

      • getTempFilePath

        org.apache.hadoop.fs.Path getTempFilePath()
        Gets the path of the intermediate file to commit.
        Returns:
        The path of the intermediate file to commit.

      • preCommit

        void preCommit()
        throws IOException
        Prepares the intermediates file for committing.
        Throws:
        IOException

      • commit

        void commit()
     throws IOException
        Commits the in-progress file to the target path.
        Throws:
        IOException

      • commitAfterRecovery

        void commitAfterRecovery()
                  throws IOException
        Re-commits the in-progress file to the target path after fail-over.
        Throws:
        IOException