Package org.apache.flink.api.java.hadoop.mapreduce

Class HadoopOutputFormatBase<K,​V,​T>

  • All Implemented Interfaces:
    Serializable, org.apache.flink.api.common.io.FinalizeOnMaster, org.apache.flink.api.common.io.OutputFormat<T>
    Direct Known Subclasses:
    HadoopOutputFormat
    @Internal
public abstract class HadoopOutputFormatBase<K,​V,​T>
extends HadoopOutputFormatCommonBase<T>
implements org.apache.flink.api.common.io.FinalizeOnMaster
    Base class shared between the Java and Scala API of Flink.
    See Also:
    Serialized Form

    • Nested Class Summary

      • Nested classes/interfaces inherited from interface org.apache.flink.api.common.io.FinalizeOnMaster

        org.apache.flink.api.common.io.FinalizeOnMaster.FinalizationContext

      • Nested classes/interfaces inherited from interface org.apache.flink.api.common.io.OutputFormat

        org.apache.flink.api.common.io.OutputFormat.InitializationContext

    • Constructor Summary

      Constructors 
      Constructor Description
      HadoopOutputFormatBase​(org.apache.hadoop.mapreduce.OutputFormat<K,​V> mapreduceOutputFormat, org.apache.hadoop.mapreduce.Job job)  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()
      commit the task by moving the output file out from the temporary directory.
      void configure​(org.apache.flink.configuration.Configuration parameters)  
      void finalizeGlobal​(org.apache.flink.api.common.io.FinalizeOnMaster.FinalizationContext context)  
      org.apache.hadoop.conf.Configuration getConfiguration()  
      void open​(org.apache.flink.api.common.io.OutputFormat.InitializationContext context)
      create the temporary output file for hadoop RecordWriter.

      • Methods inherited from class org.apache.flink.api.common.io.RichOutputFormat

        getRuntimeContext, setRuntimeContext

      • Methods inherited from interface org.apache.flink.api.common.io.OutputFormat

        writeRecord

    • Field Detail

      • OPEN_MUTEX

        protected static final Object OPEN_MUTEX

      • CONFIGURE_MUTEX

        protected static final Object CONFIGURE_MUTEX

      • CLOSE_MUTEX

        protected static final Object CLOSE_MUTEX

      • configuration

        protected org.apache.hadoop.conf.Configuration configuration

      • mapreduceOutputFormat

        protected org.apache.hadoop.mapreduce.OutputFormat<K,​V> mapreduceOutputFormat

      • recordWriter

        protected transient org.apache.hadoop.mapreduce.RecordWriter<K,​V> recordWriter

      • outputCommitter

        protected transient org.apache.hadoop.mapreduce.OutputCommitter outputCommitter

      • context

        protected transient org.apache.hadoop.mapreduce.TaskAttemptContext context

      • taskNumber

        protected transient int taskNumber

    • Constructor Detail

      • HadoopOutputFormatBase

        public HadoopOutputFormatBase​(org.apache.hadoop.mapreduce.OutputFormat<K,​V> mapreduceOutputFormat,
                              org.apache.hadoop.mapreduce.Job job)

    • Method Detail

      • getConfiguration

        public org.apache.hadoop.conf.Configuration getConfiguration()

      • configure

        public void configure​(org.apache.flink.configuration.Configuration parameters)
        Specified by:
        configure in interface org.apache.flink.api.common.io.OutputFormat<K>

      • open

        public void open​(org.apache.flink.api.common.io.OutputFormat.InitializationContext context)
          throws IOException
        create the temporary output file for hadoop RecordWriter.
        Specified by:
        open in interface org.apache.flink.api.common.io.OutputFormat<K>
        Throws:
        IOException

      • close

        public void close()
           throws IOException
        commit the task by moving the output file out from the temporary directory.
        Specified by:
        close in interface org.apache.flink.api.common.io.OutputFormat<K>
        Throws:
        IOException

      • finalizeGlobal

        public void finalizeGlobal​(org.apache.flink.api.common.io.FinalizeOnMaster.FinalizationContext context)
                    throws IOException
        Specified by:
        finalizeGlobal in interface org.apache.flink.api.common.io.FinalizeOnMaster
        Throws:
        IOException