public class KafkaOutputFormat<K,V>
extends org.apache.hadoop.mapreduce.OutputFormat<K,V>
Modifier and Type | Field and Description |
---|---|
static java.lang.String |
KAFKA_CONFIG_PREFIX |
static int |
KAFKA_QUEUE_BYTES
Bytes to buffer before the OutputFormat does a send (i.e., the amortization window):
We set the default to a million bytes so that the server will not reject the batch of messages
with a MessageSizeTooLargeException.
|
static java.lang.String |
KAFKA_URL |
Constructor and Description |
---|
KafkaOutputFormat() |
Modifier and Type | Method and Description |
---|---|
void |
checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext jobContext) |
org.apache.hadoop.mapreduce.OutputCommitter |
getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext taskAttemptContext) |
static org.apache.hadoop.fs.Path |
getOutputPath(org.apache.hadoop.mapreduce.JobContext job) |
org.apache.hadoop.mapreduce.RecordWriter<K,V> |
getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context) |
static void |
setOutputPath(org.apache.hadoop.mapreduce.Job job,
org.apache.hadoop.fs.Path outputUrl) |
public static final java.lang.String KAFKA_URL
public static final int KAFKA_QUEUE_BYTES
public static final java.lang.String KAFKA_CONFIG_PREFIX
public static void setOutputPath(org.apache.hadoop.mapreduce.Job job, org.apache.hadoop.fs.Path outputUrl)
public static org.apache.hadoop.fs.Path getOutputPath(org.apache.hadoop.mapreduce.JobContext job)
public void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext jobContext) throws java.io.IOException, java.lang.InterruptedException
public org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext taskAttemptContext) throws java.io.IOException, java.lang.InterruptedException