public class AvroTrevniOutputFormat<T> extends FileOutputFormat<AvroWrapper<T>,NullWritable>
OutputFormat that writes Avro data to
Trevni files.
Writes a directory of files per task, each comprising a single filesystem block. To reduce the number of files, increase the default filesystem block size for the job. Each task also requires enough memory to buffer a filesystem block.
FileOutputFormat.Counter| Modifier and Type | Field and Description |
|---|---|
static String |
EXT
The file name extension for trevni files.
|
static String |
META_PREFIX |
| Constructor and Description |
|---|
AvroTrevniOutputFormat() |
| Modifier and Type | Method and Description |
|---|---|
RecordWriter<AvroWrapper<T>,NullWritable> |
getRecordWriter(FileSystem ignore,
JobConf job,
String name,
Progressable prog) |
static void |
setMeta(JobConf job,
String key,
String value)
Add metadata to job output files.
|
checkOutputSpecs, getCompressOutput, getOutputCompressorClass, getOutputPath, getPathForCustomFile, getTaskOutputPath, getUniqueName, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputPath, setWorkOutputPathpublic static final String EXT
public static final String META_PREFIX
public static void setMeta(JobConf job, String key, String value)
public RecordWriter<AvroWrapper<T>,NullWritable> getRecordWriter(FileSystem ignore, JobConf job, String name, Progressable prog) throws IOException
getRecordWriter in interface OutputFormat<AvroWrapper<T>,NullWritable>getRecordWriter in class FileOutputFormat<AvroWrapper<T>,NullWritable>IOExceptionCopyright © 2009–2023 The Apache Software Foundation. All rights reserved.