@InterfaceAudience.Public @InterfaceStability.Evolving public class PathOutputCommitterFactory extends Configured
PathOutputCommitter
methods, and so can be used from FileOutputFormat.
The base implementation returns FileOutputCommitter instances.
Algorithm:
FileOutputCommitter is
created.FileOutputFormat, the created factory has its method
createOutputCommitter(Path, TaskAttemptContext) with a task
attempt context and a possibly null path.| Modifier and Type | Field and Description |
|---|---|
static String |
COMMITTER_FACTORY_CLASS
Name of the configuration option used to configure the
output committer factory to use unless there is a specific
one for a schema.
|
static String |
COMMITTER_FACTORY_DEFAULT
Default committer factory name: "org.apache.hadoop.mapreduce.lib.output.FileOutputCommitterFactory".
|
static String |
COMMITTER_FACTORY_SCHEME
Scheme prefix for per-filesystem scheme committers.
|
static String |
COMMITTER_FACTORY_SCHEME_PATTERN
String format pattern for per-filesystem scheme committers.
|
static String |
FILE_COMMITTER_FACTORY
The
FileOutputCommitter factory. |
static String |
NAMED_COMMITTER_CLASS
The named output committer.
|
static String |
NAMED_COMMITTER_FACTORY
The
FileOutputCommitter factory. |
| Constructor and Description |
|---|
PathOutputCommitterFactory() |
| Modifier and Type | Method and Description |
|---|---|
static PathOutputCommitter |
createCommitter(Path outputPath,
TaskAttemptContext context)
Create the committer factory for a task attempt and destination, then
create the committer from it.
|
protected PathOutputCommitter |
createFileOutputCommitter(Path outputPath,
TaskAttemptContext context)
Create an instance of the default committer, a
FileOutputCommitter
for a task. |
PathOutputCommitter |
createOutputCommitter(Path outputPath,
TaskAttemptContext context)
Create an output committer for a task attempt.
|
static PathOutputCommitterFactory |
getCommitterFactory(Path outputPath,
Configuration conf)
Get the committer factory for a configuration.
|
getConf, setConfpublic static final String COMMITTER_FACTORY_CLASS
public static final String COMMITTER_FACTORY_SCHEME
public static final String COMMITTER_FACTORY_SCHEME_PATTERN
public static final String FILE_COMMITTER_FACTORY
FileOutputCommitter factory.public static final String NAMED_COMMITTER_FACTORY
FileOutputCommitter factory.public static final String NAMED_COMMITTER_CLASS
public static final String COMMITTER_FACTORY_DEFAULT
public PathOutputCommitter createOutputCommitter(Path outputPath, TaskAttemptContext context) throws IOException
outputPath - output path. This may be null.context - contextIOException - problems instantiating the committerprotected final PathOutputCommitter createFileOutputCommitter(Path outputPath, TaskAttemptContext context) throws IOException
FileOutputCommitter
for a task.outputPath - the task's output path, or or null if no output path
has been defined.context - the task attempt contextIOException - problems instantiating the committerpublic static PathOutputCommitterFactory getCommitterFactory(Path outputPath, Configuration conf)
outputPath - the job's output path. If null, it means that the
schema is unknown and a per-schema factory cannot be determined.conf - configurationpublic static PathOutputCommitter createCommitter(Path outputPath, TaskAttemptContext context) throws IOException
outputPath - the task's output path, or or null if no output path
has been defined.context - the task attempt contextIOException - problems instantiating the committerCopyright © 2024 Apache Software Foundation. All rights reserved.