public abstract static class PigGenericMapReduce.Reduce extends org.apache.hadoop.mapreduce.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>
| Modifier and Type | Field and Description |
|---|---|
protected boolean |
errorInReduce |
protected boolean |
initialized |
protected org.apache.commons.logging.Log |
log |
protected org.apache.hadoop.mapreduce.Reducer.Context |
outputCollector |
protected POPackage |
pack |
protected PhysicalPlan |
rp |
protected List<POStore> |
stores |
| Constructor and Description |
|---|
Reduce() |
| Modifier and Type | Method and Description |
|---|---|
protected void |
cleanup(org.apache.hadoop.mapreduce.Reducer.Context context)
Will be called once all the intermediate keys and values are
processed.
|
abstract org.apache.hadoop.mapreduce.Reducer.Context |
getIllustratorContext(org.apache.hadoop.mapred.jobcontrol.Job job,
List<Pair<PigNullableWritable,org.apache.hadoop.io.Writable>> input,
POPackage pkg)
Get reducer's illustrator context
|
abstract POPackage |
getPack(org.apache.hadoop.mapreduce.Reducer.Context context) |
abstract boolean |
inIllustrator(org.apache.hadoop.mapreduce.Reducer.Context context) |
boolean |
processOnePackageOutput(org.apache.hadoop.mapreduce.Reducer.Context oc) |
protected void |
reduce(PigNullableWritable key,
Iterable<NullableTuple> tupIter,
org.apache.hadoop.mapreduce.Reducer.Context context)
The reduce function which packages the key and List<Tuple>
into key, Bag<Tuple> after converting Hadoop type key into Pig type.
|
protected void |
runPipeline(PhysicalOperator leaf) |
void |
setReducePlan(PhysicalPlan plan)
Set the reduce plan: to be used by local runner for illustrator
|
protected void |
setup(org.apache.hadoop.mapreduce.Reducer.Context context)
Configures the Reduce plan, the POPackage operator
and the reporter thread
|
protected final org.apache.commons.logging.Log log
protected PhysicalPlan rp
protected POPackage pack
protected org.apache.hadoop.mapreduce.Reducer.Context outputCollector
protected boolean errorInReduce
protected volatile boolean initialized
public void setReducePlan(PhysicalPlan plan)
plan - Reduce planprotected void setup(org.apache.hadoop.mapreduce.Reducer.Context context)
throws IOException,
InterruptedException
setup in class org.apache.hadoop.mapreduce.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>IOExceptionInterruptedExceptionprotected void reduce(PigNullableWritable key, Iterable<NullableTuple> tupIter, org.apache.hadoop.mapreduce.Reducer.Context context) throws IOException, InterruptedException
reduce in class org.apache.hadoop.mapreduce.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>IOExceptionInterruptedExceptionpublic boolean processOnePackageOutput(org.apache.hadoop.mapreduce.Reducer.Context oc)
throws IOException,
InterruptedException
IOExceptionInterruptedExceptionprotected void runPipeline(PhysicalOperator leaf) throws InterruptedException, IOException
leaf - InterruptedExceptionIOExceptionprotected void cleanup(org.apache.hadoop.mapreduce.Reducer.Context context)
throws IOException,
InterruptedException
cleanup in class org.apache.hadoop.mapreduce.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>IOExceptionInterruptedExceptionpublic abstract org.apache.hadoop.mapreduce.Reducer.Context getIllustratorContext(org.apache.hadoop.mapred.jobcontrol.Job job,
List<Pair<PigNullableWritable,org.apache.hadoop.io.Writable>> input,
POPackage pkg)
throws IOException,
InterruptedException
input - Input buffer as output by mapspkg - packageIOExceptionInterruptedExceptionpublic abstract boolean inIllustrator(org.apache.hadoop.mapreduce.Reducer.Context context)
public abstract POPackage getPack(org.apache.hadoop.mapreduce.Reducer.Context context)
Copyright © 2007-2017 The Apache Software Foundation