public class LogToPhyTranslationVisitor extends LogicalRelationalNodesVisitor
| Modifier and Type | Field and Description |
|---|---|
protected PhysicalPlan |
currentPlan |
protected Deque<PhysicalPlan> |
currentPlans |
protected Map<Operator,PhysicalOperator> |
logToPhyMap |
protected NodeIdGenerator |
nodeGen |
protected PigContext |
pc |
currentWalker, DEFAULT_SCOPE, plan| Constructor and Description |
|---|
LogToPhyTranslationVisitor(OperatorPlan plan) |
| Modifier and Type | Method and Description |
|---|---|
Map<Operator,PhysicalOperator> |
getLogToPhyMap() |
PhysicalPlan |
getPhysicalPlan() |
void |
setPigContext(PigContext pc) |
static void |
updateWithEmptyBagCheck(PhysicalPlan fePlan,
Operator joinInput)
updates plan with check for empty bag and if bag is empty to flatten a bag
with as many null's as dictated by the schema
|
void |
visit(LOCogroup cg) |
void |
visit(LOCross cross) |
void |
visit(LODistinct loDistinct) |
void |
visit(LOFilter filter) |
void |
visit(LOForEach foreach) |
void |
visit(LOInnerLoad load) |
void |
visit(LOJoin loj) |
void |
visit(LOLimit loLimit) |
void |
visit(LOLoad loLoad) |
void |
visit(LONative loNative) |
void |
visit(LORank loRank)
Transformation from Logical to Physical Plan involves the following steps:
First, it is generated a random number which will link a POCounter within a PORank.
|
void |
visit(LOSort sort) |
void |
visit(LOSplit loSplit) |
void |
visit(LOSplitOutput loSplitOutput) |
void |
visit(LOStore loStore) |
void |
visit(LOStream stream) |
void |
visit(LOUnion loUnion) |
visit, visitgetPlan, popWalker, pushWalker, visitprotected Map<Operator,PhysicalOperator> logToPhyMap
protected Deque<PhysicalPlan> currentPlans
protected PhysicalPlan currentPlan
protected NodeIdGenerator nodeGen
protected PigContext pc
public LogToPhyTranslationVisitor(OperatorPlan plan) throws FrontendException
FrontendExceptionpublic void setPigContext(PigContext pc)
public Map<Operator,PhysicalOperator> getLogToPhyMap()
public PhysicalPlan getPhysicalPlan()
public void visit(LOLoad loLoad) throws FrontendException
visit in class LogicalRelationalNodesVisitorFrontendExceptionpublic void visit(LONative loNative) throws FrontendException
visit in class LogicalRelationalNodesVisitorFrontendExceptionpublic void visit(LOFilter filter) throws FrontendException
visit in class LogicalRelationalNodesVisitorFrontendExceptionpublic void visit(LOSort sort) throws FrontendException
visit in class LogicalRelationalNodesVisitorFrontendExceptionpublic void visit(LORank loRank) throws FrontendException
In case of a RANK operation (row number mode), are used two steps:
1.- Each tuple is counted sequentially on each mapper, and are produced global counters
2.- Global counters are gathered and summed, each tuple calls to the respective counter value
in order to calculate the corresponding rank value.
or not:
In case of a RANK BY operation, then are necessary five steps: 1.- Group by the fields involved on the rank operation: POPackage 2.- In case of multi-fields, the key (group field) is flatten: POForEach 3.- Sort operation by the fields available after flattening: POSort 4.- Each group is sequentially counted on each mapper through a global counter: POCounter 5.- Global counters are summed and passed to the rank operation: PORank
visit in class LogicalRelationalNodesVisitorloRank - describe if the rank operation is on a row number mode
or is rank by (dense or not)FrontendExceptionpublic void visit(LOCross cross) throws FrontendException
visit in class LogicalRelationalNodesVisitorFrontendExceptionpublic void visit(LOStream stream) throws FrontendException
visit in class LogicalRelationalNodesVisitorFrontendExceptionpublic void visit(LOInnerLoad load) throws FrontendException
visit in class LogicalRelationalNodesVisitorFrontendExceptionpublic void visit(LOForEach foreach) throws FrontendException
visit in class LogicalRelationalNodesVisitorFrontendExceptionpublic void visit(LOStore loStore) throws FrontendException
visit in class LogicalRelationalNodesVisitorFrontendExceptionpublic void visit(LOCogroup cg) throws FrontendException
visit in class LogicalRelationalNodesVisitorFrontendExceptionpublic void visit(LOJoin loj) throws FrontendException
visit in class LogicalRelationalNodesVisitorFrontendExceptionpublic void visit(LOUnion loUnion) throws FrontendException
visit in class LogicalRelationalNodesVisitorFrontendExceptionpublic void visit(LODistinct loDistinct) throws FrontendException
visit in class LogicalRelationalNodesVisitorFrontendExceptionpublic void visit(LOLimit loLimit) throws FrontendException
visit in class LogicalRelationalNodesVisitorFrontendExceptionpublic void visit(LOSplit loSplit) throws FrontendException
visit in class LogicalRelationalNodesVisitorFrontendExceptionpublic void visit(LOSplitOutput loSplitOutput) throws FrontendException
visit in class LogicalRelationalNodesVisitorFrontendExceptionpublic static void updateWithEmptyBagCheck(PhysicalPlan fePlan, Operator joinInput) throws FrontendException
fePlan - the plan to updatejoinInput - the relation for which the corresponding bag is being checkedFrontendExceptionCopyright © 2007-2017 The Apache Software Foundation