public class StreamingCommand extends Object implements Serializable, Cloneable
StreamingCommand represents the specification of an external
command to be executed in a Pig Query.
StreamingCommand encapsulates all relevant details of the
command specified by the user either directly via the STREAM
operator or indirectly via a DEFINE operator. It includes
details such as input/output/error specifications and also files to be
shipped to the cluster and files to be cached.| Modifier and Type | Class and Description |
|---|---|
static class |
StreamingCommand.Handle
Handle to communicate with the external process.
|
static class |
StreamingCommand.HandleSpec
Specification about the usage of the
StreamingCommand.Handle to communicate
with the external process. |
| Modifier and Type | Field and Description |
|---|---|
static int |
MAX_TASKS |
| Constructor and Description |
|---|
StreamingCommand(PigContext pigContext,
String[] argv)
Create a new
StreamingCommand with the given command. |
| Modifier and Type | Method and Description |
|---|---|
void |
addHandleSpec(StreamingCommand.Handle handle,
StreamingCommand.HandleSpec handleSpec)
Attach a
StreamingCommand.HandleSpec to a given StreamingCommand.Handle |
void |
addPathToCache(String path)
Add a file to be cached on execute nodes on the cluster.
|
void |
addPathToShip(String path)
Add a file to be shipped to the cluster.
|
Object |
clone() |
List<String> |
getCacheSpecs()
Get the list of files which need to be cached on the execute nodes.
|
String[] |
getCommandArgs()
Get the parsed command arguments.
|
String |
getExecutable()
Get the command to be executed.
|
List<StreamingCommand.HandleSpec> |
getHandleSpecs(StreamingCommand.Handle handle)
Get specifications for the given
Handle. |
StreamingCommand.HandleSpec |
getInputSpec()
Get the input specification of the
StreamingCommand. |
String |
getLogDir()
Get the directory where the log-files of the command are persisted.
|
int |
getLogFilesLimit()
Get the maximum number of tasks whose stderr logs files are persisted.
|
StreamingCommand.HandleSpec |
getOutputSpec()
Get the specification of the primary output of the
StreamingCommand. |
boolean |
getPersistStderr()
Should the stderr of the managed process be persisted?
|
boolean |
getShipFiles()
Get whether files for this command should be shipped or not.
|
List<String> |
getShipSpecs()
Get the list of files which need to be shipped to the cluster.
|
void |
setCommandArgs(String[] argv)
Set the command line arguments for the
StreamingCommand. |
void |
setExecutable(String executable)
Set the executable for the
StreamingCommand. |
void |
setInputSpec(StreamingCommand.HandleSpec spec)
Set the input specification for the
StreamingCommand. |
void |
setLogDir(String logDir)
Set the directory where the log-files of the command are persisted.
|
void |
setLogFilesLimit(int logFilesLimit)
Set the maximum number of tasks whose stderr logs files are persisted.
|
void |
setOutputSpec(StreamingCommand.HandleSpec spec)
Set the specification for the primary output of the
StreamingCommand. |
void |
setPersistStderr(boolean persistStderr)
Specify if the stderr of the managed process should be persisted.
|
void |
setShipFiles(boolean shipFiles)
Set whether files should be shipped or not.
|
String |
toString() |
public static final int MAX_TASKS
public StreamingCommand(PigContext pigContext, String[] argv)
StreamingCommand with the given command.pigContext - PigContext structureargv - parsed arguments of the commandpublic String getExecutable()
public void setExecutable(String executable)
StreamingCommand.executable - the executable for the StreamingCommandpublic void setCommandArgs(String[] argv)
StreamingCommand.argv - the command line arguments for the
StreamingCommandpublic String[] getCommandArgs()
String[]public List<String> getShipSpecs()
public List<String> getCacheSpecs()
public void addPathToShip(String path) throws IOException
path - path of the file to be shipped to the clusterIOExceptionpublic void addPathToCache(String path) throws IOException
path - path of the file to be cached on the execute nodesIOExceptionpublic void addHandleSpec(StreamingCommand.Handle handle, StreamingCommand.HandleSpec handleSpec)
StreamingCommand.HandleSpec to a given StreamingCommand.Handlehandle - Handle to which the specification is to
be attached.handleSpec - HandleSpec for the given handle.public void setInputSpec(StreamingCommand.HandleSpec spec)
StreamingCommand.spec - input specificationpublic StreamingCommand.HandleSpec getInputSpec()
StreamingCommand.StreamingCommandpublic void setOutputSpec(StreamingCommand.HandleSpec spec)
StreamingCommand.spec - specification for the primary output of the
StreamingCommandpublic StreamingCommand.HandleSpec getOutputSpec()
StreamingCommand.StreamingCommandpublic List<StreamingCommand.HandleSpec> getHandleSpecs(StreamingCommand.Handle handle)
Handle.handle - Handle of the streamHandlepublic boolean getPersistStderr()
true if the stderr of the managed process should be
persisted, false otherwise.public void setPersistStderr(boolean persistStderr)
persistStderr - true if the stderr of the managed
process should be persisted, else falsepublic String getLogDir()
public void setLogDir(String logDir)
logDir - the directory where the log-files of the command are persistedpublic int getLogFilesLimit()
public void setLogFilesLimit(int logFilesLimit)
logFilesLimit - the maximum number of tasks whose stderr logs files
are persistedpublic void setShipFiles(boolean shipFiles)
shipFiles - true if files of this command should be
shipped, false otherwisepublic boolean getShipFiles()
true if files of this command should be shipped,
false otherwiseCopyright © 2007-2017 The Apache Software Foundation