Class Stampede
- java.lang.Object
-
- edu.isi.pegasus.planner.code.generator.Stampede
-
- All Implemented Interfaces:
CodeGenerator
public class Stampede extends java.lang.Object implements CodeGenerator
A Stampede Events Code Generator that generates events in netlogger format for the exectuable workflow. This generators generates the events aboutthe tasks int he abstract workflow the edges in the abstract workflow jobs in the executable workflow the edges in the executable workflow relationship about how the tasks in the abstract workflow map to jobs in the executable workflow.
- Version:
- $Revision$
- Author:
- Karan Vahi
-
-
Field Summary
Fields Modifier and Type Field Description static java.lang.String
ARGUMENTS_KEY
The attribute key for task arguments.static java.lang.String
CHILD_JOB_ID_KEY
The atrribute key for child job id.static java.lang.String
CHILD_TASK_ID_KEY
The atrribute key for child task id.static java.lang.String
JOB_CLUSTERED_KEY
The attribute key for whether a job is clustered or notstatic java.lang.String
JOB_EDGE_EVENT_NAME
The event name for job edgestatic java.lang.String
JOB_EVENT_NAME
The event name for a jobstatic java.lang.String
JOB_EXECUTABLE_KEY
The attribute key for the executablestatic java.lang.String
JOB_ID_KEY
The attribute key for job idstatic java.lang.String
JOB_MAX_RETRIES_KEY
The attribute key for how many times a job is retriedstatic java.lang.String
JOB_SUBMIT_FILE_KEY
Teh attribute key for the submit filestatic java.lang.String
JOB_TASK_COUNT_KEY
The attribute key for the number of tasks in the jobprotected PegasusBag
mBag
The bag of initialization objects.private LogFormatter
mLogFormatter
The handle to the netlogger log formatter.protected LogManager
mLogger
The handle to the logging object.protected PlannerOptions
mPOptions
The object containing the command line options specified to the planner at runtime.protected PegasusProperties
mProps
The object holding all the properties pertaining to Pegasus.protected java.lang.String
mSubmitFileDir
The directory where all the submit files are to be generated.static java.lang.String
NETLOGGER_BP_FILE_SUFFIX
The suffix to use while constructing the name of the metrics filestatic java.lang.String
NETLOGGER_LOG_FORMATTER_IMPLEMENTOR
static java.lang.String
PARENT_JOB_ID_KEY
The atrribute key for parent job id.static java.lang.String
PARENT_TASK_ID_KEY
The atrribute key for parent task id.static java.lang.String
TASK_EDGE_EVENT_NAME
The event name for task edgestatic java.lang.String
TASK_EVENT_NAME
The event name for task infostatic java.lang.String
TASK_ID_KEY
The attribute key for task idstatic java.lang.String
TASK_MAP_EVENT_NAME
The event name for task map eventstatic java.lang.String
TASK_TRANSFORMATION_KEY
The attribute key for transformationstatic java.lang.String
TYPE_DESCRIPTION_KEY
The attribute key for type descriptionstatic java.lang.String
TYPE_KEY
The attribute key for task typestatic java.lang.String
WORKFLOW_ID_KEY
The attribute key for workflow id.-
Fields inherited from interface edu.isi.pegasus.planner.code.CodeGenerator
VERSION
-
-
Constructor Summary
Constructors Constructor Description Stampede()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description java.lang.String
booleanToInt(boolean value)
Returns boolean as an integerjava.util.Collection<java.io.File>
generateCode(ADag dag)
Generates the code for the executable workflow in terms of a braindump file that contains workflow metadata useful for monitoring daemons etc.void
generateCode(ADag dag, Job job)
Method not implemented.protected void
generateEventsForDAXTask(java.io.PrintWriter writer, ADag workflow, Job job)
Generates stampede events corresponding to jobs/tasks in the DAXprotected void
generateEventsForExecutableJob(java.io.PrintWriter writer, ADag dag, Job job)
Generates stampede events corresponding to an executable jobprotected void
generateTaskMapEvents(java.io.PrintWriter writer, ADag dag, Job job)
Generates the task.map events that link the jobs in the DAX with the jobs in the executable workflowprivate int
getTaskCount(Job job)
Returns the task count for a job.void
initialize(PegasusBag bag)
Initializes the Code Generator implementation.void
reset()
Resets the Code Generator implementation.boolean
startMonitoring()
Starts monitoring of the workflow by invoking a workflow monitor daemon.
-
-
-
Field Detail
-
NETLOGGER_BP_FILE_SUFFIX
public static final java.lang.String NETLOGGER_BP_FILE_SUFFIX
The suffix to use while constructing the name of the metrics file- See Also:
- Constant Field Values
-
NETLOGGER_LOG_FORMATTER_IMPLEMENTOR
public static final java.lang.String NETLOGGER_LOG_FORMATTER_IMPLEMENTOR
- See Also:
- Constant Field Values
-
WORKFLOW_ID_KEY
public static final java.lang.String WORKFLOW_ID_KEY
The attribute key for workflow id.- See Also:
- Constant Field Values
-
TASK_EVENT_NAME
public static final java.lang.String TASK_EVENT_NAME
The event name for task info- See Also:
- Constant Field Values
-
TASK_ID_KEY
public static final java.lang.String TASK_ID_KEY
The attribute key for task id- See Also:
- Constant Field Values
-
TYPE_KEY
public static final java.lang.String TYPE_KEY
The attribute key for task type- See Also:
- Constant Field Values
-
TYPE_DESCRIPTION_KEY
public static final java.lang.String TYPE_DESCRIPTION_KEY
The attribute key for type description- See Also:
- Constant Field Values
-
TASK_TRANSFORMATION_KEY
public static final java.lang.String TASK_TRANSFORMATION_KEY
The attribute key for transformation- See Also:
- Constant Field Values
-
ARGUMENTS_KEY
public static final java.lang.String ARGUMENTS_KEY
The attribute key for task arguments.- See Also:
- Constant Field Values
-
TASK_EDGE_EVENT_NAME
public static final java.lang.String TASK_EDGE_EVENT_NAME
The event name for task edge- See Also:
- Constant Field Values
-
PARENT_TASK_ID_KEY
public static final java.lang.String PARENT_TASK_ID_KEY
The atrribute key for parent task id.- See Also:
- Constant Field Values
-
CHILD_TASK_ID_KEY
public static final java.lang.String CHILD_TASK_ID_KEY
The atrribute key for child task id.- See Also:
- Constant Field Values
-
JOB_EVENT_NAME
public static final java.lang.String JOB_EVENT_NAME
The event name for a job- See Also:
- Constant Field Values
-
JOB_ID_KEY
public static final java.lang.String JOB_ID_KEY
The attribute key for job id- See Also:
- Constant Field Values
-
JOB_SUBMIT_FILE_KEY
public static final java.lang.String JOB_SUBMIT_FILE_KEY
Teh attribute key for the submit file- See Also:
- Constant Field Values
-
JOB_CLUSTERED_KEY
public static final java.lang.String JOB_CLUSTERED_KEY
The attribute key for whether a job is clustered or not- See Also:
- Constant Field Values
-
JOB_MAX_RETRIES_KEY
public static final java.lang.String JOB_MAX_RETRIES_KEY
The attribute key for how many times a job is retried- See Also:
- Constant Field Values
-
JOB_TASK_COUNT_KEY
public static final java.lang.String JOB_TASK_COUNT_KEY
The attribute key for the number of tasks in the job- See Also:
- Constant Field Values
-
JOB_EXECUTABLE_KEY
public static final java.lang.String JOB_EXECUTABLE_KEY
The attribute key for the executable- See Also:
- Constant Field Values
-
JOB_EDGE_EVENT_NAME
public static final java.lang.String JOB_EDGE_EVENT_NAME
The event name for job edge- See Also:
- Constant Field Values
-
PARENT_JOB_ID_KEY
public static final java.lang.String PARENT_JOB_ID_KEY
The atrribute key for parent job id.- See Also:
- Constant Field Values
-
CHILD_JOB_ID_KEY
public static final java.lang.String CHILD_JOB_ID_KEY
The atrribute key for child job id.- See Also:
- Constant Field Values
-
TASK_MAP_EVENT_NAME
public static final java.lang.String TASK_MAP_EVENT_NAME
The event name for task map event- See Also:
- Constant Field Values
-
mLogFormatter
private LogFormatter mLogFormatter
The handle to the netlogger log formatter.
-
mBag
protected PegasusBag mBag
The bag of initialization objects.
-
mSubmitFileDir
protected java.lang.String mSubmitFileDir
The directory where all the submit files are to be generated.
-
mProps
protected PegasusProperties mProps
The object holding all the properties pertaining to Pegasus.
-
mPOptions
protected PlannerOptions mPOptions
The object containing the command line options specified to the planner at runtime.
-
mLogger
protected LogManager mLogger
The handle to the logging object.
-
-
Method Detail
-
initialize
public void initialize(PegasusBag bag) throws CodeGeneratorException
Initializes the Code Generator implementation.- Specified by:
initialize
in interfaceCodeGenerator
- Parameters:
bag
- the bag of initialization objects.- Throws:
CodeGeneratorException
- in case of any error occuring code generation.
-
generateCode
public java.util.Collection<java.io.File> generateCode(ADag dag) throws CodeGeneratorException
Generates the code for the executable workflow in terms of a braindump file that contains workflow metadata useful for monitoring daemons etc.- Specified by:
generateCode
in interfaceCodeGenerator
- Parameters:
dag
- the concrete workflow.- Returns:
- the Collection of
File
objects for the files written out. - Throws:
CodeGeneratorException
- in case of any error occuring code generation.
-
generateEventsForDAXTask
protected void generateEventsForDAXTask(java.io.PrintWriter writer, ADag workflow, Job job) throws CodeGeneratorException
Generates stampede events corresponding to jobs/tasks in the DAX- Parameters:
writer
- the writer stream to write the events tooworkflow
- the workflow.job
- the job for which to generate the events.- Throws:
CodeGeneratorException
-
generateEventsForExecutableJob
protected void generateEventsForExecutableJob(java.io.PrintWriter writer, ADag dag, Job job) throws CodeGeneratorException
Generates stampede events corresponding to an executable job- Parameters:
writer
- the writer stream to write the events tooworkflow
- the workflow.job
- the job for which to generate the events.- Throws:
CodeGeneratorException
-
generateTaskMapEvents
protected void generateTaskMapEvents(java.io.PrintWriter writer, ADag dag, Job job)
Generates the task.map events that link the jobs in the DAX with the jobs in the executable workflow- Parameters:
writer
- the writer stream to write the events tooworkflow
- the workflow.job
- the job for which to generate the events.
-
generateCode
public void generateCode(ADag dag, Job job) throws CodeGeneratorException
Method not implemented. Throws an exception.- Specified by:
generateCode
in interfaceCodeGenerator
- Parameters:
dag
- the workflowjob
- the job for which the code is to be generated.- Throws:
CodeGeneratorException
-
getTaskCount
private int getTaskCount(Job job)
Returns the task count for a job. The task count is the number of tasks/jobs in the DAX that map to this job. jobs inserted by Pegasus, which do not have a mapped task from the DAX, will have its task_count set to 0.- Parameters:
job
- the executable job.- Returns:
- task count
-
booleanToInt
public java.lang.String booleanToInt(boolean value)
Returns boolean as an integer- Parameters:
value
- the boolean value- Returns:
- 0 for false and 1 for true
-
startMonitoring
public boolean startMonitoring()
Description copied from interface:CodeGenerator
Starts monitoring of the workflow by invoking a workflow monitor daemon. The monitoring should start only after the output files have been generated. FIXME: It should actually happen after the workflow has been submitted. Eventually should be a separate monitor interface, and submit writers should be loaded by an AbstractFactory.- Specified by:
startMonitoring
in interfaceCodeGenerator
- Returns:
- boolean indicating whether could successfully start the monitor daemon or not.
-
reset
public void reset() throws CodeGeneratorException
Description copied from interface:CodeGenerator
Resets the Code Generator implementation.- Specified by:
reset
in interfaceCodeGenerator
- Throws:
CodeGeneratorException
- in case of any error occuring code generation.
-
-