Package edu.isi.pegasus.planner.classes
Class ADag
- java.lang.Object
-
- edu.isi.pegasus.planner.classes.Data
-
- edu.isi.pegasus.planner.classes.ADag
-
- All Implemented Interfaces:
Graph
,GraphNodeContent
,java.lang.Cloneable
public class ADag extends Data implements Graph
This class object contains the info about a Dag. DagInfo object contains the information to create the .dax file. vJobSubInfos is a Vector containing Job objects of jobs making the Dag. Each subinfo object contains information needed to generate a submit file for that job.
-
-
Field Summary
Fields Modifier and Type Field Description private DagInfo
mDAGInfo
The DagInfo object which contains the information got from parsing the dax file.private Graph
mGraphImplementor
Handle to the Graph implementor.protected Notifications
mNotifications
All the notifications associated with the jobprotected ReplicaStore
mReplicaStore
Handle to the replica store that stores the replica catalog user specifies in the DAXprivate java.lang.String
mRequestID
The optional request ID associated with the DAX.protected java.lang.String
mRootWorkflowUUID
The Root Workflow UUID.private java.lang.String
mSubmitDirectory
The root of the submit directory hierarchy for the DAG.protected TransformationStore
mTransformationStore
Handle to the transformation store that stores the transformation catalog user specifies in the DAXprotected boolean
mWorkflowRefinementStarted
Boolean indicating whether the refinement process on the workflow has started or not.protected java.lang.String
mWorkflowUUID
The UUID associated with the workflow.
-
Constructor Summary
Constructors Constructor Description ADag()
Initialises the class member variables.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
add(Job job)
This adds a new job to the ADAG object.void
addEdge(GraphNode parent, GraphNode child)
Adds an edge between two already existing nodes in the graph.void
addEdge(java.lang.String parent, java.lang.String child)
Adds an edge between two already existing nodes in the graph.void
addEdges(java.lang.String child, java.util.List<java.lang.String> parents)
A convenience method that allows for bulk addition of edges between already existing nodes in the graph.void
addNewRelation(java.lang.String parent, java.lang.String child)
Adds a new PCRelation pair to the Vector ofPCRelation
pairs.void
addNode(GraphNode node)
Adds a node to the Graph.void
addNotification(Invoke invoke)
Adds a Invoke object corresponding to a notification.void
addNotifications(Notifications invokes)
Adds all the notifications passed to the underlying container.void
addRoot(GraphNode root)
Adds a single root node to the Graph.java.util.Iterator<GraphNode>
bottomUpIterator()
Returns an iterator that traverses the graph bottom up from the leaves.void
checkForCorruption()
Checks the underlying graph structure for any corruption.java.lang.Object
clone()
Returns a new copy of the Object.private java.lang.String
complain(java.lang.String desc, GraphNode node, GraphNode linkedNode)
Convenience method to complain for a linked node from a node that does not exist in the DAGprivate void
edgeToDOT(java.io.Writer stream, java.lang.String indent, java.lang.String parent, java.lang.String child)
Returns the DOT description of the object.void
generateFlowID()
Generates the flow id for this current run.void
generateFlowName()
Generates the name of the flow.protected java.lang.String
generateWorkflowUUID()
Generates the UUID for the workflowjava.lang.String
getAbstractWorkflowName()
Returns the workflow idjava.lang.String
getBaseSubmitDirectory()
Returns the root of submit directory hierarchy for the workflow.java.lang.String
getCount()
Returns the count of the workflow, that was specified in the DAX.DagInfo
getDAGInfo()
Returns the DAGInfo that stores the metadata about the DAXjava.lang.String
getDAXVersion()
Returns the dax versionjava.lang.String
getExecutableWorkflowName()
Returns the workflow idjava.lang.String
getFlowID()
Returns the flow ID for the workflow.java.lang.String
getFlowName()
Returns the flow namejava.lang.String
getFlowTimestamp()
Returns the flow timestamp for the workflow.java.lang.String
getIndex()
Returns the index of the workflow, that was specified in the DAX.java.lang.String
getLabel()
Returns the label of the workflow, that was specified in the DAX.java.util.List<GraphNode>
getLeaves()
Returns the leaf nodes of the Graph.java.lang.String
getMTime()
Returns the last modified time for the file containing the workflow description.GraphNode
getNode(java.lang.String identifier)
Returns the node matching the id passed.int
getNoOfJobs()
Returns the number of jobs in the dag on the basis of number of elements in thedagJobs
Vector.Notifications
getNotifications()
Returns all the notifications associated with the job.java.util.Collection<Invoke>
getNotifications(Invoke.WHEN when)
Returns a collection of all the notifications that need to be done for a particular conditionjava.lang.String
getReleaseVersion()
Return the release versionReplicaStore
getReplicaStore()
Returns the Replica Storejava.lang.String
getRequestID()
Returns the request id.java.util.List<GraphNode>
getRoots()
Returns the root nodes of the Graph.java.lang.String
getRootWorkflowUUID()
Returns the UUID for the Root workflowTransformationStore
getTransformationStore()
Returns the Transformation StoreWorkflowMetrics
getWorkflowMetrics()
Returns the metrics about the workflow.java.lang.String
getWorkflowUUID()
Returns the UUID for the workflowboolean
hasWorkflowRefinementStarted()
Returns a boolean indicating whether the workflow refinement has started or notboolean
isEmpty()
Returns a boolean if there are no nodes in the graph.java.util.Iterator<GraphNode>
iterator()
Returns an iterator that traverses through the graph using a graph traversal algorithm.java.util.Iterator<GraphNode>
jobIterator()
Returns an iterator for traversing through the jobs in the workflow.java.util.Iterator<GraphNode>
nodeIterator()
Returns an iterator for the nodes in the Graph.boolean
remove(Job job)
Removes a particular job from the workflow.boolean
remove(java.lang.String identifier)
Removes a node from the Graph.void
resetEdges()
Resets all the dependencies in the Graph, while preserving the nodes.void
resetStores()
Resets the replica and transformation stores;void
setBaseSubmitDirectory(java.lang.String dir)
Sets the submit directory for the workflow.void
setCount(java.lang.String count)
Set the count of the workflow, that was specified in the DAX.void
setDAXMTime(java.io.File f)
Sets the mtime (last modified time) for the DAX.void
setDAXVersion(java.lang.String version)
Sets the dax versionvoid
setFlowTimestamp(java.lang.String timestamp)
Sets the flow timestamp for the workflow.void
setIndex(java.lang.String index)
Set the index of the workflow, that was specified in the DAX.void
setLabel(java.lang.String label)
Sets the label for the workflow.void
setReleaseVersion()
Grabs the release version from VDS.Properties file.void
setReplicaStore(ReplicaStore store)
Sets the Replica Storevoid
setRequestID(java.lang.String id)
Sets the request id.void
setRootWorkflowUUID(java.lang.String uuid)
Sets the root UUID for the workflowvoid
setTransformationStore(TransformationStore store)
Sets the Transformation Storevoid
setWorkflowRefinementStarted(boolean state)
Sets whether the workflow refinement has started or notvoid
setWorkflowUUID(java.lang.String uuid)
Sets the UUID for the workflowint
size()
Returns the number of nodes in the graph.java.lang.String
toDOT()
Returns the DOT description of the object.void
toDOT(java.io.Writer stream, java.lang.String indent)
Returns the DOT description of the object.java.util.Iterator<GraphNode>
topologicalSortIterator()
Returns an iterator for the graph that traverses in topological sort order.java.lang.String
toString()
Returns the String description of the dag associated with this object.void
writeDOTHeader(java.io.Writer stream, java.lang.String indent)
Writes out the static DOT Header.-
Methods inherited from class edu.isi.pegasus.planner.classes.Data
setToString, vectorToString
-
-
-
-
Field Detail
-
mDAGInfo
private DagInfo mDAGInfo
The DagInfo object which contains the information got from parsing the dax file.
-
mSubmitDirectory
private java.lang.String mSubmitDirectory
The root of the submit directory hierarchy for the DAG. This is the directory where generally the DAG related files like the log files, .dag and dagman output files reside.
-
mRequestID
private java.lang.String mRequestID
The optional request ID associated with the DAX.
-
mReplicaStore
protected ReplicaStore mReplicaStore
Handle to the replica store that stores the replica catalog user specifies in the DAX
-
mTransformationStore
protected TransformationStore mTransformationStore
Handle to the transformation store that stores the transformation catalog user specifies in the DAX
-
mRootWorkflowUUID
protected java.lang.String mRootWorkflowUUID
The Root Workflow UUID.
-
mWorkflowUUID
protected java.lang.String mWorkflowUUID
The UUID associated with the workflow.
-
mWorkflowRefinementStarted
protected boolean mWorkflowRefinementStarted
Boolean indicating whether the refinement process on the workflow has started or not.
-
mNotifications
protected Notifications mNotifications
All the notifications associated with the job
-
mGraphImplementor
private Graph mGraphImplementor
Handle to the Graph implementor.
-
-
Method Detail
-
addNotification
public void addNotification(Invoke invoke)
Adds a Invoke object corresponding to a notification.- Parameters:
invoke
- the invoke object containing the notification
-
addNotifications
public void addNotifications(Notifications invokes)
Adds all the notifications passed to the underlying container.- Parameters:
invokes
- the notifications to be added
-
getNotifications
public java.util.Collection<Invoke> getNotifications(Invoke.WHEN when)
Returns a collection of all the notifications that need to be done for a particular condition- Parameters:
when
- the condition- Returns:
-
getNotifications
public Notifications getNotifications()
Returns all the notifications associated with the job.- Returns:
- the notifications
-
resetStores
public void resetStores()
Resets the replica and transformation stores;
-
clone
public java.lang.Object clone()
Returns a new copy of the Object.- Overrides:
clone
in classjava.lang.Object
- Returns:
- the clone of the object.
-
getRootWorkflowUUID
public java.lang.String getRootWorkflowUUID()
Returns the UUID for the Root workflow- Returns:
- the UUID of the workflow
-
setRootWorkflowUUID
public void setRootWorkflowUUID(java.lang.String uuid)
Sets the root UUID for the workflow- Parameters:
uuid
- the UUID of the workflow
-
getWorkflowUUID
public java.lang.String getWorkflowUUID()
Returns the UUID for the workflow- Returns:
- the UUID of the workflow
-
setWorkflowUUID
public void setWorkflowUUID(java.lang.String uuid)
Sets the UUID for the workflow- Parameters:
uuid
- the UUID of the workflow
-
generateWorkflowUUID
protected java.lang.String generateWorkflowUUID()
Generates the UUID for the workflow- Returns:
- the UUID of the workflow
-
hasWorkflowRefinementStarted
public boolean hasWorkflowRefinementStarted()
Returns a boolean indicating whether the workflow refinement has started or not- Returns:
- boolean
-
setWorkflowRefinementStarted
public void setWorkflowRefinementStarted(boolean state)
Sets whether the workflow refinement has started or not- Parameters:
state
- the boolean value
-
toString
public java.lang.String toString()
Returns the String description of the dag associated with this object.
-
add
public void add(Job job)
This adds a new job to the ADAG object. It ends up adding both the job name and the job description to the internal structure.- Parameters:
job
- the new job that is to be added to the ADag.
-
remove
public boolean remove(Job job)
Removes a particular job from the workflow. It however does not delete the relations the edges that refer to the job.- Parameters:
job
- theJob
object containing the job description.- Returns:
- boolean indicating whether the removal was successful or not.
-
getNoOfJobs
public int getNoOfJobs()
Returns the number of jobs in the dag on the basis of number of elements in thedagJobs
Vector.- Returns:
- the number of jobs.
-
setRequestID
public void setRequestID(java.lang.String id)
Sets the request id.- Parameters:
id
- the request id.
-
getRequestID
public java.lang.String getRequestID()
Returns the request id.- Returns:
- the request id.
-
getAbstractWorkflowName
public java.lang.String getAbstractWorkflowName()
Returns the workflow id- Returns:
- the abstract workflow id
-
getExecutableWorkflowName
public java.lang.String getExecutableWorkflowName()
Returns the workflow id- Returns:
- the executable workflow id
-
addNewRelation
public void addNewRelation(java.lang.String parent, java.lang.String child)
Adds a new PCRelation pair to the Vector ofPCRelation
pairs. For the new relation the isDeleted parameter is set to false.- Parameters:
parent
- The parent in the relation pairchild
- The child in the relation pair
-
setBaseSubmitDirectory
public void setBaseSubmitDirectory(java.lang.String dir)
Sets the submit directory for the workflow.- Parameters:
dir
- the submit directory.
-
getLabel
public java.lang.String getLabel()
Returns the label of the workflow, that was specified in the DAX.- Returns:
- the label of the workflow.
-
setLabel
public void setLabel(java.lang.String label)
Sets the label for the workflow.- Parameters:
label
- the label to be assigned to the workflow
-
getIndex
public java.lang.String getIndex()
Returns the index of the workflow, that was specified in the DAX.- Returns:
- the index of the workflow.
-
setIndex
public void setIndex(java.lang.String index)
Set the index of the workflow, that was specified in the DAX.- Parameters:
index
- the count
-
setCount
public void setCount(java.lang.String count)
Set the count of the workflow, that was specified in the DAX.- Parameters:
count
- the count
-
getCount
public java.lang.String getCount()
Returns the count of the workflow, that was specified in the DAX.- Returns:
- the count
-
getDAXVersion
public java.lang.String getDAXVersion()
Returns the dax version- Returns:
- teh dax version.
-
getMTime
public java.lang.String getMTime()
Returns the last modified time for the file containing the workflow description.- Returns:
- the MTime
-
getBaseSubmitDirectory
public java.lang.String getBaseSubmitDirectory()
Returns the root of submit directory hierarchy for the workflow.- Returns:
- the directory.
-
checkForCorruption
public void checkForCorruption()
Checks the underlying graph structure for any corruption. Corruption can be where a parent or a child of a node refers to an object, that is not in underlying graph node list.- Throws:
java.lang.RuntimeException
- in case of corruption.
-
complain
private java.lang.String complain(java.lang.String desc, GraphNode node, GraphNode linkedNode)
Convenience method to complain for a linked node from a node that does not exist in the DAG- Parameters:
desc
-node
-linkedNode
-
-
setReplicaStore
public void setReplicaStore(ReplicaStore store)
Sets the Replica Store- Parameters:
store
- the Replica Store
-
getReplicaStore
public ReplicaStore getReplicaStore()
Returns the Replica Store- Returns:
- the Replica Store
-
setTransformationStore
public void setTransformationStore(TransformationStore store)
Sets the Transformation Store- Parameters:
store
- the Transformation Store
-
getTransformationStore
public TransformationStore getTransformationStore()
Returns the Transformation Store- Returns:
- the Replica Store
-
getDAGInfo
public DagInfo getDAGInfo()
Returns the DAGInfo that stores the metadata about the DAX- Returns:
-
generateFlowID
public void generateFlowID()
Generates the flow id for this current run. It is made of the name of the dag and a timestamp. This is a simple concat of the mFlowTimestamp and the flowName. For it work correctly the function needs to be called after the flow name and timestamp have been generated.
-
getFlowID
public java.lang.String getFlowID()
Returns the flow ID for the workflow.- Returns:
-
generateFlowName
public void generateFlowName()
Generates the name of the flow. It is same as the mNameOfADag if specified in dax generated by Chimera.
-
getFlowName
public java.lang.String getFlowName()
Returns the flow name
-
setDAXVersion
public void setDAXVersion(java.lang.String version)
Sets the dax version- Parameters:
version
- the version of the DAX
-
setDAXMTime
public void setDAXMTime(java.io.File f)
Sets the mtime (last modified time) for the DAX. It is the time, when the DAX file was last modified. If the DAX file does not exist or an IO error occurs, the MTime is set to OL i.e . The DAX mTime is always generated in an extended format. Generating not in extended format, leads to the XML parser tripping while parsing the invocation record generated by Kickstart.- Parameters:
f
- the file descriptor to the DAX|PDAX file.
-
getReleaseVersion
public java.lang.String getReleaseVersion()
Return the release version
-
setReleaseVersion
public void setReleaseVersion()
Grabs the release version from VDS.Properties file.
-
getFlowTimestamp
public java.lang.String getFlowTimestamp()
Returns the flow timestamp for the workflow.- Returns:
- the flowtimestamp
-
setFlowTimestamp
public void setFlowTimestamp(java.lang.String timestamp)
Sets the flow timestamp for the workflow.- Parameters:
timestamp
- the flowtimestamp
-
jobIterator
public java.util.Iterator<GraphNode> jobIterator()
Returns an iterator for traversing through the jobs in the workflow.- Returns:
- a bative java failsafe iterator to the underlying collection.
-
getWorkflowMetrics
public WorkflowMetrics getWorkflowMetrics()
Returns the metrics about the workflow.- Returns:
- the WorkflowMetrics
-
toDOT
public java.lang.String toDOT() throws java.io.IOException
Returns the DOT description of the object. This is used for visualizing the workflow.- Returns:
- String containing the Partition object in XML.
- Throws:
java.io.IOException
- if something fishy happens to the stream.
-
toDOT
public void toDOT(java.io.Writer stream, java.lang.String indent) throws java.io.IOException
Returns the DOT description of the object. This is used for visualizing the workflow.- Parameters:
stream
- is a stream opened and ready for writing. This can also be a StringWriter for efficient output.indent
- is aString
of spaces used for pretty printing. The initial amount of spaces should be an empty string. The parameter is used internally for the recursive traversal.- Throws:
java.io.IOException
- if something fishy happens to the stream.
-
edgeToDOT
private void edgeToDOT(java.io.Writer stream, java.lang.String indent, java.lang.String parent, java.lang.String child) throws java.io.IOException
Returns the DOT description of the object. This is used for visualizing the workflow.- Parameters:
stream
- is a stream opened and ready for writing. This can also be a StringWriter for efficient output.indent
- is aString
of spaces used for pretty printing. The initial amount of spaces should be an empty string. The parameter is used internally for the recursive traversal.parent
- the parentchild
- the child- Throws:
java.io.IOException
- if something fishy happens to the stream.
-
writeDOTHeader
public void writeDOTHeader(java.io.Writer stream, java.lang.String indent) throws java.io.IOException
Writes out the static DOT Header.- Parameters:
stream
- is a stream opened and ready for writing. This can also be a StringWriter for efficient output.indent
- is aString
of spaces used for pretty printing. The initial amount of spaces should be an empty string. The parameter is used internally for the recursive traversal.- Throws:
java.io.IOException
- if something fishy happens to the stream.
-
addNode
public void addNode(GraphNode node)
Adds a node to the Graph. It overwrites an already existing node with the same ID.
-
addEdge
public void addEdge(java.lang.String parent, java.lang.String child)
Adds an edge between two already existing nodes in the graph.
-
addEdge
public void addEdge(GraphNode parent, GraphNode child)
Adds an edge between two already existing nodes in the graph.
-
addEdges
public void addEdges(java.lang.String child, java.util.List<java.lang.String> parents)
A convenience method that allows for bulk addition of edges between already existing nodes in the graph.
-
getNode
public GraphNode getNode(java.lang.String identifier)
Returns the node matching the id passed.
-
addRoot
public void addRoot(GraphNode root)
Adds a single root node to the Graph. All the exisitng roots of the Graph become children of the root.
-
remove
public boolean remove(java.lang.String identifier)
Removes a node from the Graph.
-
resetEdges
public void resetEdges()
Resets all the dependencies in the Graph, while preserving the nodes. The resulting Graph is a graph of independent nodes.- Specified by:
resetEdges
in interfaceGraph
-
nodeIterator
public java.util.Iterator<GraphNode> nodeIterator()
Returns an iterator for the nodes in the Graph. These iterators are fail safe.- Specified by:
nodeIterator
in interfaceGraph
- Returns:
- Iterator
-
iterator
public java.util.Iterator<GraphNode> iterator()
Returns an iterator that traverses through the graph using a graph traversal algorithm.
-
bottomUpIterator
public java.util.Iterator<GraphNode> bottomUpIterator()
Returns an iterator that traverses the graph bottom up from the leaves. At any one time, only one iterator can iterate through the graph.- Specified by:
bottomUpIterator
in interfaceGraph
- Returns:
- Iterator through the nodes of the graph.
-
topologicalSortIterator
public java.util.Iterator<GraphNode> topologicalSortIterator()
Returns an iterator for the graph that traverses in topological sort order.- Specified by:
topologicalSortIterator
in interfaceGraph
- Returns:
- Iterator through the nodes of the graph.
-
size
public int size()
Returns the number of nodes in the graph.
-
getRoots
public java.util.List<GraphNode> getRoots()
Returns the root nodes of the Graph.
-
getLeaves
public java.util.List<GraphNode> getLeaves()
Returns the leaf nodes of the Graph.
-
-