Package edu.isi.pegasus.planner.common
Class PegasusProperties
- java.lang.Object
-
- edu.isi.pegasus.planner.common.PegasusProperties
-
- All Implemented Interfaces:
java.lang.Cloneable
public class PegasusProperties extends java.lang.Object implements java.lang.Cloneable
A Central Properties class that keeps track of all the properties used by Pegasus. All other classes access the methods in this class to get the value of the property. It access the CommonProperties class to read the property file.- Version:
- $Revision$
- Author:
- Karan Vahi, Gaurang Mehta
- See Also:
org.griphyn.common.util.CommonProperties
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
PegasusProperties.CLEANUP_SCOPE
An enum defining The scope for cleanup algorithm
-
Field Summary
Fields Modifier and Type Field Description static java.lang.String
ALL_TRANSFER_PRIORITY_PROPERTY_KEY
Default properties that applies priorities to all kinds of transfer jobs.static java.lang.String
CONDOR_KICKSTART
static java.lang.String
DEFAULT_COMP_ERROR_PERCENTAGE
static java.lang.String
DEFAULT_COMP_MULTIPLICATION_FACTOR
static java.lang.String
DEFAULT_COMP_VARIANCE_PERCENTAGE
static java.lang.String
DEFAULT_CONDOR_BIN_DIR
static java.lang.String
DEFAULT_CONDOR_CONFIG_DIR
static java.lang.String
DEFAULT_DAGMAN_MAX_PRE_VALUE
The default value to be assigned for dagman.maxpre .static java.lang.String
DEFAULT_DATA_MULTIPLICATION_FACTOR
private static java.lang.String
DEFAULT_DAX_CALLBACK
The default DAXCallback that is loaded, if none is specified by the user.static java.lang.String
DEFAULT_EXEC_DIR
static java.lang.String
DEFAULT_GRIDSTART_MODE
static java.lang.String
DEFAULT_INVOKE_LENGTH
static java.lang.String
DEFAULT_JOB_AGGREGATOR
static java.lang.String
DEFAULT_LOGGING_FILE
static java.lang.String
DEFAULT_LRC_IGNORE_URL
static java.lang.String
DEFAULT_POOL_MODE
static java.lang.String
DEFAULT_RC_COLLECTION
static java.lang.String
DEFAULT_RLI_URL
static java.lang.String
DEFAULT_RLS_EXIT_MODE
static java.lang.String
DEFAULT_RLS_QUERY_ATTRIB
static java.lang.String
DEFAULT_RLS_QUERY_MODE
static java.lang.String
DEFAULT_RLS_TIMEOUT
static java.lang.String
DEFAULT_SITE_SELECTOR
static java.lang.String
DEFAULT_SITE_SELECTOR_KEEP
static java.lang.String
DEFAULT_SITE_SELECTOR_TIMEOUT
static java.lang.String
DEFAULT_STAGING_DELIMITER
static java.lang.String
DEFAULT_STORAGE_DIR
static java.lang.String
DEFAULT_TC_MAPPER_MODE
static java.lang.String
DEFAULT_TC_MODE
static java.lang.String
DEFAULT_TRANSFER_PROCESSES
static java.lang.String
DEFAULT_TRANSFER_STREAMS
static java.lang.String
DEFAULT_TX_SELECTOR_MODE
static java.lang.String
DISABLE_INVOKE_PROPERTY
the name of the property to disable invoke functionalityprivate java.lang.String
mDefaultPoolFile
The default path to the pool file.private java.lang.String
mDefaultTC
The default path to the transformation catalog.private java.lang.String
mDefaultTransferPriority
The default transfer priority that needs to be applied to the transfer jobs.private java.util.Set
mDeprecatedProperties
The set containing the deprecated properties specified by the user.private static java.util.Map<Profiles.NAMESPACES,java.lang.String>
mNamepsaceToPropertiesPrefix
private java.lang.String
mPegasusHome
The value of the PEGASUS_HOME environment variable.private Profiles
mProfiles
Profiles that are specified in the propertiesprivate CommonProperties
mProps
The object holding all the properties pertaining to the VDS system.private java.lang.String
mPropsInSubmitDir
The pointer to the properties file that is written out in the submit directory.static java.lang.String
PEGASUS_APP_METRICS_PREFIX
static java.lang.String
PEGASUS_REPLICA_CATALOG_FILE_PROPERTY
static java.lang.String
PEGASUS_REPLICA_CATALOG_PROPERTY
static java.lang.String
PEGASUS_SITE_CATALOG_FILE_PROPERTY
static java.lang.String
PEGASUS_SITE_CATALOG_PROPERTY
static java.lang.String
PEGASUS_TRANSFER_WORKER_PACKAGE_PROPERTY
static java.lang.String
PEGASUS_TRANSFORMATION_CATALOG_FILE_PROPERTY
static java.lang.String
PEGASUS_TRANSFORMATION_CATALOG_PROPERTY
static java.lang.String
PEGASUS_WORKER_NODE_EXECUTION_PROPERTY
static java.lang.String
ROOT_WORKFLOW_UUID_PROPERTY_KEY
The property key designated the root workflow uuid.static java.lang.String
SC_XML_FILE
static java.lang.String
SC_XML3_FILE
static java.lang.String
TC_DATA_FILE
static java.lang.String
TC_TEXT_FILE
-
Constructor Summary
Constructors Modifier Constructor Description private
PegasusProperties(java.lang.String confProperties)
The constructor that constructs the default paths to the various configuration files, and populates the singleton instance as required.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description boolean
abortOnFirstJobFailure()
Returns a boolean indicating whether seqexec trips on the first job failure.boolean
addDataDependencies()
Returns a boolean indicating whether to automatically add edges as a result of underlying data dependecnies between jobs.boolean
assignDefaultJobPriorities()
Returns a boolean indicating whether we assign job priorities or not to the jobs Referred to by the "pegasus.job.priority.assign" property.boolean
bypassFirstLevelStagingForInputs()
Returns a boolean indicating whether to bypass first level staging of inputs.java.lang.Object
clone()
Returns the clone of the object.boolean
createRegistrationJobs()
Returns a boolean indicating whether we create registration jobs or not.boolean
disableInvokeInGridStart()
Returns a boolean indicating whether to disable use of invoke or not.boolean
doStatWithKickstart()
Return a boolean indicating whether to turn the stat option for kickstart on or not.boolean
executeOnWorkerNode()
Returns a boolean indicating whether to have jobs executing on worker node tmp or not.boolean
generateKickstartExtraOptions()
Returns a boolean indicating whehter to pass extra options to kickstart or not.boolean
generateLOFFiles()
Return a boolean indicating whether to generate the LOF files for the jobs or not.java.lang.String
getAllIgnoredSites()
Returns a comma separated list of sites, from which to ignore data transfers for all sites.java.lang.String
getAllPreferredSites()
Returns a comma separated list of sites, from which to prefer data transfers for all sites.java.lang.String
getBaseSourceURLForSetupTransfers()
Returns the base source URL where pointing to the directory where the worker package executables for pegasus releases are kept.java.io.File
getBinDir()
Accessor to the bin directory of the Pegasus installjava.lang.String
getChmodDisabledSites()
Returns the list of sites for which the chmod job creation has to be disabled for executable staging.java.lang.String
getCleanupImplementation()
Returns the name of the class that the user wants, to render the cleanup jobs.PegasusProperties.CLEANUP_SCOPE
getCleanupScope()
Returns the scope for file cleanup.java.lang.String
getCleanupStrategy()
Returns the name of the Strategy class that the user wants, to insert the cleanup jobs in the graph.java.lang.String
getClustererLabelKey()
Returns the key that is to be used as a label key, for labelled clustering.java.lang.String
getClusterSizeCleanupJobsPerLevel()
Returns the fraction of cleanup jobs clustered into a single clustered cleanup job.java.lang.String
getCollapseFactors()
Returns a comma separated list for the node collapsing criteria for the execution pools.java.lang.String
getCreateDirClass()
Returns the name of the class that the user wants, to insert the create directory jobs in the graph in case of creating random directories.java.lang.String
getCreateDirImplementation()
Returns the name of the class that the user wants, to render the directory creation jobs.java.lang.String
getCredName()
Returns the credential name to be used for the stork transfer jobs.java.lang.String
getDAXSchemaLocation()
Returns the location of the schema for the DAX.java.lang.String
getDAXSchemaLocation(java.lang.String defaultLocation)
Returns the location of the schema for the DAX.java.lang.String
getDefaultPathToCondorKickstart()
Returns the default path to the condor kickstart.java.lang.String
getDefaultPathToSC()
Returns the default path to the site catalog file.java.lang.String
getDefaultPathToTC()
Returns the default path to the transformation catalog.private java.lang.String
getDefaultThirdPartySites()
Returns the default list of third party sites.private java.lang.String
getDefaultTransferImplementation()
Returns the default transfer implementation to be picked up for constructing transfer jobs.private java.lang.String
getDefaultTransferPriority()
Returns the default priority for the transfer jobs if specified in the properties file.java.lang.String
getExecDirectory()
Returns the execution directory suffix or absolute specified that is appended/replaced to the exec-mount-point specified in the pool catalog for the various pools.java.lang.String
getGridFTPTimeout()
Returns the timeout value in seconds after which to timeout in case of opening sockets to grid ftp server.java.lang.String
getGridStart()
Returns the GRIDSTART that is to be used to launch the jobs on the grid.long
getGridStartInvokeLength()
Returns the trigger value for invoking an application through kickstart using kickstart.java.lang.String
getHorizontalClusterPreference()
Returns the users horizontal clustering preference.java.lang.String
getHorizontalPartitionerBundleValue(java.lang.String name)
Returns the bundle value for a particular transformation.java.lang.String
getHorizontalPartitionerCollapseValue(java.lang.String name)
Returns the collapse value for a particular transformation.java.lang.String
getHttpLog4jURL()
Returns the http url for log4j properties for windward project.static PegasusProperties
getInstance()
Returns an instance to this properties object.static PegasusProperties
getInstance(java.lang.String confProperties)
Returns an instance to this properties object.java.lang.String
getJobAggregator()
Returns what job aggregator is to be used to aggregate multiple compute jobs into a single condor job.java.lang.String
getLog4JLogFile()
Returns the location of the local log file where you want the messages to be logged.java.lang.String
getLogFormatter()
Returns the log formatter to use.java.lang.String
getLoggingFile()
Returns the file to which all the logging needs to be directed to.java.lang.String
getLogManager()
Returns the log manager to use.java.util.List
getMatchingProperties(java.lang.String prefix, boolean system)
Returns the properties matching a particular prefix as a list of sorted name value pairs, where name is the full name of the matching property (including the prefix) and value is it's value in the properties file.java.lang.String
getMaximumCleanupJobsPerLevel()
Returns the maximum number of clean up jobs created per level of the workflow in case of InPlace cleanup.java.lang.String
getMetricsLogFile()
Returns the path to the file that is used to be logging metrics Referred to by the "pegasus.log.metrics.file" property.java.lang.String
getNumOfTransferProcesses()
It returns the number of processes of g-u-c that the transfer script needs to spawn to do the transfers.java.lang.String
getNumOfTransferStreams()
It returns the number of streams that each transfer process uses to do the ftp transfer.java.lang.String
getPartitionerDAXCallback()
Returns the DAXCallback that is to be used while parsing the DAX.java.lang.String
getPartitionerLabelKey()
Returns the key that is to be used as a label key, for labelled partitioning.java.lang.String
getPartitionParsingMode()
Returns the mode for parsing the dax while writing out the partitioned daxes.java.lang.String
getPDAXSchemaLocation(java.lang.String defaultLocation)
Returns the location of the schema for the PDAX.java.lang.String
getPoolFile()
Returns the path to the pool file.java.lang.String
getPoolMode()
Returns the mode to be used for accessing the pool information.java.lang.String
getPoolSchemaLocation()
Returns the location of the schema for the DAX.java.lang.String
getPoolSchemaLocation(java.lang.String defaultLocation)
Returns the location of the schema for the site catalog file.Namespace
getProfiles(Profiles.NAMESPACES ns)
Returns all the profiles relevant to a particular namespacejava.lang.String
getPropertiesInSubmitDirectory()
Returns the path to the property file that has been writting out in the submit directory.java.lang.String
getProperty(java.lang.String key)
It allows you to get any property from the property file without going through the corresponding accesor function in this class.private java.lang.String
getProperty(java.lang.String newProperty, java.lang.String deprecatedProperty)
This function is used to check whether a deprecated property is used or not.private java.lang.String
getProperty(java.lang.String newProperty, java.lang.String deprecatedProperty, java.lang.String defaultValue)
This function is used to check whether a deprecated property is used or not.java.lang.String
getRefinementProvenanceStore()
Returns the provenance store to use to log the refiner actions.java.lang.String
getReplicaMode()
Returns the replica mode.java.lang.String
getReplicaSelector()
Returns the name of the selector to be used for selection amongst the various replicas of a single lfn.java.lang.String
getRLIURL()
Returns the url to the RLI of the RLS.int
getRLSTimeout()
It returns the timeout value in seconds after which to timeout in case of no activity from the RLS.java.lang.String
getRootWorkflowUUID()
Returns the root workflow UUID if defined in the properties, else null Referred to by the "pegasus.workflow.root.uuid" property.java.io.File
getSchemaDir()
Accessor to the schema directory of the Pegasus installjava.io.File
getSharedDir()
Accessor to the bin directory of the Pegasus installjava.lang.String
getSiteSelectorKeep()
Returns a value designating whether we need to keep the temporary files that are passed to the external site selectors.java.lang.String
getSiteSelectorMode()
Returns the class name of the site selector, that needs to be invoked to do the site selection.java.lang.String
getSiteSelectorPath()
Returns the path to the external site selector that needs to be called out to make the decision of site selection.int
getSiteSelectorTimeout()
It returns the timeout value in seconds after which to timeout in case of no activity from the external site selector.java.lang.String
getSLSTransferArguments()
Returns the extra arguments with which the transfer executable used in PegasusLite needs to be invoked.java.lang.String
getSLSTransferImplementation()
Returns the sls transfer implementation that is to be used for constructing the transfer jobs.java.lang.String
getStagingDelimiter()
Returns the delimiter to be used for constructing the staged executable name, during transfer of executables to remote sites.java.lang.String
getStorageDirectory()
Returns the storage directory suffix or absolute specified that is appended/replaced to the storage-mount-point specified in the pool catalog for the various pools.java.lang.String
getSubmitLogsDirectory()
Returns the the path to the logs directory on the submit host.java.lang.String
getSubmitMode()
Returns which submit mode to be used to submit the jobs on to the grid.java.io.File
getSysConfDir()
Accessor to $PEGASUS_HOME/etc.java.lang.String
getTCMapperMode()
Returns the mode for loading the transformation mapper that sits in front of the transformation catalog.java.lang.String
getTCMode()
Returns the mode to be used for accessing the Transformation Catalog.java.lang.String
getTCPath()
Returns the location of the transformation catalog.java.lang.String
getThirdPartySites(java.lang.String property)
Returns the comma separated list of third party sites, specified in the properties.java.lang.String
getThirdPartySitesRemote(java.lang.String property)
Returns the comma separated list of third party sites for which the third party transfers are executed on the remote sites.java.lang.String
getTransferArguments()
Returns the arguments with which the transfer executable needs to be invoked.java.lang.String
getTransferImplementation()
Returns the transfer implementation that is to be used for constructing the transfer jobs.java.lang.String
getTransferImplementation(java.lang.String property)
Returns the transfer implementation.java.lang.String
getTransferInterPriority()
Returns the priority to be set for the interpool transfer job.private java.lang.String
getTransferPriority(java.lang.String property)
Returns the transfer priority.java.lang.String
getTransferRefiner()
Returns the transfer refiner that is to be used for adding in the transfer jobs in the workflow Referred to by the "pegasus.transfer.refiner" property.java.lang.String
getTransferStageInPriority()
Returns the priority to be set for the stage in transfer job.java.lang.String
getTransferStageOutPriority()
Returns the priority to be set for the stage out transfer job.java.lang.String
getTXSelectorMode()
Returns the mode for loading the transformation selector that selects amongst the various candidate transformation catalog entry objects.boolean
getUseOfSymbolicLinks()
It returns whether the use of symbolic links in case where the source and destination files happen to be on the same file system.CommonProperties
getVDSProperties()
Returns the CommonProperties that this object encapsulates.java.lang.String
getWingsPropertiesFile()
Returns the path to the wings properties file.java.lang.String
getWingsRequestID()
Returns the request id.private void
initializePropertyFile(java.lang.String confProperties)
Gets the handle to the properties file.boolean
labelBasedSubmitDirectoryForSubWorkflows()
Returns a boolean indicating whether the submit directory for the sub workflows should include the label of the sub workflow or not.private void
logDeprecatedWarning(java.lang.String deprecatedProperty, java.lang.String newProperty)
Logs a warning about the deprecated property.boolean
logJobAggregatorProgress()
Returns whether the seqexec job aggregator should log progress to a log or not.boolean
logJobAggregatorProgressToGlobal()
Returns whether the seqexec job aggregator should write to a global log or not.boolean
logMemoryUsage()
Returns a boolean indicating whether to log JVM memory usage or not.java.util.Properties
matchingSubset(java.lang.String prefix, boolean keepPrefix)
Extracts a specific property key subset from the known properties.java.util.Map<Profiles.NAMESPACES,java.lang.String>
namespaceToPropertiesPrefix()
static PegasusProperties
nonSingletonInstance()
To get a reference to the the object.protected static PegasusProperties
nonSingletonInstance(java.lang.String confProperties)
To get a reference to the the object.boolean
preserveParserLineBreaks()
Returns a boolean indicating whether to preserver line breaks.boolean
quoteTransferURL()
Returns whether to introduce quotes around url's before handing to g-u-c and condor.Profiles
retrieveProfilesFromProperties()
Retrieves profiles from the propertiesprotected Profiles
retrieveProfilesFromProperties(CommonProperties properties)
Retrieves profiles from the propertiesprivate void
sanitizePathForProperty(java.util.Properties properties, java.lang.String key)
Santizes the value in the properties .protected static void
sanityCheck(java.io.File dir)
Checks the destination location for existence, if it can be created, if it is writable etc.boolean
setPostSCRIPTDebugON()
Returns a boolean indicating whether to turn debug on or not for exitcode.java.lang.Object
setProperty(java.lang.String key, java.lang.String value)
Accessor: Overwrite any properties from within the program.boolean
setXBitWithKickstart()
Returns a boolean indicating whether kickstart should set x bit on staged executables before launching them.boolean
stageSLSFilesViaFirstLevelStaging()
Returns a boolean indicating whether to stage sls files via Pegasus First Level Staging or let Condor do it.boolean
symlinkCommonLog()
Completely disable placing a symlink for Condor common log (indiscriminately).boolean
transferWorkerPackage()
It specifies if the worker package needs to be staged to the remote site or not.boolean
treatCacheAsRC()
Returns a boolean indicating whether to treat the entries in the cache files as a replica catalog or not.boolean
useCondorQuotingForArguments()
Returns a boolean indicating whether we want to Condor Quote the arguments of the job or not.boolean
useDeepStorageDirectoryStructure()
Returns a boolean indicating whether to have a deep storage directory structure or not while staging out data to the output site.boolean
useExtendedTimeStamp()
It specifies whether to use the extended timestamp format for generation of timestamps that are used to create the random directory name, and for the classads generation.boolean
useForceInTransfer()
It specifies whether the underlying transfer mechanism being used should use the force option if available to transfer the files.boolean
useInvokeInGridStart()
Returns a boolean indicating whether to use invoke in kickstart always or not.boolean
useTimestampForDirectoryStructure()
Returns a boolean indicating whether to use timestamp for directory name creation or not.boolean
writeOutMetrics()
Returns a boolean indicating whether to write out the planner metrics or not.java.lang.String
writeOutProperties(java.lang.String directory)
Writes out the properties to a temporary file in the directory passed.java.lang.String
writeOutProperties(java.lang.String directory, boolean sanitizePath)
Writes out the properties to a temporary file in the directory passed.java.lang.String
writeOutProperties(java.lang.String directory, boolean sanitizePath, boolean setInternalVariable)
Writes out the properties to a temporary file in the directory passed.
-
-
-
Field Detail
-
DISABLE_INVOKE_PROPERTY
public static final java.lang.String DISABLE_INVOKE_PROPERTY
the name of the property to disable invoke functionality- See Also:
- Constant Field Values
-
PEGASUS_WORKER_NODE_EXECUTION_PROPERTY
public static final java.lang.String PEGASUS_WORKER_NODE_EXECUTION_PROPERTY
- See Also:
- Constant Field Values
-
PEGASUS_TRANSFER_WORKER_PACKAGE_PROPERTY
public static final java.lang.String PEGASUS_TRANSFER_WORKER_PACKAGE_PROPERTY
- See Also:
- Constant Field Values
-
PEGASUS_TRANSFORMATION_CATALOG_PROPERTY
public static final java.lang.String PEGASUS_TRANSFORMATION_CATALOG_PROPERTY
- See Also:
- Constant Field Values
-
PEGASUS_TRANSFORMATION_CATALOG_FILE_PROPERTY
public static final java.lang.String PEGASUS_TRANSFORMATION_CATALOG_FILE_PROPERTY
- See Also:
- Constant Field Values
-
PEGASUS_REPLICA_CATALOG_PROPERTY
public static final java.lang.String PEGASUS_REPLICA_CATALOG_PROPERTY
- See Also:
- Constant Field Values
-
PEGASUS_REPLICA_CATALOG_FILE_PROPERTY
public static final java.lang.String PEGASUS_REPLICA_CATALOG_FILE_PROPERTY
- See Also:
- Constant Field Values
-
PEGASUS_SITE_CATALOG_PROPERTY
public static final java.lang.String PEGASUS_SITE_CATALOG_PROPERTY
- See Also:
- Constant Field Values
-
PEGASUS_SITE_CATALOG_FILE_PROPERTY
public static final java.lang.String PEGASUS_SITE_CATALOG_FILE_PROPERTY
- See Also:
- Constant Field Values
-
PEGASUS_APP_METRICS_PREFIX
public static final java.lang.String PEGASUS_APP_METRICS_PREFIX
- See Also:
- Constant Field Values
-
DEFAULT_RC_COLLECTION
public static final java.lang.String DEFAULT_RC_COLLECTION
- See Also:
- Constant Field Values
-
DEFAULT_RLI_URL
public static final java.lang.String DEFAULT_RLI_URL
-
DEFAULT_RLS_QUERY_MODE
public static final java.lang.String DEFAULT_RLS_QUERY_MODE
- See Also:
- Constant Field Values
-
DEFAULT_RLS_EXIT_MODE
public static final java.lang.String DEFAULT_RLS_EXIT_MODE
- See Also:
- Constant Field Values
-
DEFAULT_RLS_QUERY_ATTRIB
public static final java.lang.String DEFAULT_RLS_QUERY_ATTRIB
- See Also:
- Constant Field Values
-
DEFAULT_LRC_IGNORE_URL
public static final java.lang.String DEFAULT_LRC_IGNORE_URL
-
DEFAULT_RLS_TIMEOUT
public static final java.lang.String DEFAULT_RLS_TIMEOUT
- See Also:
- Constant Field Values
-
DEFAULT_EXEC_DIR
public static final java.lang.String DEFAULT_EXEC_DIR
- See Also:
- Constant Field Values
-
DEFAULT_STORAGE_DIR
public static final java.lang.String DEFAULT_STORAGE_DIR
- See Also:
- Constant Field Values
-
DEFAULT_TC_MODE
public static final java.lang.String DEFAULT_TC_MODE
- See Also:
- Constant Field Values
-
TC_TEXT_FILE
public static final java.lang.String TC_TEXT_FILE
- See Also:
- Constant Field Values
-
TC_DATA_FILE
public static final java.lang.String TC_DATA_FILE
- See Also:
- Constant Field Values
-
DEFAULT_POOL_MODE
public static final java.lang.String DEFAULT_POOL_MODE
- See Also:
- Constant Field Values
-
DEFAULT_CONDOR_BIN_DIR
public static final java.lang.String DEFAULT_CONDOR_BIN_DIR
- See Also:
- Constant Field Values
-
DEFAULT_CONDOR_CONFIG_DIR
public static final java.lang.String DEFAULT_CONDOR_CONFIG_DIR
- See Also:
- Constant Field Values
-
SC_XML_FILE
public static final java.lang.String SC_XML_FILE
- See Also:
- Constant Field Values
-
SC_XML3_FILE
public static final java.lang.String SC_XML3_FILE
- See Also:
- Constant Field Values
-
CONDOR_KICKSTART
public static final java.lang.String CONDOR_KICKSTART
- See Also:
- Constant Field Values
-
DEFAULT_STAGING_DELIMITER
public static final java.lang.String DEFAULT_STAGING_DELIMITER
- See Also:
- Constant Field Values
-
DEFAULT_TRANSFER_PROCESSES
public static final java.lang.String DEFAULT_TRANSFER_PROCESSES
- See Also:
- Constant Field Values
-
DEFAULT_TRANSFER_STREAMS
public static final java.lang.String DEFAULT_TRANSFER_STREAMS
- See Also:
- Constant Field Values
-
DEFAULT_GRIDSTART_MODE
public static final java.lang.String DEFAULT_GRIDSTART_MODE
- See Also:
- Constant Field Values
-
DEFAULT_INVOKE_LENGTH
public static final java.lang.String DEFAULT_INVOKE_LENGTH
- See Also:
- Constant Field Values
-
DEFAULT_SITE_SELECTOR
public static final java.lang.String DEFAULT_SITE_SELECTOR
- See Also:
- Constant Field Values
-
DEFAULT_SITE_SELECTOR_TIMEOUT
public static final java.lang.String DEFAULT_SITE_SELECTOR_TIMEOUT
- See Also:
- Constant Field Values
-
DEFAULT_SITE_SELECTOR_KEEP
public static final java.lang.String DEFAULT_SITE_SELECTOR_KEEP
- See Also:
- Constant Field Values
-
DEFAULT_DATA_MULTIPLICATION_FACTOR
public static final java.lang.String DEFAULT_DATA_MULTIPLICATION_FACTOR
- See Also:
- Constant Field Values
-
DEFAULT_COMP_MULTIPLICATION_FACTOR
public static final java.lang.String DEFAULT_COMP_MULTIPLICATION_FACTOR
- See Also:
- Constant Field Values
-
DEFAULT_COMP_ERROR_PERCENTAGE
public static final java.lang.String DEFAULT_COMP_ERROR_PERCENTAGE
- See Also:
- Constant Field Values
-
DEFAULT_COMP_VARIANCE_PERCENTAGE
public static final java.lang.String DEFAULT_COMP_VARIANCE_PERCENTAGE
- See Also:
- Constant Field Values
-
DEFAULT_JOB_AGGREGATOR
public static final java.lang.String DEFAULT_JOB_AGGREGATOR
- See Also:
- Constant Field Values
-
DEFAULT_TC_MAPPER_MODE
public static final java.lang.String DEFAULT_TC_MAPPER_MODE
- See Also:
- Constant Field Values
-
DEFAULT_TX_SELECTOR_MODE
public static final java.lang.String DEFAULT_TX_SELECTOR_MODE
- See Also:
- Constant Field Values
-
DEFAULT_LOGGING_FILE
public static final java.lang.String DEFAULT_LOGGING_FILE
- See Also:
- Constant Field Values
-
ALL_TRANSFER_PRIORITY_PROPERTY_KEY
public static final java.lang.String ALL_TRANSFER_PRIORITY_PROPERTY_KEY
Default properties that applies priorities to all kinds of transfer jobs.- See Also:
- Constant Field Values
-
ROOT_WORKFLOW_UUID_PROPERTY_KEY
public static final java.lang.String ROOT_WORKFLOW_UUID_PROPERTY_KEY
The property key designated the root workflow uuid.- See Also:
- Constant Field Values
-
DEFAULT_DAGMAN_MAX_PRE_VALUE
public static final java.lang.String DEFAULT_DAGMAN_MAX_PRE_VALUE
The default value to be assigned for dagman.maxpre .- See Also:
- Constant Field Values
-
DEFAULT_DAX_CALLBACK
private static final java.lang.String DEFAULT_DAX_CALLBACK
The default DAXCallback that is loaded, if none is specified by the user.- See Also:
- Constant Field Values
-
mPegasusHome
private java.lang.String mPegasusHome
The value of the PEGASUS_HOME environment variable.
-
mProps
private CommonProperties mProps
The object holding all the properties pertaining to the VDS system.
-
mDefaultTC
private java.lang.String mDefaultTC
The default path to the transformation catalog.
-
mDefaultPoolFile
private java.lang.String mDefaultPoolFile
The default path to the pool file.
-
mDefaultTransferPriority
private java.lang.String mDefaultTransferPriority
The default transfer priority that needs to be applied to the transfer jobs.
-
mDeprecatedProperties
private java.util.Set mDeprecatedProperties
The set containing the deprecated properties specified by the user.
-
mPropsInSubmitDir
private java.lang.String mPropsInSubmitDir
The pointer to the properties file that is written out in the submit directory.
-
mProfiles
private Profiles mProfiles
Profiles that are specified in the properties
-
mNamepsaceToPropertiesPrefix
private static java.util.Map<Profiles.NAMESPACES,java.lang.String> mNamepsaceToPropertiesPrefix
-
-
Constructor Detail
-
PegasusProperties
private PegasusProperties(java.lang.String confProperties)
The constructor that constructs the default paths to the various configuration files, and populates the singleton instance as required. If the properties file passed is null, then the singleton instance is invoked, else the non singleton instance is invoked.- Parameters:
confProperties
- the path to conf properties, that supersede the loading of properties from $PEGASUS_HOME/.pegasusrc
-
-
Method Detail
-
namespaceToPropertiesPrefix
public java.util.Map<Profiles.NAMESPACES,java.lang.String> namespaceToPropertiesPrefix()
-
getInstance
public static PegasusProperties getInstance()
Returns an instance to this properties object.- Returns:
- a handle to the Properties class.
-
getInstance
public static PegasusProperties getInstance(java.lang.String confProperties)
Returns an instance to this properties object.- Parameters:
confProperties
- the path to conf properties, that supersede the loading of properties from $PEGASUS_HOME/.pegasusrc- Returns:
- a handle to the Properties class.
-
nonSingletonInstance
protected static PegasusProperties nonSingletonInstance(java.lang.String confProperties)
To get a reference to the the object. The properties file that is loaded is from the path specified in the argument. This is *not implemented* as singleton. However the invocation of this does modify the internally held singleton object.- Parameters:
confProperties
- the path to conf properties, that supersede the loading of properties from $PEGASUS_HOME/.pegasusrc- Returns:
- a handle to the Properties class.
-
nonSingletonInstance
public static PegasusProperties nonSingletonInstance()
To get a reference to the the object. The properties file that is loaded is from the path specified in the argument. This is *not implemented* as singleton. However the invocation of this does modify the internally held singleton object.- Returns:
- a handle to the Properties class.
-
retrieveProfilesFromProperties
public Profiles retrieveProfilesFromProperties()
Retrieves profiles from the properties- Parameters:
properties
- the common properties so far- Returns:
- profiles object.
-
retrieveProfilesFromProperties
protected Profiles retrieveProfilesFromProperties(CommonProperties properties)
Retrieves profiles from the properties- Parameters:
properties
- the common properties so far- Returns:
- profiles object.
-
clone
public java.lang.Object clone()
Returns the clone of the object.- Overrides:
clone
in classjava.lang.Object
- Returns:
- the clone
-
getBinDir
public java.io.File getBinDir()
Accessor to the bin directory of the Pegasus install- Returns:
- the "etc" directory of the VDS runtime system.
-
getSchemaDir
public java.io.File getSchemaDir()
Accessor to the schema directory of the Pegasus install- Returns:
- the "etc" directory of the VDS runtime system.
-
getSharedDir
public java.io.File getSharedDir()
Accessor to the bin directory of the Pegasus install- Returns:
- the "etc" directory of the VDS runtime system.
-
getProfiles
public Namespace getProfiles(Profiles.NAMESPACES ns)
Returns all the profiles relevant to a particular namespace- Parameters:
ns
- the namespace corresponding to which you need the profiles
-
getDefaultPathToTC
public java.lang.String getDefaultPathToTC()
Returns the default path to the transformation catalog. Currently the default path defaults to $PEGASUS_HOME/etc/tc.text if transformation type is Text else $PEGASUS_HOME/etc/tc.data- Returns:
- the default path to transformation catalog file
-
getDefaultPathToSC
public java.lang.String getDefaultPathToSC()
Returns the default path to the site catalog file. The default path is constructed on the basis of the mode set by the user.- Returns:
- $PEGASUS_HOME/etc/sites.xml3 if the pool mode is XML3, else $PEGASUS_HOME/etc/sites.xml
- See Also:
getPoolMode()
-
getDefaultPathToCondorKickstart
public java.lang.String getDefaultPathToCondorKickstart()
Returns the default path to the condor kickstart. Currently the path defaults to $PEGASUS_HOME/bin/kickstart-condor.- Returns:
- default path to kickstart condor.
-
initializePropertyFile
private void initializePropertyFile(java.lang.String confProperties)
Gets the handle to the properties file. The singleton instance is invoked if the properties file is null (partly due to the way CommonProperties is implemented ), else the non singleton is invoked.- Parameters:
confProperties
- the path to conf properties, that supersede the loading of properties from $PEGASUS_HOME/.pegasusrc
-
getProperty
public java.lang.String getProperty(java.lang.String key)
It allows you to get any property from the property file without going through the corresponding accesor function in this class. For coding and clarity purposes, the function should be used judiciously, and the accessor function should be used as far as possible.- Parameters:
key
- the property whose value is desired.- Returns:
- String
-
getVDSProperties
public CommonProperties getVDSProperties()
Returns the CommonProperties that this object encapsulates. Use only when absolutely necessary. Use accessor methods whereever possible.- Returns:
- CommonProperties
-
setProperty
public java.lang.Object setProperty(java.lang.String key, java.lang.String value)
Accessor: Overwrite any properties from within the program.- Parameters:
key
- is the key to look upvalue
- is the new property value to place in the system.- Returns:
- the old value, or null if it didn't exist before.
-
matchingSubset
public java.util.Properties matchingSubset(java.lang.String prefix, boolean keepPrefix)
Extracts a specific property key subset from the known properties. The prefix may be removed from the keys in the resulting dictionary, or it may be kept. In the latter case, exact matches on the prefix will also be copied into the resulting dictionary.- Parameters:
prefix
- is the key prefix to filter the properties by.keepPrefix
- if true, the key prefix is kept in the resulting dictionary. As side-effect, a key that matches the prefix exactly will also be copied. If false, the resulting dictionary's keys are shortened by the prefix. An exact prefix match will not be copied, as it would result in an empty string key.- Returns:
- a property dictionary matching the filter key. May be an empty dictionary, if no prefix matches were found.
- See Also:
is used to assemble matches
-
getMatchingProperties
public java.util.List getMatchingProperties(java.lang.String prefix, boolean system)
Returns the properties matching a particular prefix as a list of sorted name value pairs, where name is the full name of the matching property (including the prefix) and value is it's value in the properties file.- Parameters:
prefix
- the prefix for the property names.system
- boolean indicating whether to match only System properties or all including the ones in the property file.- Returns:
- list of
NameValue
objects corresponding to the matched properties sorted by keys. null if no matching property is found.
-
getSysConfDir
public java.io.File getSysConfDir()
Accessor to $PEGASUS_HOME/etc. The files in this directory have a low change frequency, are effectively read-only, they reside on a per-machine basis, and they are valid usually for a single user.- Returns:
- the "etc" directory of the VDS runtime system.
-
getDAXSchemaLocation
public java.lang.String getDAXSchemaLocation()
Returns the location of the schema for the DAX. Referred to by the "pegasus.schema.dax" property.- Returns:
- location to the DAX schema.
-
getDAXSchemaLocation
public java.lang.String getDAXSchemaLocation(java.lang.String defaultLocation)
Returns the location of the schema for the DAX. Referred to by the "pegasus.schema.dax" property.- Parameters:
defaultLocation
- the default location to the schema.- Returns:
- location to the DAX schema specified in the properties file, else the default location if no value specified.
-
getPDAXSchemaLocation
public java.lang.String getPDAXSchemaLocation(java.lang.String defaultLocation)
Returns the location of the schema for the PDAX. Referred to by the "pegasus.schema.pdax" property- Parameters:
defaultLocation
- the default location to the schema.- Returns:
- location to the PDAX schema specified in the properties file, else the default location if no value specified.
-
getCreateDirClass
public java.lang.String getCreateDirClass()
Returns the name of the class that the user wants, to insert the create directory jobs in the graph in case of creating random directories. Referred to by the "pegasus.dir.create.strategy" property.- Returns:
- the create dir classname if specified in the properties file, else Minimal.
-
getCreateDirImplementation
public java.lang.String getCreateDirImplementation()
Returns the name of the class that the user wants, to render the directory creation jobs. It dictates what mechanism is used to create the directory for a workflow. Referred to by the "pegasus.dir.create.impl" property.- Returns:
- the create dir classname if specified in the properties file, else DefaultImplementation.
-
useExtendedTimeStamp
public boolean useExtendedTimeStamp()
It specifies whether to use the extended timestamp format for generation of timestamps that are used to create the random directory name, and for the classads generation. Referred to by the "pegasus.dir.timestamp.extended" property.- Returns:
- the value specified in the properties file if valid boolean, else false.
-
useTimestampForDirectoryStructure
public boolean useTimestampForDirectoryStructure()
Returns a boolean indicating whether to use timestamp for directory name creation or not. Referred to by "pegasus.dir.useTimestamp" property.- Returns:
- the boolean value specified in the properties files, else false.
-
getExecDirectory
public java.lang.String getExecDirectory()
Returns the execution directory suffix or absolute specified that is appended/replaced to the exec-mount-point specified in the pool catalog for the various pools. Referred to by the "pegasus.dir.exec" property- Returns:
- the value specified in the properties file, else the default suffix.
- See Also:
DEFAULT_EXEC_DIR
-
getSubmitLogsDirectory
public java.lang.String getSubmitLogsDirectory()
Returns the the path to the logs directory on the submit host. This is the directory where the condor logs for the workflows are created. The logs directory should be on the local filesystem else condor may complain Referred to by the "pegasus.dir.submit.logs" property- Returns:
- the value in the properties file, else null
-
labelBasedSubmitDirectoryForSubWorkflows
public boolean labelBasedSubmitDirectoryForSubWorkflows()
Returns a boolean indicating whether the submit directory for the sub workflows should include the label of the sub workflow or not. Referred to by the "pegasus.dir.submit.subwf.labelbased" property- Returns:
- the value in the properties file, else false
-
getStorageDirectory
public java.lang.String getStorageDirectory()
Returns the storage directory suffix or absolute specified that is appended/replaced to the storage-mount-point specified in the pool catalog for the various pools. Referred to by the "pegasus.dir.storage" property.- Returns:
- the value specified in the properties file, else the default suffix.
- See Also:
DEFAULT_STORAGE_DIR
-
useDeepStorageDirectoryStructure
public boolean useDeepStorageDirectoryStructure()
Returns a boolean indicating whether to have a deep storage directory structure or not while staging out data to the output site. Referred to by the "pegasus.dir.storage.deep" property.- Returns:
- the boolean value specified in the properties files, else false.
-
getCleanupStrategy
public java.lang.String getCleanupStrategy()
Returns the name of the Strategy class that the user wants, to insert the cleanup jobs in the graph. Referred to by the "pegasus.file.cleanup.strategy" property.- Returns:
- the create dir classname if specified in the properties file, else InPlace.
-
getCleanupImplementation
public java.lang.String getCleanupImplementation()
Returns the name of the class that the user wants, to render the cleanup jobs. It dictates what mechanism is used to remove the files on a remote system. Referred to by the "pegasus.file.cleanup.impl" property.- Returns:
- the cleanup implementation classname if specified in the properties file, else Cleanup.
-
getMaximumCleanupJobsPerLevel
public java.lang.String getMaximumCleanupJobsPerLevel()
Returns the maximum number of clean up jobs created per level of the workflow in case of InPlace cleanup. Referred to by the "pegasus.file.cleanup.clusters.num" property- Returns:
- the value in the property file , else null
-
getClusterSizeCleanupJobsPerLevel
public java.lang.String getClusterSizeCleanupJobsPerLevel()
Returns the fraction of cleanup jobs clustered into a single clustered cleanup job. Referred to by the "pegasus.file.cleanup.clusters.size" property- Returns:
- the value in the property file , else null
-
getCleanupScope
public PegasusProperties.CLEANUP_SCOPE getCleanupScope()
Returns the scope for file cleanup. It is used to trigger cleanup in case of deferred planning. The vaild property values accepted are - fullahead - deferred Referred to by the property "pegasus.file.cleanup.scope"- Returns:
- the value in property file if specified, else fullahead
-
getTCMode
public java.lang.String getTCMode()
Returns the mode to be used for accessing the Transformation Catalog. Referred to by the "pegasus.catalog.transformation" property.- Returns:
- the value specified in properties file, else DEFAULT_TC_MODE.
- See Also:
DEFAULT_TC_MODE
-
getTCPath
public java.lang.String getTCPath()
Returns the location of the transformation catalog. Referred to by "pegasus.catalog.transformation.file" property.- Returns:
- the value specified in the properties file, else default path specified by mDefaultTC.
- See Also:
mDefaultTC
-
getTCMapperMode
public java.lang.String getTCMapperMode()
Returns the mode for loading the transformation mapper that sits in front of the transformation catalog. Referred to by the "pegasus.catalog.transformation.mapper" property.- Returns:
- the value specified in the properties file, else default tc mapper mode.
- See Also:
DEFAULT_TC_MAPPER_MODE
-
getReplicaMode
public java.lang.String getReplicaMode()
Returns the replica mode. It identifies the ReplicaMechanism being used by Pegasus to determine logical file locations. Referred to by the "pegasus.catalog.replica" property.- Returns:
- the replica mode, that is used to load the appropriate implementing class if property is specified, else null
-
getRLIURL
public java.lang.String getRLIURL()
Returns the url to the RLI of the RLS. Referred to by the "pegasus.rls.url" property.- Returns:
- the value specified in properties file, else DEFAULT_RLI_URL.
- See Also:
DEFAULT_RLI_URL
-
getRLSTimeout
public int getRLSTimeout()
It returns the timeout value in seconds after which to timeout in case of no activity from the RLS. Referred to by the "pegasus.rc.rls.timeout" property.- Returns:
- the timeout value if specified else, DEFAULT_RLS_TIMEOUT.
- See Also:
DEFAULT_RLS_TIMEOUT
-
getPoolMode
public java.lang.String getPoolMode()
Returns the mode to be used for accessing the pool information. Referred to by the "pegasus.catalog.site" property.- Returns:
- the pool mode, that is used to load the appropriate implementing class if the property is specified, else default pool mode specified by DEFAULT_POOL_MODE
- See Also:
DEFAULT_POOL_MODE
-
getPoolFile
public java.lang.String getPoolFile()
Returns the path to the pool file. Referred to by the "pegasus.catalog.site.file" property.- Returns:
- the path to the pool file specified in the properties file, else the default path specified by mDefaultPoolFile.
- See Also:
mDefaultPoolFile
-
getPoolSchemaLocation
public java.lang.String getPoolSchemaLocation()
Returns the location of the schema for the DAX. Referred to by the "pegasus.schema.sc" property.- Returns:
- the location of pool schema if specified in properties file, else null.
-
getPoolSchemaLocation
public java.lang.String getPoolSchemaLocation(java.lang.String defaultLocation)
Returns the location of the schema for the site catalog file. Referred to by the "pegasus.schema.sc" property- Parameters:
defaultLocation
- the default location where the schema should be if no other location is specified.- Returns:
- the location specified by the property, else defaultLocation.
-
getRefinementProvenanceStore
public java.lang.String getRefinementProvenanceStore()
Returns the provenance store to use to log the refiner actions. Referred to by the "pegasus.catalog.provenance.refinement" property.- Returns:
- the value set in the properties, else null if not set.
-
getTransferImplementation
public java.lang.String getTransferImplementation()
Returns the transfer implementation that is to be used for constructing the transfer jobs. Referred to by the "pegasus.transfer.*.impl" property.- Returns:
- the transfer implementation
-
getSLSTransferImplementation
public java.lang.String getSLSTransferImplementation()
Returns the sls transfer implementation that is to be used for constructing the transfer jobs. Referred to by the "pegasus.transfer.lite.*.impl" property.- Returns:
- the transfer implementation
-
getTransferImplementation
public java.lang.String getTransferImplementation(java.lang.String property)
Returns the transfer implementation.- Parameters:
property
- property name.- Returns:
- the transfer implementation, else the one specified by "pegasus.transfer.*.impl",
-
stageSLSFilesViaFirstLevelStaging
public boolean stageSLSFilesViaFirstLevelStaging()
Returns a boolean indicating whether to stage sls files via Pegasus First Level Staging or let Condor do it. Referred to by the property "pegasus.transfer.stage.lite.file"- Returns:
- boolean value mentioned in the properties or else the default value which is true.
-
getDefaultThirdPartySites
private java.lang.String getDefaultThirdPartySites()
Returns the default list of third party sites. Referred to by the "pegasus.transfer.*.thirdparty.sites" property.- Returns:
- the value specified in the properties file, else null.
-
getDefaultTransferImplementation
private java.lang.String getDefaultTransferImplementation()
Returns the default transfer implementation to be picked up for constructing transfer jobs. Referred to by the "pegasus.transfer.*.impl" property.- Returns:
- the value specified in the properties file, else null.
-
bypassFirstLevelStagingForInputs
public boolean bypassFirstLevelStagingForInputs()
Returns a boolean indicating whether to bypass first level staging of inputs. Useful in case of PegasusLite setup Referred to by the "pegasus.transfer.bypass.input.staging" property.- Returns:
- boolean value specified , else false
-
getDefaultTransferPriority
private java.lang.String getDefaultTransferPriority()
Returns the default priority for the transfer jobs if specified in the properties file.- Returns:
- the value specified in the properties file, else null if non integer value or no value specified.
-
getBaseSourceURLForSetupTransfers
public java.lang.String getBaseSourceURLForSetupTransfers()
Returns the base source URL where pointing to the directory where the worker package executables for pegasus releases are kept. Referred to by the "pegasus.transfer.setup.source.base.url- Returns:
- the value in the property file, else null
-
getTransferRefiner
public java.lang.String getTransferRefiner()
Returns the transfer refiner that is to be used for adding in the transfer jobs in the workflow Referred to by the "pegasus.transfer.refiner" property.- Returns:
- the transfer refiner, else null
-
quoteTransferURL
public boolean quoteTransferURL()
Returns whether to introduce quotes around url's before handing to g-u-c and condor. Referred to by "pegasus.transfer.single.quote" property.- Returns:
- boolean value specified in the properties file, else true in case of non boolean value being specified or property not being set.
-
getNumOfTransferProcesses
public java.lang.String getNumOfTransferProcesses()
It returns the number of processes of g-u-c that the transfer script needs to spawn to do the transfers. This is applicable only in the case where the transfer executable has the capability of spawning processes. It should not be confused with the number of streams that each process opens. By default it is set to 4. In case a non integer value is specified in the properties file it returns the default value. Referred to by "pegasus.transfer.throttle.processes" property.- Returns:
- the number of processes specified in properties file, else DEFAULT_TRANSFER_PROCESSES
- See Also:
DEFAULT_TRANSFER_PROCESSES
-
getNumOfTransferStreams
public java.lang.String getNumOfTransferStreams()
It returns the number of streams that each transfer process uses to do the ftp transfer. By default it is set to 1.In case a non integer value is specified in the properties file it returns the default value. Referred to by "pegasus.transfer.throttle.streams" property.- Returns:
- the number of streams specified in the properties file, else DEFAULT_TRANSFER_STREAMS.
- See Also:
DEFAULT_TRANSFER_STREAMS
-
useForceInTransfer
public boolean useForceInTransfer()
It specifies whether the underlying transfer mechanism being used should use the force option if available to transfer the files. Referred to by "pegasus.transfer.force" property.- Returns:
- boolean value specified in the properties file,else false in case of non boolean value being specified or property not being set.
-
getUseOfSymbolicLinks
public boolean getUseOfSymbolicLinks()
It returns whether the use of symbolic links in case where the source and destination files happen to be on the same file system. Referred to by "pegasus.transfer.links" property.- Returns:
- boolean value specified in the properties file, else false in case of non boolean value being specified or property not being set.
-
getThirdPartySites
public java.lang.String getThirdPartySites(java.lang.String property)
Returns the comma separated list of third party sites, specified in the properties.- Parameters:
property
- property name.- Returns:
- the comma separated list of sites.
-
getThirdPartySitesRemote
public java.lang.String getThirdPartySitesRemote(java.lang.String property)
Returns the comma separated list of third party sites for which the third party transfers are executed on the remote sites.- Parameters:
property
- property name.- Returns:
- the comma separated list of sites.
-
getStagingDelimiter
public java.lang.String getStagingDelimiter()
Returns the delimiter to be used for constructing the staged executable name, during transfer of executables to remote sites. Referred to by the "pegasus.transfer.staging.delimiter" property.- Returns:
- the value specified in the properties file, else DEFAULT_STAGING_DELIMITER
- See Also:
DEFAULT_STAGING_DELIMITER
-
getChmodDisabledSites
public java.lang.String getChmodDisabledSites()
Returns the list of sites for which the chmod job creation has to be disabled for executable staging. Referred to by the "pegasus.transfer.disable.chmod" property.- Returns:
- a comma separated list of site names.
-
transferWorkerPackage
public boolean transferWorkerPackage()
It specifies if the worker package needs to be staged to the remote site or not. Referred to by "pegasus.transfer.worker.package" property.- Returns:
- boolean value specified in the properties file,else false in case of non boolean value being specified or property not being set.
-
getTransferArguments
public java.lang.String getTransferArguments()
Returns the arguments with which the transfer executable needs to be invoked. Referred to by "pegasus.transfer.arguments" property.- Returns:
- the arguments specified in the properties file, else null if property is not specified.
-
getSLSTransferArguments
public java.lang.String getSLSTransferArguments()
Returns the extra arguments with which the transfer executable used in PegasusLite needs to be invoked. Referred to by "pegasus.transfer.lite.arguments" property.- Returns:
- the arguments specified in the properties file, else null if property is not specified.
-
getTransferStageInPriority
public java.lang.String getTransferStageInPriority()
Returns the priority to be set for the stage in transfer job. Referred to by "pegasus.transfer.stagein.priority" property if set, else by "pegasus.transfer.*.priority" property.- Returns:
- the priority as String if a valid integer specified in the properties, else null.
-
getTransferStageOutPriority
public java.lang.String getTransferStageOutPriority()
Returns the priority to be set for the stage out transfer job. Referred to by "pegasus.transfer.stageout.priority" property if set, else by "pegasus.transfer.*.priority" property.- Returns:
- the priority as String if a valid integer specified in the properties, else null.
-
getTransferInterPriority
public java.lang.String getTransferInterPriority()
Returns the priority to be set for the interpool transfer job. Referred to by "pegasus.transfer.inter.priority" property if set, else by "pegasus.transfer.*.priority" property.- Returns:
- the priority as String if a valid integer specified in the properties, else null.
-
getTransferPriority
private java.lang.String getTransferPriority(java.lang.String property)
Returns the transfer priority.- Parameters:
property
- property name.- Returns:
- the priority as String if a valid integer specified in the properties as value to property, else null.
-
getTXSelectorMode
public java.lang.String getTXSelectorMode()
Returns the mode for loading the transformation selector that selects amongst the various candidate transformation catalog entry objects. Referred to by the "pegasus.selector.transformation" property.- Returns:
- the value specified in the properties file, else default transformation selector.
- See Also:
DEFAULT_TC_MAPPER_MODE
-
getReplicaSelector
public java.lang.String getReplicaSelector()
Returns the name of the selector to be used for selection amongst the various replicas of a single lfn. Referred to by the "pegasus.selector.replica" property.- Returns:
- the name of the selector if the property is specified, else null
-
getAllPreferredSites
public java.lang.String getAllPreferredSites()
Returns a comma separated list of sites, from which to prefer data transfers for all sites. Referred to by the "pegasus.selector.replica.*.prefer.stagein.sites" property.- Returns:
- comma separated list of sites.
-
getAllIgnoredSites
public java.lang.String getAllIgnoredSites()
Returns a comma separated list of sites, from which to ignore data transfers for all sites. Replaces the old pegasus.rc.restricted.sites property. Referred to by the "pegasus.selector.ignore.*.prefer.stagein.sites" property.- Returns:
- comma separated list of sites.
-
getSiteSelectorMode
public java.lang.String getSiteSelectorMode()
Returns the class name of the site selector, that needs to be invoked to do the site selection. Referred to by the "pegasus.selector.site" property.- Returns:
- the classname corresponding to the site selector that needs to be invoked if specified in the properties file, else the default selector specified by DEFAULT_SITE_SELECTOR.
- See Also:
DEFAULT_SITE_SELECTOR
-
getSiteSelectorPath
public java.lang.String getSiteSelectorPath()
Returns the path to the external site selector that needs to be called out to make the decision of site selection. Referred to by the "pegasus.selector.site.path" property.- Returns:
- the path to the external site selector if specified in the properties file, else null.
-
getSiteSelectorTimeout
public int getSiteSelectorTimeout()
It returns the timeout value in seconds after which to timeout in case of no activity from the external site selector. Referred to by the "pegasus.selector.site.timeout" property.- Returns:
- the timeout value if specified else, DEFAULT_SITE_SELECTOR_TIMEOUT.
- See Also:
DEFAULT_SITE_SELECTOR_TIMEOUT
-
getSiteSelectorKeep
public java.lang.String getSiteSelectorKeep()
Returns a value designating whether we need to keep the temporary files that are passed to the external site selectors. The check for the valid tristate value should be done at the calling function end. This just passes on the value user specified in the properties file. Referred to by the "pegasus.selector.site.keep.tmp" property.- Returns:
- the value of the property is specified, else DEFAULT_SITE_SELECTOR_KEEP
- See Also:
DEFAULT_SITE_SELECTOR_KEEP
-
getGridStart
public java.lang.String getGridStart()
Returns the GRIDSTART that is to be used to launch the jobs on the grid. Referred to by the "pegasus.gridstart" property.- Returns:
- the value specified in the property file, else DEFAULT_GRIDSTART_MODE
- See Also:
DEFAULT_GRIDSTART_MODE
-
setXBitWithKickstart
public boolean setXBitWithKickstart()
Returns a boolean indicating whether kickstart should set x bit on staged executables before launching them. Referred to by the "pegasus.gridstart.kickstart.set.xbit" property.- Returns:
- the value specified in the property file, else false
-
doStatWithKickstart
public boolean doStatWithKickstart()
Return a boolean indicating whether to turn the stat option for kickstart on or not. By default it is turned on. Referred to by the "pegasus.gridstart.kickstart.stat" property.- Returns:
- the boolean value specified in the property file, else false if not specified or non boolean specified.
-
generateLOFFiles
public boolean generateLOFFiles()
Return a boolean indicating whether to generate the LOF files for the jobs or not. This is used to generate LOF files, but not trigger the stat option Referred to by the "pegasus.gridstart.kickstart.generate.loft" property.- Returns:
- the boolean value specified in the property file, else false if not specified or non boolean specified.
-
useInvokeInGridStart
public boolean useInvokeInGridStart()
Returns a boolean indicating whether to use invoke in kickstart always or not. Referred to by the "pegasus.gridstart.invoke.always" property.- Returns:
- the boolean value specified in the property file, else false if not specified or non boolean specified.
-
disableInvokeInGridStart
public boolean disableInvokeInGridStart()
Returns a boolean indicating whether to disable use of invoke or not. Referred to by the "pegasus.gridstart.invoke.disable" property.- Returns:
- the boolean value specified in the property file, else false if not specified or non boolean specified.
-
getGridStartInvokeLength
public long getGridStartInvokeLength()
Returns the trigger value for invoking an application through kickstart using kickstart. If the arguments value being constructed in the condor submit file is more than this value, then invoke is used to pass the arguments to the remote end. Helps in bypassing the Condor 4K limit. Referred to by "pegasus.gridstart.invoke.length" property.- Returns:
- the long value specified in the properties files, else DEFAULT_INVOKE_LENGTH
- See Also:
DEFAULT_INVOKE_LENGTH
-
generateKickstartExtraOptions
public boolean generateKickstartExtraOptions()
Returns a boolean indicating whehter to pass extra options to kickstart or not. The extra options have appeared only in VDS version 1.4.2 (like -L and -T). Referred to by "pegasus.gridstart.label" property.- Returns:
- the boolean value specified in the property file, else true if not specified or non boolean specified.
-
setPostSCRIPTDebugON
public boolean setPostSCRIPTDebugON()
Returns a boolean indicating whether to turn debug on or not for exitcode. By default false is returned. Referred to by the "pegasus.exitcode.debug" property.- Returns:
- boolean value.
-
symlinkCommonLog
public boolean symlinkCommonLog()
Completely disable placing a symlink for Condor common log (indiscriminately). Starting 4.2.1 this defaults to "false" . Referred to by the "pegasus.condor.logs.symlink" property.- Returns:
- value specified by the property. Defaults to false.
-
useCondorQuotingForArguments
public boolean useCondorQuotingForArguments()
Returns a boolean indicating whether we want to Condor Quote the arguments of the job or not. Referred to by the "pegasus.condor.arguments.quote" property.- Returns:
- boolean
-
getCredName
public java.lang.String getCredName()
Returns the credential name to be used for the stork transfer jobs. Referred to by the "pegasus.transfer.stork.cred" property.- Returns:
- the credential name if specified by the property, else null.
-
getLogManager
public java.lang.String getLogManager()
Returns the log manager to use. Referred to by the "pegasus.log.manager" property.- Returns:
- the value in the properties file, else Default
-
getLogFormatter
public java.lang.String getLogFormatter()
Returns the log formatter to use. Referred to by the "pegasus.log.formatter" property.- Returns:
- the value in the properties file, else Simple
-
getHttpLog4jURL
public java.lang.String getHttpLog4jURL()
Returns the http url for log4j properties for windward project. Referred to by the "log4j.configuration" property.- Returns:
- the value in the properties file, else null
-
getLoggingFile
public java.lang.String getLoggingFile()
Returns the file to which all the logging needs to be directed to. Referred to by the "pegasus.log.*" property.- Returns:
- the value of the property that is specified, else null
-
getLog4JLogFile
public java.lang.String getLog4JLogFile()
Returns the location of the local log file where you want the messages to be logged. Not used for the moment. Referred to by the "pegasus.log4j.log" property.- Returns:
- the value specified in the property file,else null.
-
writeOutMetrics
public boolean writeOutMetrics()
Returns a boolean indicating whether to write out the planner metrics or not. Referred to by the "pegasus.log.metrics" property.- Returns:
- boolean in the properties, else true
-
getMetricsLogFile
public java.lang.String getMetricsLogFile()
Returns the path to the file that is used to be logging metrics Referred to by the "pegasus.log.metrics.file" property.- Returns:
- path to the metrics file if specified, else rundir/pegasus.metrics
-
logMemoryUsage
public boolean logMemoryUsage()
Returns a boolean indicating whether to log JVM memory usage or not. Referred to by the "pegasus.log.memory.usage" property.- Returns:
- boolean value specified in properties else false.
-
assignDefaultJobPriorities
public boolean assignDefaultJobPriorities()
Returns a boolean indicating whether we assign job priorities or not to the jobs Referred to by the "pegasus.job.priority.assign" property.- Returns:
- boolean value specified in properties else true.
-
createRegistrationJobs
public boolean createRegistrationJobs()
Returns a boolean indicating whether we create registration jobs or not. Referred to by the "pegasus.register" property.- Returns:
- boolean value specified in properties else true.
-
executeOnWorkerNode
public boolean executeOnWorkerNode()
Returns a boolean indicating whether to have jobs executing on worker node tmp or not. Referred to by the "pegasus.execute.*.filesystem.local" property.- Returns:
- boolean value in the properties file, else false if not specified or an invalid value specified.
-
treatCacheAsRC
public boolean treatCacheAsRC()
Returns a boolean indicating whether to treat the entries in the cache files as a replica catalog or not.- Returns:
- boolean
-
preserveParserLineBreaks
public boolean preserveParserLineBreaks()
Returns a boolean indicating whether to preserver line breaks. Referred to by the "pegasus.parser.dax.preserve.linebreaks" property.- Returns:
- boolean value in the properties file, else false if not specified or an invalid value specified.
-
addDataDependencies
public boolean addDataDependencies()
Returns a boolean indicating whether to automatically add edges as a result of underlying data dependecnies between jobs. Referred to by the "pegasus.parser.dax.data.dependencies" property.- Returns:
- boolean value in the properties file, else true if not specified or an invalid value specified.
-
getWingsPropertiesFile
public java.lang.String getWingsPropertiesFile()
Returns the path to the wings properties file. Referred to by the "pegasus.wings.properties" property.- Returns:
- value in the properties file, else null.
-
getWingsRequestID
public java.lang.String getWingsRequestID()
Returns the request id. Referred to by the "pegasus.wings.request-id" property.- Returns:
- value in the properties file, else null.
-
getGridFTPTimeout
public java.lang.String getGridFTPTimeout()
Returns the timeout value in seconds after which to timeout in case of opening sockets to grid ftp server. Referred to by the "pegasus.auth.gridftp.timeout" property.- Returns:
- the timeout value if specified else, null.
- See Also:
DEFAULT_SITE_SELECTOR_TIMEOUT
-
getSubmitMode
public java.lang.String getSubmitMode()
Returns which submit mode to be used to submit the jobs on to the grid. Referred to by the "pegasus.code.generator" property.- Returns:
- the submit mode specified in the property file, else the default i.e condor.
-
getPartitionParsingMode
public java.lang.String getPartitionParsingMode()
Returns the mode for parsing the dax while writing out the partitioned daxes. Referred to by the "pegasus.partition.parser.load" property.- Returns:
- the value specified in the properties file, else the default value i.e single.
-
getCollapseFactors
public java.lang.String getCollapseFactors()
Returns a comma separated list for the node collapsing criteria for the execution pools. This determines how many jobs one fat node gobbles up. Referred to by the "pegasus.cluster.nodes" property.- Returns:
- the value specified in the properties file, else null.
-
getHorizontalClusterPreference
public java.lang.String getHorizontalClusterPreference()
Returns the users horizontal clustering preference. This property determines how to cluster horizontal jobs. If this property is set with a value value of runtime, the jobs will be grouped into into clusters according to their runtimes as specified byjob.runtime
property. For all other cases the default horizontal clustering approach will be used.- Returns:
- the value specified in the properties file, else null.
-
getJobAggregator
public java.lang.String getJobAggregator()
Returns what job aggregator is to be used to aggregate multiple compute jobs into a single condor job. Referred to by the "pegasus.cluster.job.aggregator" property.- Returns:
- the value specified in the properties file, else DEFAULT_JOB_AGGREGATOR
- See Also:
DEFAULT_JOB_AGGREGATOR
-
logJobAggregatorProgress
public boolean logJobAggregatorProgress()
Returns whether the seqexec job aggregator should log progress to a log or not. Referred to by the "pegasus.clusterer.job.aggregator.seqexec.log" property.- Returns:
- the value specified in the properties file, else false
-
logJobAggregatorProgressToGlobal
public boolean logJobAggregatorProgressToGlobal()
Returns whether the seqexec job aggregator should write to a global log or not. This comes into play only if "pegasus.clusterer.job.aggregator.seqexec.log" is set to true. Referred to by the "pegasus.clusterer.job.aggregator.seqexec.log.global" property.- Returns:
- the value specified in the properties file, else true
-
abortOnFirstJobFailure
public boolean abortOnFirstJobFailure()
Returns a boolean indicating whether seqexec trips on the first job failure. Referred to by the "pegasus.cluster.job.aggregator.seqexec.firstjobfail" property.- Returns:
- the value specified in the properties file, else true
-
getRootWorkflowUUID
public java.lang.String getRootWorkflowUUID()
Returns the root workflow UUID if defined in the properties, else null Referred to by the "pegasus.workflow.root.uuid" property.- Returns:
- the value in the properties file else, null
-
getPartitionerDAXCallback
public java.lang.String getPartitionerDAXCallback()
Returns the DAXCallback that is to be used while parsing the DAX. Referred to by the "pegasus.partitioner.parser.dax.callback" property.- Returns:
- the value specified in the properties file, else DEFAULT_DAX_CALLBACK
- See Also:
DEFAULT_DAX_CALLBACK
-
getPartitionerLabelKey
public java.lang.String getPartitionerLabelKey()
Returns the key that is to be used as a label key, for labelled partitioning. Referred to by the "pegasus.partitioner.label.key" property.- Returns:
- the value specified in the properties file.
-
getHorizontalPartitionerBundleValue
public java.lang.String getHorizontalPartitionerBundleValue(java.lang.String name)
Returns the bundle value for a particular transformation. Referred to by the "pegasus.partitioner.horziontal.bundle.[txname]" property, where [txname] is replaced by the name passed an input to this function.- Parameters:
name
- the logical name of the transformation.- Returns:
- the path to the postscript if specified in properties file, else null.
-
getHorizontalPartitionerCollapseValue
public java.lang.String getHorizontalPartitionerCollapseValue(java.lang.String name)
Returns the collapse value for a particular transformation. Referred to by the "pegasus.partitioner.horziontal.collapse.[txname]" property, where [txname] is replaced by the name passed an input to this function.- Parameters:
name
- the logical name of the transformation.- Returns:
- the path to the postscript if specified in properties file, else null.
-
getClustererLabelKey
public java.lang.String getClustererLabelKey()
Returns the key that is to be used as a label key, for labelled clustering. Referred to by the "pegasus.clusterer.label.key" property.- Returns:
- the value specified in the properties file.
-
getPropertiesInSubmitDirectory
public java.lang.String getPropertiesInSubmitDirectory()
Returns the path to the property file that has been writting out in the submit directory.- Returns:
- path to the property file
- Throws:
java.lang.RuntimeException
- in case of file not being generated.
-
writeOutProperties
public java.lang.String writeOutProperties(java.lang.String directory) throws java.io.IOException
Writes out the properties to a temporary file in the directory passed.- Parameters:
directory
- the directory in which the properties file needs to be written to.- Returns:
- the absolute path to the properties file written in the directory.
- Throws:
java.io.IOException
- in case of error while writing out file.
-
writeOutProperties
public java.lang.String writeOutProperties(java.lang.String directory, boolean sanitizePath) throws java.io.IOException
Writes out the properties to a temporary file in the directory passed.- Parameters:
directory
- the directory in which the properties file needs to be written to.sanitizePath
- boolean indicating whether to sanitize paths for certain properties or not.- Returns:
- the absolute path to the properties file written in the directory.
- Throws:
java.io.IOException
- in case of error while writing out file.
-
writeOutProperties
public java.lang.String writeOutProperties(java.lang.String directory, boolean sanitizePath, boolean setInternalVariable) throws java.io.IOException
Writes out the properties to a temporary file in the directory passed.- Parameters:
directory
- the directory in which the properties file needs to be written to.sanitizePath
- boolean indicating whether to sanitize paths for certain properties or not.setInternalVariable
- whether to set the internal variable that stores the path to the properties file.- Returns:
- the absolute path to the properties file written in the directory.
- Throws:
java.io.IOException
- in case of error while writing out file.
-
sanitizePathForProperty
private void sanitizePathForProperty(java.util.Properties properties, java.lang.String key)
Santizes the value in the properties . Ensures that the path is absolute.- Parameters:
properties
- the propertieskey
- the key whose value needs to be sanitized
-
sanityCheck
protected static void sanityCheck(java.io.File dir) throws java.io.IOException
Checks the destination location for existence, if it can be created, if it is writable etc.- Parameters:
dir
- is the new base directory to optionally create.- Throws:
java.io.IOException
- in case of error while writing out files.
-
getProperty
private java.lang.String getProperty(java.lang.String newProperty, java.lang.String deprecatedProperty)
This function is used to check whether a deprecated property is used or not. If a deprecated property is used,it logs a warning message specifying the new property. If both properties are not set by the user, the function returns the default property. If no default property then null.- Parameters:
newProperty
- the new property that should be used.deprecatedProperty
- the deprecated property that needs to be replaced.- Returns:
- the appropriate value.
-
getProperty
private java.lang.String getProperty(java.lang.String newProperty, java.lang.String deprecatedProperty, java.lang.String defaultValue)
This function is used to check whether a deprecated property is used or not. If a deprecated property is used,it logs a warning message specifying the new property. If both properties are not set by the user, the function returns the default property. If no default property then null.- Parameters:
newProperty
- the new property that should be used.deprecatedProperty
- the deprecated property that needs to be replaced.defaultValue
- the default value that should be returned.- Returns:
- the appropriate value.
-
logDeprecatedWarning
private void logDeprecatedWarning(java.lang.String deprecatedProperty, java.lang.String newProperty)
Logs a warning about the deprecated property. Logs a warning only if it has not been displayed before.- Parameters:
deprecatedProperty
- the deprecated property that needs to be replaced.newProperty
- the new property that should be used.
-
-