Class AbstractMultipleFTPerXFERJob
- java.lang.Object
-
- edu.isi.pegasus.planner.transfer.implementation.Abstract
-
- edu.isi.pegasus.planner.transfer.implementation.AbstractMultipleFTPerXFERJob
-
- All Implemented Interfaces:
Implementation
,MultipleFTPerXFERJob
public abstract class AbstractMultipleFTPerXFERJob extends Abstract implements MultipleFTPerXFERJob
An abstract implementation for implementations that can handle multiple file transfers in a single file transfer job.- Version:
- $Revision$
- Author:
- Karan Vahi
-
-
Field Summary
-
Fields inherited from class edu.isi.pegasus.planner.transfer.implementation.Abstract
CHANGE_XBIT_TRANSFORMATION, mAddNodesForSettingXBit, mChmodDisabledForAllSites, mDisabledChmodSites, mLocalIrodsEnv, mLocalIrodsEnvBasename, mLocalS3cfg, mLocalS3cfgBasename, mLocalUserProxy, mLocalUserProxyBasename, mLogger, mPOptions, mProps, mRefiner, mSiteStore, mTCHandle, NOOP_PREFIX, SET_XBIT_PREFIX, XBIT_DERIVATION_NS, XBIT_DERIVATION_VERSION, XBIT_EXECUTABLE_BASENAME, XBIT_TRANSFORMATION_NS, XBIT_TRANSFORMATION_VERSION
-
Fields inherited from interface edu.isi.pegasus.planner.transfer.Implementation
TRANSFER_UNIVERSE, VERSION
-
-
Constructor Summary
Constructors Constructor Description AbstractMultipleFTPerXFERJob(PegasusBag bag)
The overloaded constructor, that is called by the Factory to load the class.
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description TransferJob
createTransferJob(Job job, java.lang.String site, java.util.Collection files, java.util.Collection execFiles, java.lang.String txJobName, int jobClass)
Constructs a general transfer job that handles multiple transfers per transfer job.protected TransformationCatalogEntry
defaultTCEntry(java.lang.String namespace, java.lang.String name, java.lang.String version, java.lang.String executableBasename, java.lang.String site)
Returns a default TC entry to be used in case entry is not found in the transformation catalog.protected abstract java.lang.String
generateArgumentString(TransferJob job)
It constructs the arguments to the transfer executable that need to be passed to the executable referred to in this transfer mode.protected abstract java.lang.String
getCompleteTCName()
Returns the complete name for the transformation that the implementation is using..protected abstract java.lang.String
getDerivationName()
Returns the logical name of the derivation that this implementation refers to.protected abstract java.lang.String
getDerivationNamespace()
Returns the namespace of the derivation that this implementation refers to.protected abstract java.lang.String
getDerivationVersion()
Returns the version of the derivation that this implementation refers to.protected abstract java.util.List<Profile>
getEnvironmentVariables(java.lang.String site)
Returns the environment profiles that are required for the default entry to sensibly work.void
postProcess(TransferJob job)
An optional method that allows the derived classes to do their own post processing on the the transfer job before it is returned to the calling module.protected java.lang.String
prepareSTDINAndAssociateCredentials(TransferJob job, java.util.Collection files, java.lang.String stagingSite, int jobClass)
Prepares the stdin for the transfer job.protected abstract void
writeStdInAndAssociateCredentials(TransferJob job, java.io.FileWriter stdIn, java.util.Collection files, java.lang.String stagingSite, int jobClass)
Writes to a FileWriter stream the stdin which goes into the magic script via standard input-
Methods inherited from class edu.isi.pegasus.planner.transfer.implementation.Abstract
addSetXBitJobs, addSetXBitJobs, addSetXBitJobs, applyPriority, checkAndTransferIrodsEnvFile, checkAndTransferProxy, construct, createNoOPJob, createSetXBitJob, createSetXBitJob, createSetXBitJob, determineDisabledChmodSites, disableChmodJobCreation, getNOOPJobName, getPriority, getSetXBitJobName, setRefiner
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface edu.isi.pegasus.planner.transfer.Implementation
doesPreserveXBit, getDescription, getTransformationCatalogEntry, useThirdPartyTransferAlways
-
-
-
-
Constructor Detail
-
AbstractMultipleFTPerXFERJob
public AbstractMultipleFTPerXFERJob(PegasusBag bag)
The overloaded constructor, that is called by the Factory to load the class.- Parameters:
bag
- the bag of Pegasus initialization objects
-
-
Method Detail
-
createTransferJob
public TransferJob createTransferJob(Job job, java.lang.String site, java.util.Collection files, java.util.Collection execFiles, java.lang.String txJobName, int jobClass)
Constructs a general transfer job that handles multiple transfers per transfer job. There are appropriate callouts to generate the implementation specific details.- Specified by:
createTransferJob
in interfaceImplementation
- Parameters:
job
- the Job object for the job, in relation to which the transfer node is being added. Either the transfer node can be transferring this jobs input files to the execution pool, or transferring this job's output files to the output pool.site
- the site where the transfer job should run.files
- collection ofFileTransfer
objects representing the data files and staged executables to be transferred.execFiles
- subset collection of the files parameter, that identifies the executable files that are being transferred.txJobName
- the name of transfer node.jobClass
- the job Class for the newly added job. Can be one of the following: stage-in stage-out inter-pool transfer stage-in worker transfer- Returns:
- the created TransferJob.
-
defaultTCEntry
protected TransformationCatalogEntry defaultTCEntry(java.lang.String namespace, java.lang.String name, java.lang.String version, java.lang.String executableBasename, java.lang.String site)
Returns a default TC entry to be used in case entry is not found in the transformation catalog.- Parameters:
namespace
- the namespace of the transfer transformationname
- the logical name of the transfer transformationversion
- the version of the transfer transformationexecutableBasename
- the basename of the executablesite
- the site for which the default entry is required.- Returns:
- the default entry.
-
getEnvironmentVariables
protected abstract java.util.List<Profile> getEnvironmentVariables(java.lang.String site)
Returns the environment profiles that are required for the default entry to sensibly work.- Parameters:
site
- the site where the job is going to run.- Returns:
- List of environment variables, else null in case where the required environment variables could not be found.
-
postProcess
public void postProcess(TransferJob job)
An optional method that allows the derived classes to do their own post processing on the the transfer job before it is returned to the calling module.- Parameters:
job
- theTransferJob
that has been created.
-
prepareSTDINAndAssociateCredentials
protected java.lang.String prepareSTDINAndAssociateCredentials(TransferJob job, java.util.Collection files, java.lang.String stagingSite, int jobClass) throws java.lang.Exception
Prepares the stdin for the transfer job. Usually involves writing out a text file that Condor transfers to the remote end. Additionally, it associates credentials with the job that are requried to for the transfers.- Parameters:
name
- the name of the transfer job.files
- Collection ofFileTransfer
objects containing the information about sourceam fin and destURL's.stagingSite
- the site where the data will be populated by first level staging jobs.jobClass
- the job Class for the newly added job. Can be one of the following: stage-in stage-out inter-pool transfer- Returns:
- the path to the prepared stdin file.
- Throws:
java.lang.Exception
- in case of error.
-
getDerivationNamespace
protected abstract java.lang.String getDerivationNamespace()
Returns the namespace of the derivation that this implementation refers to.- Returns:
- the namespace of the derivation.
-
getDerivationName
protected abstract java.lang.String getDerivationName()
Returns the logical name of the derivation that this implementation refers to.- Returns:
- the name of the derivation.
-
getDerivationVersion
protected abstract java.lang.String getDerivationVersion()
Returns the version of the derivation that this implementation refers to.- Returns:
- the version of the derivation.
-
generateArgumentString
protected abstract java.lang.String generateArgumentString(TransferJob job)
It constructs the arguments to the transfer executable that need to be passed to the executable referred to in this transfer mode.- Parameters:
job
- the object containing the transfer node.- Returns:
- the argument string
-
writeStdInAndAssociateCredentials
protected abstract void writeStdInAndAssociateCredentials(TransferJob job, java.io.FileWriter stdIn, java.util.Collection files, java.lang.String stagingSite, int jobClass) throws java.lang.Exception
Writes to a FileWriter stream the stdin which goes into the magic script via standard input- Parameters:
job
- the transfer job .stdIn
- the writer to the stdin file.files
- Collection ofFileTransfer
objects containing the information about sourceam fin and destURL's.stagingSite
- the site where the data will be populated by first level staging jobs.jobClass
- the job Class for the newly added job. Can be one of the following: stage-in stage-out inter-pool transfer- Throws:
java.lang.Exception
-
getCompleteTCName
protected abstract java.lang.String getCompleteTCName()
Returns the complete name for the transformation that the implementation is using..- Returns:
- the complete name.
-
-