Package edu.isi.pegasus.planner.transfer
Interface Implementation
-
- All Known Implementing Classes:
Abstract
,AbstractMultipleFTPerXFERJob
,AbstractSingleFTPerXFERJob
,GUC
,Stork
,T2
,TPTGUC
,Transfer
public interface Implementation
The interface defines the functions that a particular Transfer Implementation should implement. The functions deal with the creation of a TransferJob that can transfer files using the transfer tool to which it refers to.- Version:
- $Revision$
- Author:
- Karan Vahi, Gaurang Mehta
-
-
Field Summary
Fields Modifier and Type Field Description static java.lang.String
TRANSFER_UNIVERSE
The universe that applies for the transfer jobs.static java.lang.String
VERSION
The version number associated with this API
-
Method Summary
All Methods Instance Methods Abstract Methods Modifier and Type Method Description boolean
addSetXBitJobs(Job computeJob, java.lang.String txJobName, java.util.Collection execFiles, int transferClass, int xbitIndex)
Adds the dirmanager job to the workflow, that do a chmod on the files being staged.void
applyPriority(TransferJob job)
Applies priorities to the transfer jobs if a priority is specified in the properties file.boolean
checkAndTransferProxy(TransferJob job)
Determines if there is a need to transfer proxy for the transfer job or not.Job
createSetXBitJob(Job computeJob, java.util.Collection<FileTransfer> execFiles, int transferClass, int xbitIndex)
Adds the dirmanager job to the workflow, that do a chmod on the files being staged.TransferJob
createTransferJob(Job job, java.lang.String site, java.util.Collection files, java.util.Collection execFiles, java.lang.String txJobName, int jobClass)
This constructs the Job object for the transfer node.boolean
doesPreserveXBit()
Returns a boolean indicating whether the transfer protocol being used by the implementation preserves the X Bit or not while staging.java.lang.String
getDescription()
Returns a textual description of the transfer implementation.java.lang.String
getSetXBitJobName(java.lang.String name, int counter)
Generates the name of the setXBitJob , that is unique for the given workflow.TransformationCatalogEntry
getTransformationCatalogEntry(java.lang.String siteHandle, int jobClass)
Retrieves the transformation catalog entry for the executable that is being used to transfer the files in the implementation.void
setRefiner(Refiner refiner)
Sets the callback to the refiner, that has loaded this implementation.boolean
useThirdPartyTransferAlways()
Return a boolean indicating whether the transfers to be done always in a third party transfer mode.
-
-
-
Field Detail
-
VERSION
static final java.lang.String VERSION
The version number associated with this API- See Also:
- Constant Field Values
-
TRANSFER_UNIVERSE
static final java.lang.String TRANSFER_UNIVERSE
The universe that applies for the transfer jobs. Used for querying to the Site Catalog.- See Also:
- Constant Field Values
-
-
Method Detail
-
setRefiner
void setRefiner(Refiner refiner)
Sets the callback to the refiner, that has loaded this implementation.- Parameters:
refiner
- the transfer refiner that loaded the implementation.
-
createTransferJob
TransferJob createTransferJob(Job job, java.lang.String site, java.util.Collection files, java.util.Collection execFiles, java.lang.String txJobName, int jobClass)
This constructs the Job object for the transfer node. The transfer is supposed to occur at job execution site. It should lead to the creation of the setup chmod jobs to the workflow, that appear as parents to compute jobs in case the transfer implementation does not preserve the X bit on the file being transferred. This is required for staging of executables as part of the workflow.- Parameters:
job
- the Job object for the job, in relation to which the transfer node is being added. Either the transfer node can be transferring this jobs input files to the execution pool, or transferring this job's output files to the output pool.site
- the site where the transfer job should run.files
- collection ofFileTransfer
objects representing the data files and staged executables to be transferred.execFiles
- subset collection of the files parameter, that identifies the executable files that are being transferred.txJobName
- the name of transfer node.jobClass
- the job Class for the newly added job. Can be one of the following: stage-in stage-out inter-pool transfer- Returns:
- the created TransferJob.
-
doesPreserveXBit
boolean doesPreserveXBit()
Returns a boolean indicating whether the transfer protocol being used by the implementation preserves the X Bit or not while staging. If it does not, then it should extend the Abstract implementation of this interface, that allows for adding of a setup job after the stagein job that changes the X Bit.
-
addSetXBitJobs
boolean addSetXBitJobs(Job computeJob, java.lang.String txJobName, java.util.Collection execFiles, int transferClass, int xbitIndex)
Adds the dirmanager job to the workflow, that do a chmod on the files being staged.- Parameters:
computeJob
- the computeJob for which the files are being staged.txJobName
- the name of the transfer job that is staging the files.execFiles
- the executable files that are being staged.transferClass
- the class of transfer jobxbitIndex
- index to be used for creating the name of XBitJob.- Returns:
- boolean indicating whether any XBitJobs were succesfully added or not.
-
createSetXBitJob
Job createSetXBitJob(Job computeJob, java.util.Collection<FileTransfer> execFiles, int transferClass, int xbitIndex)
Adds the dirmanager job to the workflow, that do a chmod on the files being staged.- Parameters:
computeJob
- the computeJob for which the files are being staged.execFiles
- the executable files that are being staged.transferClass
- the class of transfer jobxbitIndex
- index to be used for creating the name of XBitJob.- Returns:
- the job object for the xBitJob
-
getSetXBitJobName
java.lang.String getSetXBitJobName(java.lang.String name, int counter)
Generates the name of the setXBitJob , that is unique for the given workflow. If the implementation preserve the X bit, then it should return null.- Parameters:
name
- the name of the compute job for which the executable is being staged.counter
- the index for the setXBit job.- Returns:
- the name of the setXBitJob, null in case the implementation preserves the XBit.
-
getTransformationCatalogEntry
TransformationCatalogEntry getTransformationCatalogEntry(java.lang.String siteHandle, int jobClass)
Retrieves the transformation catalog entry for the executable that is being used to transfer the files in the implementation.- Parameters:
siteHandle
- the handle of the site where the transformation is to be searched.jobClass
- the job Class for the newly added job. Can be one of the following: stage-in stage-out inter-pool transfer stage-in worker transfer- Returns:
- the transformation catalog entry if found, else null.
-
useThirdPartyTransferAlways
boolean useThirdPartyTransferAlways()
Return a boolean indicating whether the transfers to be done always in a third party transfer mode. A value of false, results in the direct or peer to peer transfers being done.A value of false does not preclude third party transfers. They still can be done, by setting the property "pegasus.transfer.*.thirdparty.sites".
- Returns:
- boolean indicating whether to always use third party transfers or not.
- See Also:
PegasusProperties.getThirdPartySites(String)
-
applyPriority
void applyPriority(TransferJob job)
Applies priorities to the transfer jobs if a priority is specified in the properties file.- Parameters:
job
- the transfer job .
-
checkAndTransferProxy
boolean checkAndTransferProxy(TransferJob job)
Determines if there is a need to transfer proxy for the transfer job or not. If there is a need to transfer proxy, then the job is modified to create the correct condor commands to transfer the proxy. Proxy is usually transferred if the VDS profile TRANSFER_PROXY is set, or the job is being run in the condor vanilla universe. The proxy is transferred from the submit host (i.e site local). The location is determined from the value of the X509_USER_PROXY profile key associated in the env namespace.- Parameters:
job
- the transfer job .- Returns:
- boolean true job was modified to transfer the proxy, else false when job is not modified.
-
getDescription
java.lang.String getDescription()
Returns a textual description of the transfer implementation.- Returns:
- a short textual description
-
-