Interface Implementation

  • All Known Implementing Classes:
    Abstract, AbstractMultipleFTPerXFERJob, AbstractSingleFTPerXFERJob, GUC, Stork, T2, TPTGUC, Transfer

    public interface Implementation
    The interface defines the functions that a particular Transfer Implementation should implement. The functions deal with the creation of a TransferJob that can transfer files using the transfer tool to which it refers to.
    Version:
    $Revision$
    Author:
    Karan Vahi, Gaurang Mehta
    • Field Summary

      Fields 
      Modifier and Type Field Description
      static java.lang.String TRANSFER_UNIVERSE
      The universe that applies for the transfer jobs.
      static java.lang.String VERSION
      The version number associated with this API
    • Method Summary

      All Methods Instance Methods Abstract Methods 
      Modifier and Type Method Description
      boolean addSetXBitJobs​(Job computeJob, java.lang.String txJobName, java.util.Collection execFiles, int transferClass, int xbitIndex)
      Adds the dirmanager job to the workflow, that do a chmod on the files being staged.
      void applyPriority​(TransferJob job)
      Applies priorities to the transfer jobs if a priority is specified in the properties file.
      boolean checkAndTransferProxy​(TransferJob job)
      Determines if there is a need to transfer proxy for the transfer job or not.
      Job createSetXBitJob​(Job computeJob, java.util.Collection<FileTransfer> execFiles, int transferClass, int xbitIndex)
      Adds the dirmanager job to the workflow, that do a chmod on the files being staged.
      TransferJob createTransferJob​(Job job, java.lang.String site, java.util.Collection files, java.util.Collection execFiles, java.lang.String txJobName, int jobClass)
      This constructs the Job object for the transfer node.
      boolean doesPreserveXBit()
      Returns a boolean indicating whether the transfer protocol being used by the implementation preserves the X Bit or not while staging.
      java.lang.String getDescription()
      Returns a textual description of the transfer implementation.
      java.lang.String getSetXBitJobName​(java.lang.String name, int counter)
      Generates the name of the setXBitJob , that is unique for the given workflow.
      TransformationCatalogEntry getTransformationCatalogEntry​(java.lang.String siteHandle, int jobClass)
      Retrieves the transformation catalog entry for the executable that is being used to transfer the files in the implementation.
      void setRefiner​(Refiner refiner)
      Sets the callback to the refiner, that has loaded this implementation.
      boolean useThirdPartyTransferAlways()
      Return a boolean indicating whether the transfers to be done always in a third party transfer mode.
    • Field Detail

      • VERSION

        static final java.lang.String VERSION
        The version number associated with this API
        See Also:
        Constant Field Values
      • TRANSFER_UNIVERSE

        static final java.lang.String TRANSFER_UNIVERSE
        The universe that applies for the transfer jobs. Used for querying to the Site Catalog.
        See Also:
        Constant Field Values
    • Method Detail

      • setRefiner

        void setRefiner​(Refiner refiner)
        Sets the callback to the refiner, that has loaded this implementation.
        Parameters:
        refiner - the transfer refiner that loaded the implementation.
      • createTransferJob

        TransferJob createTransferJob​(Job job,
                                      java.lang.String site,
                                      java.util.Collection files,
                                      java.util.Collection execFiles,
                                      java.lang.String txJobName,
                                      int jobClass)
        This constructs the Job object for the transfer node. The transfer is supposed to occur at job execution site. It should lead to the creation of the setup chmod jobs to the workflow, that appear as parents to compute jobs in case the transfer implementation does not preserve the X bit on the file being transferred. This is required for staging of executables as part of the workflow.
        Parameters:
        job - the Job object for the job, in relation to which the transfer node is being added. Either the transfer node can be transferring this jobs input files to the execution pool, or transferring this job's output files to the output pool.
        site - the site where the transfer job should run.
        files - collection of FileTransfer objects representing the data files and staged executables to be transferred.
        execFiles - subset collection of the files parameter, that identifies the executable files that are being transferred.
        txJobName - the name of transfer node.
        jobClass - the job Class for the newly added job. Can be one of the following: stage-in stage-out inter-pool transfer
        Returns:
        the created TransferJob.
      • doesPreserveXBit

        boolean doesPreserveXBit()
        Returns a boolean indicating whether the transfer protocol being used by the implementation preserves the X Bit or not while staging. If it does not, then it should extend the Abstract implementation of this interface, that allows for adding of a setup job after the stagein job that changes the X Bit.
      • addSetXBitJobs

        boolean addSetXBitJobs​(Job computeJob,
                               java.lang.String txJobName,
                               java.util.Collection execFiles,
                               int transferClass,
                               int xbitIndex)
        Adds the dirmanager job to the workflow, that do a chmod on the files being staged.
        Parameters:
        computeJob - the computeJob for which the files are being staged.
        txJobName - the name of the transfer job that is staging the files.
        execFiles - the executable files that are being staged.
        transferClass - the class of transfer job
        xbitIndex - index to be used for creating the name of XBitJob.
        Returns:
        boolean indicating whether any XBitJobs were succesfully added or not.
      • createSetXBitJob

        Job createSetXBitJob​(Job computeJob,
                             java.util.Collection<FileTransfer> execFiles,
                             int transferClass,
                             int xbitIndex)
        Adds the dirmanager job to the workflow, that do a chmod on the files being staged.
        Parameters:
        computeJob - the computeJob for which the files are being staged.
        execFiles - the executable files that are being staged.
        transferClass - the class of transfer job
        xbitIndex - index to be used for creating the name of XBitJob.
        Returns:
        the job object for the xBitJob
      • getSetXBitJobName

        java.lang.String getSetXBitJobName​(java.lang.String name,
                                           int counter)
        Generates the name of the setXBitJob , that is unique for the given workflow. If the implementation preserve the X bit, then it should return null.
        Parameters:
        name - the name of the compute job for which the executable is being staged.
        counter - the index for the setXBit job.
        Returns:
        the name of the setXBitJob, null in case the implementation preserves the XBit.
      • getTransformationCatalogEntry

        TransformationCatalogEntry getTransformationCatalogEntry​(java.lang.String siteHandle,
                                                                 int jobClass)
        Retrieves the transformation catalog entry for the executable that is being used to transfer the files in the implementation.
        Parameters:
        siteHandle - the handle of the site where the transformation is to be searched.
        jobClass - the job Class for the newly added job. Can be one of the following: stage-in stage-out inter-pool transfer stage-in worker transfer
        Returns:
        the transformation catalog entry if found, else null.
      • useThirdPartyTransferAlways

        boolean useThirdPartyTransferAlways()
        Return a boolean indicating whether the transfers to be done always in a third party transfer mode. A value of false, results in the direct or peer to peer transfers being done.

        A value of false does not preclude third party transfers. They still can be done, by setting the property "pegasus.transfer.*.thirdparty.sites".

        Returns:
        boolean indicating whether to always use third party transfers or not.
        See Also:
        PegasusProperties.getThirdPartySites(String)
      • applyPriority

        void applyPriority​(TransferJob job)
        Applies priorities to the transfer jobs if a priority is specified in the properties file.
        Parameters:
        job - the transfer job .
      • checkAndTransferProxy

        boolean checkAndTransferProxy​(TransferJob job)
        Determines if there is a need to transfer proxy for the transfer job or not. If there is a need to transfer proxy, then the job is modified to create the correct condor commands to transfer the proxy. Proxy is usually transferred if the VDS profile TRANSFER_PROXY is set, or the job is being run in the condor vanilla universe. The proxy is transferred from the submit host (i.e site local). The location is determined from the value of the X509_USER_PROXY profile key associated in the env namespace.
        Parameters:
        job - the transfer job .
        Returns:
        boolean true job was modified to transfer the proxy, else false when job is not modified.
      • getDescription

        java.lang.String getDescription()
        Returns a textual description of the transfer implementation.
        Returns:
        a short textual description