Class PoolInfoProvider

  • Direct Known Subclasses:
    Abstract

    public abstract class PoolInfoProvider
    extends java.lang.Object
    This is an abstract class which defines the interface for the information providers like sites.xml, sites.catalog.
    Version:
    $Revision$
    Author:
    Karan Vahi, Gaurang Mehta
    • Field Summary

      Fields 
      Modifier and Type Field Description
      protected boolean mDeepStorageStructure
      A boolean indicating whether to have a deep directory structure for the storage directory or not.
      protected LogManager mLogger
      The LogManager object which is used to log all the messages.
      protected java.lang.String mLogMsg
      The String holding the log message.
      protected java.lang.String mPoolProvider
      The path to the pool information provider.
      protected PegasusProperties mProps
      The object holding all the properties pertaining to Pegasus.
      protected java.lang.String mStorageDir
      This contains the storage directory relative to the se mount point of the pool.
      protected UserOptions mUserOpts
      Handle to the Singleton instance containing the options passed to the planner at run time.
      protected java.lang.String mWorkDir
      The working directory relative to the mount point of the execution pool.
      static java.lang.String PEGASUS_HOME
      The name of the environment variable PEGASUS_HOME.
      static java.lang.String VDS_HOME
      The name of the environment variable VDS_HOME.
    • Method Summary

      All Methods Static Methods Instance Methods Abstract Methods Concrete Methods 
      Modifier and Type Method Description
      java.lang.String getEnvironmentVariable​(java.lang.String siteID, java.lang.String envVariable)
      Returns an environment variable for a particular site set in the Site Catalog.
      java.lang.String getExecPoolWorkDir​(Job job)
      This determines the working directory on remote execution pool for a particular job.
      java.lang.String getExecPoolWorkDir​(java.lang.String executionPool)
      This determines the working directory on remote execution pool on the basis of whether an absolute path is specified in the pegasus.dir.exec directory or a relative path.
      java.lang.String getExecPoolWorkDir​(java.lang.String siteID, java.lang.String path)
      This determines the working directory on remote execution pool on the basis of whether an absolute path is specified in the pegasus.dir.exec directory or a relative path.
      java.lang.String getExecPoolWorkDir​(java.lang.String siteID, java.lang.String path, int jobClass)
      This determines the working directory on remote execution pool on the basis of whether an absolute path is specified in the pegasus.dir.exec directory or a relative path.
      abstract java.util.List getGridFTPServers​(java.lang.String siteID)
      It returns all the gridftp servers corresponding to a specified pool.
      abstract java.util.List getJobmanagers​(java.lang.String siteID)
      It returns all the jobmanagers corresponding to a specified site.
      abstract java.util.List getJobmanagers​(java.lang.String siteID, java.lang.String universe)
      It returns all the jobmanagers corresponding to a specified pool and universe.
      java.lang.String getPegasusHome​(java.lang.String siteID)
      Returns the value of PEGASUS_HOME for a site.
      abstract SiteInfo getPoolEntry​(java.lang.String siteID, java.lang.String universe)
      Gets the pool information from the pool.config file on the basis of the name of the pool, and the universe.
      abstract java.lang.String getPoolMode()
      Returns a textual description of the pool mode being used.
      abstract java.util.List getPoolProfile​(java.lang.String siteID)
      It returns the profile information associated with a particular pool.
      java.util.List getPoolProfile​(java.lang.String siteID, java.lang.String namespace)
      It returns profile information associated with a particular namespace and pool.
      abstract java.util.List getPools()
      It returns all the pools available in the site catalog
      java.lang.String getSeMountPoint​(SiteInfo site)
      Return the storage mount point for a particular pool.
      abstract VDSSysInfo getSysinfo​(java.lang.String siteID)
      Returns the System information for a single site.
      abstract java.util.Map getSysinfos​(java.util.List siteids)
      Returns the System information for a bunch of sites.
      SiteInfo getTXPoolEntry​(java.lang.String poolName)
      Gets the pool object to be used for the transfer universe.
      java.lang.String getURLPrefix​(java.lang.String poolName)
      Returns the url prefix of a gridftp server on the pool.
      java.lang.String getVDS_HOME​(java.lang.String siteID)
      Returns the value of VDS_HOME for a site.
      void loadNonSingletonObjects​(java.lang.String propFileName)
      It loads the objects using their non singleton implementations.
      protected void loadSingletonObjects()
      It loads the objects that the pool providers need in a singleton manner, wherever possible.
      protected void logMessage​(java.lang.String msg)
      Logs the message to a logging stream.
      static PoolInfoProvider nonSingletonInstance​(java.lang.String poolProvider, java.lang.String propFileName)
      The method that returns a Non Singleton instance of the dervived InfoProvider class.
      abstract boolean removeGridFtp​(java.lang.String siteID, java.lang.String urlPrefix)
      This is a soft state remove, that removes a gridftp server from a particular pool entry.
      abstract boolean removeJobManager​(java.lang.String siteID, java.lang.String universe, java.lang.String jobManagerContact)
      This is a soft state remove, that removes a jobmanager from a particular pool entry.
      GridFTPServer selectGridFtp​(java.util.ArrayList ftp)
      Return a random gridftp url from the list of gridftp url's.
      java.lang.String selectLRC​(java.util.ArrayList lrcs)
      Return a random lrc url from the list of lrc url's.
      java.lang.String selectWorkdir​(WorkDir workdir)
      Returns the path to the execution mount point (The Workdir).
      static PoolInfoProvider singletonInstance​(java.lang.String poolProvider)
      The method which returns a Singleton instance of the derived InfoProvider class.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • PEGASUS_HOME

        public static final java.lang.String PEGASUS_HOME
        The name of the environment variable PEGASUS_HOME.
        See Also:
        Constant Field Values
      • VDS_HOME

        public static final java.lang.String VDS_HOME
        The name of the environment variable VDS_HOME.
        See Also:
        Constant Field Values
      • mLogger

        protected LogManager mLogger
        The LogManager object which is used to log all the messages. It's values are set in the CPlanner class.
      • mLogMsg

        protected java.lang.String mLogMsg
        The String holding the log message.
      • mPoolProvider

        protected java.lang.String mPoolProvider
        The path to the pool information provider.
      • mProps

        protected PegasusProperties mProps
        The object holding all the properties pertaining to Pegasus.
      • mWorkDir

        protected java.lang.String mWorkDir
        The working directory relative to the mount point of the execution pool. It is populated from the pegasus.dir.exec property from the properties file. If not specified then it work_dir is supposed to be the exec mount point of the execution pool.
      • mStorageDir

        protected java.lang.String mStorageDir
        This contains the storage directory relative to the se mount point of the pool. It is populated from the pegasus.dir.storage property from the properties file. If not specified then the storage directory is the se mount point from the pool.config file.
      • mUserOpts

        protected UserOptions mUserOpts
        Handle to the Singleton instance containing the options passed to the planner at run time.
      • mDeepStorageStructure

        protected boolean mDeepStorageStructure
        A boolean indicating whether to have a deep directory structure for the storage directory or not.
    • Constructor Detail

      • PoolInfoProvider

        public PoolInfoProvider()
    • Method Detail

      • singletonInstance

        public static PoolInfoProvider singletonInstance​(java.lang.String poolProvider)
        The method which returns a Singleton instance of the derived InfoProvider class. It must be overriden, by implementing classes.
        Parameters:
        poolProvider - the url to site catalog source. Can be a URL.
        Returns:
        PoolInfoProvider
      • nonSingletonInstance

        public static PoolInfoProvider nonSingletonInstance​(java.lang.String poolProvider,
                                                            java.lang.String propFileName)
        The method that returns a Non Singleton instance of the dervived InfoProvider class. This method if invoked should also ensure that all other internal Pegasus objectslike PegasusProperties are invoked in a non singleton manner. It must be overriden, by implementing classes.
        Parameters:
        poolProvider - the path to the file containing the pool information.
        propFileName - the name of the properties file that needs to be picked up from PEGASUS_HOME/etc directory.If it is null, then the default file should be picked up.
        Returns:
        PoolInfoProvider
      • loadSingletonObjects

        protected void loadSingletonObjects()
        It loads the objects that the pool providers need in a singleton manner, wherever possible. If the class in not implemented in Singleton manner, the objects would be loaded normally.
      • loadNonSingletonObjects

        public void loadNonSingletonObjects​(java.lang.String propFileName)
        It loads the objects using their non singleton implementations.
        Parameters:
        propFileName - the name of the properties file that needs to be picked up from PEGASUS_HOME/etc directory.If it is null, then the default properties file should be picked up.
      • getSysinfos

        public abstract java.util.Map getSysinfos​(java.util.List siteids)
        Returns the System information for a bunch of sites.
        Parameters:
        siteids - List The siteid whose system information is required
        Returns:
        Map The key is the siteid and the value is a VDSSysInfo object
        See Also:
        org.griphyn.common.classes.VDSSysInfo
      • getSysinfo

        public abstract VDSSysInfo getSysinfo​(java.lang.String siteID)
        Returns the System information for a single site.
        Parameters:
        siteID - String The site whose system information is requested
        Returns:
        VDSSysInfo The system information as a VDSSysInfo object
        See Also:
        org.griphyn.common.classes.VDSSysInfo
      • getPoolEntry

        public abstract SiteInfo getPoolEntry​(java.lang.String siteID,
                                              java.lang.String universe)
        Gets the pool information from the pool.config file on the basis of the name of the pool, and the universe.
        Parameters:
        siteID - the name of the site
        universe - the execution universe for the job
        Returns:
        the corresponding pool object for the entry if found else null
      • getPoolProfile

        public abstract java.util.List getPoolProfile​(java.lang.String siteID)
        It returns the profile information associated with a particular pool. If the pool provider has no such information it should return null. The name of the object may purport that it is specific to GVDS format, but in fact it a tuple consisting of namespace, key and value that can be used by other Pool providers too.
        Parameters:
        siteID - the name of the site, whose profile information you want.
        Returns:
        List of Profile objects null if the information about the site is not with the pool provider.
        See Also:
        org.griphyn.cPlanner.classes.Profile
      • getJobmanagers

        public abstract java.util.List getJobmanagers​(java.lang.String siteID)
        It returns all the jobmanagers corresponding to a specified site.
        Parameters:
        siteID - the name of the site at which the jobmanager runs.
        Returns:
        list of JobManager, each referring to one jobmanager contact string. An empty list if no jobmanagers found.
      • getJobmanagers

        public abstract java.util.List getJobmanagers​(java.lang.String siteID,
                                                      java.lang.String universe)
        It returns all the jobmanagers corresponding to a specified pool and universe.
        Parameters:
        siteID - the name of the site at which the jobmanager runs.
        universe - the gvds universe with which it is associated.
        Returns:
        list of JobManager, each referring to one jobmanager contact string. An empty list if no jobmanagers found.
      • getGridFTPServers

        public abstract java.util.List getGridFTPServers​(java.lang.String siteID)
        It returns all the gridftp servers corresponding to a specified pool.
        Parameters:
        siteID - the name of the site at which the jobmanager runs.
        Returns:
        List of GridFTPServer, each referring to one GridFtp Server.
      • getPools

        public abstract java.util.List getPools()
        It returns all the pools available in the site catalog
        Returns:
        List of names of the pools available as String
      • removeJobManager

        public abstract boolean removeJobManager​(java.lang.String siteID,
                                                 java.lang.String universe,
                                                 java.lang.String jobManagerContact)
        This is a soft state remove, that removes a jobmanager from a particular pool entry. The cause of this removal could be the inability to authenticate against it at runtime. The successful removal lead Pegasus not to schedule job on that particular jobmanager.
        Parameters:
        siteID - the name of the site at which the jobmanager runs.
        universe - the gvds universe with which it is associated.
        jobManagerContact - the contact string to the jobmanager.
        Returns:
        true if was able to remove the jobmanager from the cache false if unable to remove, or the matching entry is not found or if the implementing class does not maintain a soft state.
      • removeGridFtp

        public abstract boolean removeGridFtp​(java.lang.String siteID,
                                              java.lang.String urlPrefix)
        This is a soft state remove, that removes a gridftp server from a particular pool entry. The cause of this removal could be the inability to authenticate against it at runtime. The successful removal lead Pegasus not to schedule any transfers on that particular gridftp server.
        Parameters:
        siteID - the name of the site at which the gridftp runs.
        urlPrefix - the url prefix containing the protocol,hostname and port.
        Returns:
        true if was able to remove the gridftp from the cache false if unable to remove, or the matching entry is not found or if the implementing class does not maintain a soft state. or the information about site is not in the site catalog.
      • getPoolMode

        public abstract java.lang.String getPoolMode()
        Returns a textual description of the pool mode being used.
        Returns:
        String
      • selectLRC

        public java.lang.String selectLRC​(java.util.ArrayList lrcs)
        Return a random lrc url from the list of lrc url's.
        Parameters:
        lrcs - Arraylist of LRC objects.
        Returns:
        String Returns one of lrc url's
        See Also:
        org.griphyn.cPlanner.classes.LRC
      • selectWorkdir

        public java.lang.String selectWorkdir​(WorkDir workdir)
                                       throws java.lang.Exception
        Returns the path to the execution mount point (The Workdir).
        Parameters:
        workdir - the WorkDir object containing the workdir information.
        Returns:
        String The exec-mount point (aka workdir)
        Throws:
        java.lang.Exception
      • selectGridFtp

        public GridFTPServer selectGridFtp​(java.util.ArrayList ftp)
        Return a random gridftp url from the list of gridftp url's.
        Parameters:
        ftp - Takes an ArrayList of GridFTPServer Objects.
        Returns:
        String Returns a single gridftp url from among many
        See Also:
        org.griphyn.cPlanner.classes.GridFTPServer
      • getVDS_HOME

        public java.lang.String getVDS_HOME​(java.lang.String siteID)
        Returns the value of VDS_HOME for a site.
        Parameters:
        siteID - the name of the site.
        Returns:
        value if set else null.
      • getPegasusHome

        public java.lang.String getPegasusHome​(java.lang.String siteID)
        Returns the value of PEGASUS_HOME for a site.
        Parameters:
        siteID - the name of the site.
        Returns:
        value if set else null.
      • getEnvironmentVariable

        public java.lang.String getEnvironmentVariable​(java.lang.String siteID,
                                                       java.lang.String envVariable)
        Returns an environment variable for a particular site set in the Site Catalog.
        Parameters:
        siteID - the name of the site.
        envVariable - the environment variable whose value is required.
        Returns:
        value of the environment variable if found, else null
      • getPoolProfile

        public java.util.List getPoolProfile​(java.lang.String siteID,
                                             java.lang.String namespace)
        It returns profile information associated with a particular namespace and pool.
        Parameters:
        siteID - the name of the site, whose profile information you want.
        namespace - the namespace correspoinding to which the profile information of a particular site is desired.
        Returns:
        List of Profile objects NULL when the information about the site is not there or no profile information associated with the site.
        See Also:
        org.griphyn.cPlanner.classes.Profile
      • getExecPoolWorkDir

        public java.lang.String getExecPoolWorkDir​(java.lang.String executionPool)
        This determines the working directory on remote execution pool on the basis of whether an absolute path is specified in the pegasus.dir.exec directory or a relative path.
        Parameters:
        executionPool - the pool where a job has to be executed.
        Returns:
        the path to the pool work dir.
        Throws:
        java.lang.RuntimeException - in case of site not found in the site catalog.
      • getExecPoolWorkDir

        public java.lang.String getExecPoolWorkDir​(Job job)
        This determines the working directory on remote execution pool for a particular job. The job should have it's execution pool set.
        Parameters:
        job - Job object for the job.
        Returns:
        the path to the pool work dir.
        Throws:
        java.lang.RuntimeException - in case of site not found in the site catalog.
      • getExecPoolWorkDir

        public java.lang.String getExecPoolWorkDir​(java.lang.String siteID,
                                                   java.lang.String path)
        This determines the working directory on remote execution pool on the basis of whether an absolute path is specified in the pegasus.dir.exec directory or a relative path.
        Parameters:
        siteID - the name of the site where a job has to be executed.
        path - the relative path that needs to be appended to the workdir from the execution pool.
        Returns:
        the path to the pool work dir.
        Throws:
        java.lang.RuntimeException - in case of site not found in the site catalog.
      • getExecPoolWorkDir

        public java.lang.String getExecPoolWorkDir​(java.lang.String siteID,
                                                   java.lang.String path,
                                                   int jobClass)
        This determines the working directory on remote execution pool on the basis of whether an absolute path is specified in the pegasus.dir.exec directory or a relative path. If the job class happens to be a create directory job it does not append the name of the random directory since the job is trying to create that random directory.
        Parameters:
        siteID - the name of the site where the job has to be executed.
        path - the relative path that needs to be appended to the workdir from the execution pool.
        jobClass - the class of the job.
        Returns:
        the path to the pool work dir.
        Throws:
        java.lang.RuntimeException - in case of site not found in the site catalog.
      • getURLPrefix

        public java.lang.String getURLPrefix​(java.lang.String poolName)
        Returns the url prefix of a gridftp server on the pool. gsiftp://dataserver.phys.uwm.edu/~/griphyn_test/ligodemo_output/ gives a URL prefix of gsiftp://dataserver.phys.uwm.edu
        Parameters:
        poolName - the name of the pool.
        Returns:
        String corresponding to the url prefix if the pool is found. null if pool entry is not found.
      • getSeMountPoint

        public java.lang.String getSeMountPoint​(SiteInfo site)
        Return the storage mount point for a particular pool.
        Parameters:
        site - SiteInfo object of the site for which you want the storage-mount-point.
        Returns:
        String corresponding to the mount point if the pool is found. null if pool entry is not found.
      • getTXPoolEntry

        public SiteInfo getTXPoolEntry​(java.lang.String poolName)
        Gets the pool object to be used for the transfer universe. If we do not get that then defaults back to globus universe for the same pool.
        Parameters:
        poolName - the name of the pool
        Returns:
        Pool
      • logMessage

        protected void logMessage​(java.lang.String msg)
        Logs the message to a logging stream. Currently does not log to any stream.
        Parameters:
        msg - the message to be logged.