SPALN information

Overview

References

[1] Gotoh, O. " A space-efficient and accurate method for mapping and aligning cDNA sequences onto genomic sequence" Nucleic Acids Research 36 (8) 2630-2638 (2008).
[2] Gotoh, O. " Direct mapping and alignment of protein sequences onto genomic sequence" Bioinformatics 24 (21) 2438-2444 (2008).
[3] Iwata, H. and Gotoh, O. " Benchmarking spliced alignment programs including Spaln2, an extended version of Spaln that incorporates additional species-specific features" Nucleic Acids Research 40 (20) e161 (2012)
[4] Gotoh, O. " Homology-based gene structure prediction: simplified matching algorithm using a translated codon (tron) and improved accuracy by allowing for long gaps" Bioinformatics 16 (3) 190-202 (2000)
[5] Nagasaki, H., Arita, M., Nishizawa, T., Suwa, M., Gotoh, O. " Automated classification of alternative splicing and transcriptional initiation and construction of a visual database of the classified patterns" Bioinformatics 22 (10) 1211-1216 (2006)
[6] Gotoh, O. Cooperation of Spaln and Prrn5 for construction of gene-structure-aware multiple sequence alignment. Methods in Molecular Biology, in press.

Present Version: 2.4.1, Last update: 2020-10-09

Install

From source

• bin : binaries
• doc : documents
• seqdb : sample sequences. In this directory you should format genomic or database files
• src : source codes
• table : parameter files used by spaln

To modify the location of executables and/or other settings, run 'configure --help' at step 6 below. (Warning: Full path name rather than relative path name must be given for executables or other directories as the arguments of the configure command.) These locations are hard coded in spaln. The locations of the 'seqdb' and 'table' directories will be respectively denoted by seqdb and table below. Hence, seqdb=download/spalnXX/seqdb, and table=download/spalnXX/table in the default settings.

1. % mkdir download
2. % cd download
3. Download spalnXX.tar.gz
4. % tar xfz spalnXX.tar.gz
5. % cd ./spalnXX/src
6. % ./configure [--help]
   • Please manually edit Makefile if $(CC) does not indicate a C++ compiler or
   • % CXX=g++ ./configure [other options]
   • To make spaln and sorgrcd handle gzipped files, ./configure command should be run with --use_zlib=1. Alternatively, you may manually edit the generated Makefile so that -DUSE_ZLIB=1 is included in the compile option.
7. % make
8. % make install
   • Executables are copied to ../bin
   • makmdm program makes mutation data matrices of various PAM levels in the ../table directory
9. % make clearall
10. Add download/spalnXX/bin to your PATH
    % setenv PATH $PATH:download/spalnXX/bin (csh/tsh)
    $ export PATH=$PATH:download/spalnXX/bin (sh/bsh)
    
    Preferably, you may add the above line in your start up rc file (e.g. ~/.bashrc)
    Alternatively, move or copy download/spalnXX/bin/* to a directory on your PATH, if you have not specified the location of executables at step 6 above.
11. If you have changed the location of table and/or seqdb directory after installation, set the env variables ALN_TAB and/or ALN_DBS as explained in the following subsection.
12. Proceed to Sequence data formation.

From binaries

Case I: Assume the directory work is your workspace where every material is stored. In this case, seqdb=work.

1. % mkdir work
2. % cd work
3. Download spalnXX.PC.tar.gz, where PC is a platform code
4. % tar xfz spalnXX.PC.tar.gz
5. Add work/bin to your PATH
   Or move or copy work/bin/* to a directory on your PATH
6. % mv ./table/* .; rmdir ./table
7. % mv ./seqdb/* .; rmdir ./seqdb
8. Now proceed to Sequence data formation.

Case II: Assume your workspace work is distinct from seqdb

1. % mkdir download
2. % cd download
3. Download spalnXX.PC.tar.gz, where PC is a platform code
4. % tar xfz spalnXX.PC.tar.gz
5. Add download/bin to your PATH
   Or move or copy download/bin/* to a directory on your PATH
6. % setenv ALN_TAB download/table (csh/tsh)
   $ export ALN_TAB=download/table (sh/bsh)
7. % setenv ALN_DBS download/seqdb (csh/tsh)
   $ export ALN_DBS=download/seqdb (sh/bsh)
8. Add the above lines to your rc file, so that you don't have to repeat the commands at every login time.
9. Now proceed to Sequence data formation

Sequence data formation

1. % cd seqdb
2. Download or copy genomic sequences or protein database sequence in multi-fasta format. If spaln is accordingly compiled, gzipped file need not be uncompressed (the file name must be X.gz).
3. To use 'makeidx.pl' command, chromosomal sequences must be concatenated into a single file. The extension of the genomic sequence file must be '.mfa' or '.gf', and protein database sequence must be '.faa', to render 'make' command effective. With 'spaln -W' command, these restrictions are not obligatory. Hereafter, the file name is assumed to be xxxgnm.gf or prosdb.faa. The total number of residues in a file must not be greater than or equal to 2**32.
4. To format xxxgnm.gf(.gz), run either of the following two commands, which are equivalent to each other except that the former is faster, accepts multiple input files, and does not need Makefile.
   % spaln -W -K[D|P] [-XGMAX_GENE] [other spaln options] xxxgnm.gf(.gz) ...
   % ./makeidx.pl -i[n|p|np] [-XGMAX_GENE] [other spaln options] xxxgnm.gf(.gz)
   
   To format protein database sequence, use either of the following two commands:
   % spaln -W -KA [other spaln options] prosdb.faa(.gz) ...
   % ./makeidx.pl -ia [other spaln options] prosdb.faa(.gz)
   
   • -KX (or corresponding -ix) option specifies the "block file" xxxgnm.bkx to be constructed, where X is 'A', 'D' or 'P' and x is 'a', 'n' or 'p'. The -inp option will construct both xxxgnm.bkn (for cDNA queries) and xxxgnm.bkp (for protein queries) together with the xxxgnm.idx and associated files. -KX option is mandatory. If -ix is omitted or x is empty, xxxgnm.idx and associated files are created but no block file is constructed.
   • The block size and k-mer size are estimated from the genome size, unless explicitly specified (see below).
   • if MAX_GENE (the length of the plausibly longest gene on the genome) is not specified, MAX_GENE is also estimated from the genome size.
   • WARNING: Don't forget to specify MAX_GENE if xxxgnm.gf represents only a part of the genome (e.g a single chromosome)!!. Otherwise, MAX_GENE may be seriously underestimated. MAX_GENE can have suffix 'k' and 'M' to indicate that the number is measured in kbp and Mbp, respectively.
   • -g option generates gzipped outputs.
   • -tN option enables N-thread operation.
   • -E option generates local lookup table.
   As Spaln -W command accepts multiple input files and generates all necessary files in a single operation, you can skip following instructions.
   
   Makeidx.pl command performs the following series of operations 5-6, if the input is a single sequence file.
5. % make xxxgnm.idx (for genomic sequence) or
   % make prosdb.idx (for protein database sequence)
   • This command converts the sequence into a binary format. Four or five files, xxxgnm.seq, xxxgnm.idx, xxxgnm.ent, xxxgnm.grp, and optionally xxxgnm.odr are constructed (prosdb instead of xxxgnm in case of make prosdb.idx). It may take several tens of minutes to construct the files for mammalian genome.
6. % make xxxgnm.bkn (for cDNA queries) or
   % make xxxgnm.bkp (for protein queries) or
   % make prosdb.bka (for protein database)
   
   • This command makes the block index table. This process may take another several tens of minutes.
   • Internally, the make command invokes
     'spaln -Wxxxgnm.bkn -KD [Options] xxxgnm.gf' or
     'spaln -Wxxxgnm.bkp -KP [Options] xxxgnm.gf' or
     'spaln -Wprosdb.bka -KA [Options] prosdb.faa'.
   • If xxxgnm.grp or prosdb.grp were successfully constructed at step 5 above, the option values below would be automatically calculated by script makblk.pl.
   • Options: (default value)
     • -XAN: Alphabet size of the reduced amino acids: 6 < N <= 20 (20)
     • -XBS: Bit patterns of the spaced seeds. The pattern should be asymmetric when the number of patterns > 2.
     • -XCN: Number of seed patterns: 0 <= N <= 5 (0: contiguous seed)
     • -XGN: Maximum gene length (262144)
     • -XaN: A parameter used to filter excessively abundant words (10)
     • -XbN: Block size (4096) N < 65536 and 'genome size' < 65536 * N must be satisfied. An estimate of N is sqrt(genome size). For mammals, N is nearly equal to 54000.
     • -XgN: Maximal distance in block number between 5' terminal and 3' terminal blocks (16)
     • -XkN: Word size (11 for DNA, 5 for protein)
     • -XsN: Distance between neighboring seeds (= k)
       
7. It is possible to generate xxxgnm.idx and other three files directly from the input files without concatenation:
   
   % makdbs -nxxxgnm -KD file1 ... fileN
   % make xxxgnm.bkn (for cDNA queries) or
   % make xxxgnm.bkp (for protein queries)
   
   This method is particularly useful when the concatenation might yield a file too large to be dealt with by the OS.

Execution

1. Prepare protein, cDNA, or genomic segment sequence(s) in (multi-)fasta or extended (multi-)fasta (see -O6 option) format (denoted by query below). From 2.3.2a, gzipped fasta file(s) may be used as the query without prior expansion. Note, however, that compressed query can considerably slow down the execution rate.
2. Store query to work
3. % cd work
4. Run spaln in one of the following four modes. Spaln does not support comparison between two genomic segments.
   (A) % spaln -Q[0|1|2|3] [-ON] [other options] genome_segment query
   (B) % spaln -Q[4|5|6|7] [-ON] [other options] -[d|D] xxxgnm query
   (C) % spaln -Q[4|5|6|7] [-ON] [other options] -[a|A] prosdb query
   (D) % spaln -Q[4|5|6|7] [-ON] [other options] prosdb.faa query
   In the last case, prosdb.faa will be internally formatted, and the formatted results will be discarded after the end of execution.
   

   Only a subset of queries may be examined if query is replaced with 'query (from to)', where 'from' and 'to' are the first and last entry numbers in query to be examined.
   To run spaln on multiple CPUs, for example, the following commands may be used and the results may be summarized with sortgrcd, as explained later.
   (a) % spaln -Q7 -O12 -oxxxO1 -dxxxgnm 'query (1 1000)'
   (b) % spaln -Q7 -O12 -oxxxO2 -dxxxgnm 'query (1001 2000)'
   (c) % spaln -Q7 ...
   However, the procedure will be simplified if a multi-thread operation is used as follows:
   (d) % spaln -Q7 -ON -oxxx -dxxxgnm -t[N] query

   Options: (default value)
   

   • -CN: Use the genetic code specified by the "transl_table number" defined in NCBI transl_table (1).
   • -E: Use local lookup table.
   • -HN: Output is suppressed if the alignment score is less than N. See also -pw. (35)
   • -LS: Smith-Waterman-type local alignment. This option may prune out weakly matched terminal regions.
   • -M[N[.M]: Single or multiple output for each query
     • No option (default): single locus
     • No argument: Multiple loci up to the maximum number specified by the program (4 in the present implementation)
     • N=1: Re-search the query region not aligned in the first trial. May be useful to detect chimera or fragmented genomic region
     • N>1: Output multiple loci maximally up to N
     • M: Maximal number of candidate loci to be subjected to spliced alignment (4). IfM < N, M is reset to N.
   • -ON[,N₂,N₃...]: Select output format for genome vs cDNA or aa (4)
     It is possible to output multiple files with extensions of .ON at a run by multiply applying this option, or by concatenating the format numbers with commas (,) or colons (:), ex. -O0,1,4. See also -o option.
     • N=0: Gff3 gene format
     • N=1: Alignment
     • N=2: Gff3 match format
     • N=3: Bed format
     • N=4: Exon-oriented format similar to the output of megablast -D 3
     • N=5: Intron-oriented output
     • N=6: Concatenated exon sequence in extended (multi-)fasta format, in which the exon-intron structure of the parental gene is supplied by one or more comment lines starting with ';C', such as
       ;C complement(join(1232555..1232760,1233786..1233849,1233947..1234119,
       ;C 1234206..1234392))
       
     • N=7: Translated amino-acid sequence in extended (multi-)fasta format. Presently not very useful for cDNA queries because the entire exon rather than an ORF is translated
     • N=8: Mapping (block) information only. Use with -Q4
     • N=10: SAM format
     • N=12: Output the same information as -O4 in the binary format. If -oOutput is set, three files named Output.grd, Output.erd, and Output.qrd will be created. Otherwise, query.grd, query.erd, and query.qrd will be created.
   • -O N[,N₂,N₃...]: Select output format for aa vs aa (4)
     It is possible to output multiple files with extensions of .ON at a run by multiply applying this option, or by concatenating the format numbers with commas (,) or colons (:), ex. -O0,1,4. See also -o option.
     • N=0: statistics (%divergence alignment_score #match #mismatch #gap #unpaired)
     • N=1: Alignment
     • N=2: Sugar format
     • N=3: Psl format
     • N=4: XYL = Coordinate + match length
     • N=5: statistics + XYL
     • N=8: Cigar format
     • N=9: Vulgar format
     • N=10: SAM format
   • -QN: Select algorithm (3)
     • 0<=N<=3: Genomic segment in the fasta format given by the first argument vs. query given by the following arguments. See also -i option. One may skip the formatting step described above if only this mode of operation is used.
     • 4<=N<=7: Genome mapping and alignment. The genomic sequence must be formatted beforehand.
     • N=0,4: DP procedure without HSP search. Considerably slow
     • N=1,2,3,5,6,7: Recursive HSP search up to the level of (N % 4)
   • -RS: Read block index table from file S. If omitted, the xxxgnm.bkn, xxxgnm.bkp, or prosdb.bka file will be read depending on the type of query. The appropriate file is searched for in the current directory, the directory specified by the env variable 'ALN_DBS', and the 'seqdb' directory specified at the compile time in this order.
   • -SN: Orientation of the DNA query sequence (3)
     • N=0: The orientation is inferred from the phrases (e.g. 5' end) in the header line of each entry within a fasta file. If no information is available, both orientations are examined, and the result with the better score is reported. Terminal polyA or polyT sequence is not trimmed.
     • N=1: Forward orientation only. PolyA tail may be trimmed off.
     • N=2: Reverse-complement orientation only. Leading polyT may be trimmed off.
     • N=3: Examine both orientations. Terminal polyA or polyT sequence may be trimmed off.
   • -TS: Specify the species-specific parameter set. S corresponds to the subdirectory in the table directory. Alternatively, S may be the 1st or the 3rd term in table/gnm2tab file, where the 2nd term on the line indicates the subdirectory.
   • -VN: Minimum space to induce Hirschberg's algorithm (16M)
   • -WS: Write block index table to file S.bkx. if S is omitted, the file name (without directory and extension) of the first argument is used as S.
   • -g: gzipped output used in combination with -W or -O12 option.
   • -i[a|p]: Input mode with -Q[0..3].
     • -ia: Alternative mode; a genomic segment of an odd numbered entry in the input file is aligned with the query of the following entry.
     • -ip: Parallel mode; the i-th entry in the file specified by the first argument is aligned with the i-th entry in the file specified by the second argument.
     • default: The genomic segment specified by the first argument is aligned with each entry in the file specified by the following arguments.
   • -oS: Destination of output file name (stdout). If multiple output formats are specified by -O option(s), S specifies the directory or prefix to which the file names with .ON extensions are concatenated.
   • -paN: Terminal polyA or polyT sequence longer than N (12) is trimmed off and the orientation is fixed accordingly. If N = 0 or empty, these functionalities are disabled.
   • -pi: Mark exon-intron junctions by color in the alignment (-O1).
   • -pq: Suppress warning messages sent to stderr.
   • -pw: Report result even if alignment score is below threshold value.
   • -px: Suppress self-comparisons in the execution mode (C) or (D).
   • -xBS: Bit pattern of seeds used for HSP search at level 1
   • -xbS: Bit pattern of seeds used for HSP search at level 3
   • -uN: Gap-extension penalty (3, 2, 2)
   • -vN: Gap-opening penalty (8, 6, 9)
   • -yaN: Dinucleotide pairs at the ends of an intron (0)
     • N=0: Accept only the canonical pairs (GT..AG,GC..AG,AT..AC)
     • N=1: accept also AT..AN
     • N=2: allow up to one mismatch from GT..AG
     • N=3: accept any pairs. An omission of N implies N = 3
   • -yiN: Intron penalty (11, 8, 11)
   • -yjN: Incline of long gap penalty (0.6)
   • -ykN: Flex point where the incline of gap penalty changes (7)
   • -ylN: Double affine gap penalty if N=3; otherwise affine gap penalty
   • -ymN: Score for a nucleotide match (2, 2)
   • -ynN: Penalty for nucleotide mismatch (6, 2)
   • -yoN: Penalty for an in-frame termination codon (100)
   • -ypN: PAM level used in the alignment (third) phase (150)
   • -yqN: PAM level used in the second phase (50)
   • -yxN: Penalty for a frame shift (100)
   • -yyN: Relative contribution of splicing signal (8)
   • -yzN: Relative contribution of coding potential (2)
   • -yAN: Relative contribution of the translational initiation or termination signal (8)
   • -yBN: Relative contribution of branch point signal (0)
   • -yEN: Minimum exon length (2)
   • -yIS: Intron distribution parameters
   • -yJN: Relative contribution of the bonus given to a conserved intron position
   • -yLN: Minimum intron length (30, 30)
   • -ySN: N specifies the percentile contribution of the species-specific splice signal. The other part is derived from the universal signal given to the dinucleotides at the ends of an intron. An omission of N implies N = 100.
   • -yXN: N = 0: set parameter values for intra-species comparison. N = 1: set parameter values for cross-species comparison. The default value for N is 0 or 1 for DNA or protein query, respectively.
   • -yYN: Relative contribution of length-dependent part of intron penalty (8)
   • -yZN: Relative contribution of oligomer composition within an intron (0)
5. Sortgrcd

Sortgrcd is used to recover the output of spaln with -O12 option, to apply some filtering, and also to rearrange the output of multiple spaln runs. It is invoked by:
% sortgrcd [options] X.grd [Y.grd ...] or
% sortgrcd [options] X.grd.gz [Y.grd.gz ...]


• Options:
  • -CN: Minimum cover rate = % nucleotides in predicted exons / length of query (x 3 if query is protein) (0-100)
  • -EN: Report only the best (N=1) or all (N=2) results per gene locus (1)
  • -FN: Filter level (N=0: no; N=1: mild; N=2: medium; N=3: stringent)
  • -IN: Minimum sequence identity (0-100)
  • -HN: Minimum alignment score (35)
  • -ON: Output mode. N=0 or 3-7: same as -ON of spaln; N=15: -O5 format for unique introns (default: N=4)
  • -SC: Sort chromosomes/contigs in the order of C=a: alphabetical, b: abundance, c: appearance in genome database, r: reverse order for minus strand
  • -VN: Internal memory size used for core sort. If the data size is greater than N, the sorting procedure will be done in pieces.
  • -mN: Maximum number of mismatches within 20 bp from the nearest exon-intron boundary
  • -nN: Maximum number of non-canonical (other than GT..AG, GC..AG, AT..AC) intron ends
  • -uN: Maximum number of unpaired (gap) sites within 20 bp from the nearest exon-intron boundary
• By default, no filter listed above is applied.
• When the output of spaln is separated into several files, the combined results are subjected to the sorting. Although *.grd files are assigned as the argument, there must be corresponding *.erd and *.qrd files in the same directory.
• In the default output format, the gene structure corresponding to each transcript is delimited by a line starting with '@', whereas each gene locus is delimited by a line starting with '!' [4]. Two transcripts belong to the same locus if their corresponding genomic regions overlap by at least one nucleotide on the same strand.
• With -O0 option, the outputs follow the instruction of Gff3 (http://www.sequenceontology.org/gff3.shtml) where a gene locus is defined as described above.

Example

• To experience the flow of procedures with the samples in seqdb, type in the following series of commands after moving to seqdb.
  % make dictdisc.cf
  % make dictdisc.faa
  % make dictdisc_g.gf
  % ./makeidx.pl -inp dictdisc_g.gf
  % make dictdisc.srd
  % make dictdisc.spn
  

Change from previous versions

1. The algorithm for delimiting a genic region has been modified to find remote terminal coding exon(s) separated by long (up to 99.6% quantile) intron(s) from the main body of the gene.
2. The -yx0 option now tries to search for missing internal micro exons and terminal very short coding exons.
3. Selenocysteine (denoted by U) is now regarded as the 21th amino acid which favorably matches an in-frame TGA termination codon (U in the Tron code) upon DNA vs amino acid sequence alignment.
4. Gene candidates are now sorted according to the final alignment score rather than the intermediate chained HSP score. This modification has improved the chance of true orthologous hits rather than paralog hits at an expense of a slight increase in computational load.
5. Compared with the previous versions, a larger number of species-specific parameter sets (247 <- 102) are provided to support more species (1495 <- 688). Note that some parameter-set identifiers are changed. Please use eight-digit species identifies (e.g. zea_mays) rather than former parameter-set identifiers (e.g. Magnolio) as the argument of -T option.

Added/modified in Ver. 2.4.0 (2019-11-18):

1. Spaln can now directly format genomic sequences without relying on 'make' command. See Sequence data formation.
2. The internal format of index files is slightly modified. Although previously-formatted files can be used by the new version, the opposite is not true. Note that use of older files with the new version can lead to a slight loss in sensitivity.
3. The above change has been done to facilitate multi-thread operation at the format time.
4. Multiple output forms can be produced at a single run. See -O and -o options.
5. The traditional bidirectional Hirschberg algorithm is changed to the unidirectional variant.
6. Also, the bidirectional 'sandwich' or 'attack by both sides' spliced alignment algorithm has been changed to unidirectional 'skipped' spliced alignment algorithm. This and the preceding changes have considerably reduced code complexity.
7. Local lookup table (xxxgnm.lun or xxxgnm.lup) is generated and used with -E option. Be cautious to use this option, as a large disk space is required to store the generated file, and a large memory is required at the runtime.
8. Many small bugs have been fixed.

Copyright (c) 2007-2020 Osamu Gotoh all rights reserved