类 Tokenizer
java.lang.Object
weka.core.tokenizers.Tokenizer
- 所有已实现的接口:
Serializable
,Enumeration
,OptionHandler
,RevisionHandler
public abstract class Tokenizer
extends Object
implements Enumeration, OptionHandler, Serializable, RevisionHandler
A superclass for all tokenizer algorithms.
- 版本:
- $Revision: 1.3 $
- 作者:
- FracPete (fracpete at waikato dot ac dot nz)
- 另请参阅:
-
构造器概要
构造器 -
方法概要
修饰符和类型方法说明String[]
Gets the current option settings for the OptionHandler.abstract String
Returns a string describing the stemmerabstract boolean
Tests if this enumeration contains more elements.Returns an enumeration of all the available options..abstract Object
Returns the next element of this enumeration if this enumeration object has at least one more element to provide.static void
runTokenizer
(Tokenizer tokenizer, String[] options) initializes the given tokenizer with the given options and runs the tokenizer over all the remaining strings in the options array.void
setOptions
(String[] options) Sets the OptionHandler's options using the given list.abstract void
Sets the string to tokenize.static String[]
initializes the given tokenizer with the given options and runs the tokenizer over all the remaining strings in the options array.从接口继承的方法 java.util.Enumeration
asIterator
从接口继承的方法 weka.core.RevisionHandler
getRevision
-
构造器详细资料
-
Tokenizer
public Tokenizer()
-
-
方法详细资料
-
globalInfo
Returns a string describing the stemmer- 返回:
- a description suitable for displaying in the explorer/experimenter gui
-
listOptions
Returns an enumeration of all the available options..- 指定者:
listOptions
在接口中OptionHandler
- 返回:
- an enumeration of all available options.
-
getOptions
Gets the current option settings for the OptionHandler.- 指定者:
getOptions
在接口中OptionHandler
- 返回:
- the list of current option settings as an array of strings
-
setOptions
Sets the OptionHandler's options using the given list. All options will be set (or reset) during this call (i.e. incremental setting of options is not possible).- 指定者:
setOptions
在接口中OptionHandler
- 参数:
options
- the list of options as an array of strings- 抛出:
Exception
- if an option is not supported
-
hasMoreElements
public abstract boolean hasMoreElements()Tests if this enumeration contains more elements.- 指定者:
hasMoreElements
在接口中Enumeration
- 返回:
- true if and only if this enumeration object contains at least one more element to provide; false otherwise.
-
nextElement
Returns the next element of this enumeration if this enumeration object has at least one more element to provide.- 指定者:
nextElement
在接口中Enumeration
- 返回:
- the next element of this enumeration.
-
tokenize
Sets the string to tokenize. Tokenization happens immediately.- 参数:
s
- the string to tokenize
-
tokenize
initializes the given tokenizer with the given options and runs the tokenizer over all the remaining strings in the options array. If no strings remained in the option string then data is read from stdin, line by line.- 参数:
tokenizer
- the tokenizer to useoptions
- the options for the tokenizer- 返回:
- the tokenized strings
- 抛出:
Exception
- if setting of options or tokenization fails
-
runTokenizer
initializes the given tokenizer with the given options and runs the tokenizer over all the remaining strings in the options array. The generated tokens are then printed to stdout. If no strings remained in the option string then data is read from stdin, line by line.- 参数:
tokenizer
- the tokenizer to useoptions
- the options for the tokenizer
-