net.java.sen.tokenizers.ja
クラス JapaneseTokenizer

java.lang.Object
  上位を拡張 net.java.sen.dictionary.Tokenizer
      上位を拡張 net.java.sen.tokenizers.ja.JapaneseTokenizer

public class JapaneseTokenizer
extends Tokenizer

A Tokenizer for Japanese text


フィールドの概要
 
クラス net.java.sen.dictionary.Tokenizer から継承されたフィールド
bosNode, dictionary, eosNode, unknownCToken, unknownPartOfSpeechDescription
 
コンストラクタの概要
JapaneseTokenizer(Dictionary dictionary, java.lang.String unknownPartOfSpeechDescription)
          Creates a JapaneseTokenizer with the given Dictionary
 
メソッドの概要
 Node lookup(SentenceIterator iterator, char[] surface)
          Searches for possible morphemes from the given SentenceIterator.
 
クラス net.java.sen.dictionary.Tokenizer から継承されたメソッド
getBOSNode, getDictionary, getEOSNode, getUnknownNode
 
クラス java.lang.Object から継承されたメソッド
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

コンストラクタの詳細

JapaneseTokenizer

public JapaneseTokenizer(Dictionary dictionary,
                         java.lang.String unknownPartOfSpeechDescription)
Creates a JapaneseTokenizer with the given Dictionary

パラメータ:
dictionary - The Dictionary in which to search for possible morphemes
unknownPartOfSpeechDescription - The part-of-speech code to use for unknown tokens
メソッドの詳細

lookup

public Node lookup(SentenceIterator iterator,
                   char[] surface)
クラス Tokenizer の記述:
Searches for possible morphemes from the given SentenceIterator. The Node that is returned links through Node.rnext to a list of matches which may be of varying lengths

定義:
クラス Tokenizer 内の lookup
パラメータ:
iterator - The iterator to search from
surface - The underlying character surface
戻り値:
The head of a chain of Nodes representing the possible morphemes beginning at the given index


Copyright ? 2008. All Rights Reserved.