AbstractTokenizer

java.lang.Object

org.luwrain.nlp.ru.AbstractTokenizer

Direct Known Subclasses:: ReaderTokenizer

public abstract class AbstractTokenizer extends Object

Field Summary

Fields

Modifier and Type

Field

Description

protected final List<Token>

output
Constructor Summary

Constructors

Constructor

Description

AbstractTokenizer()
Method Summary

Modifier and Type

Method

Description

abstract void

backCh(char ch)

Token[]

getOutput()

abstract boolean

hasCh()

void

tokenize()

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details
- output
  
  protected final List<Token> output
Constructor Details
- AbstractTokenizer
  
  public AbstractTokenizer()
Method Details
- hasCh
  
  public abstract boolean hasCh()
- backCh
  
  public abstract void backCh(char ch)
- tokenize
  
  public void tokenize()
- getOutput
  
  public Token[] getOutput()