ThaiTokenizer (Lucene 10.5.0 common API)
Description: declaration: package: org.apache.lucene.analysis.th, class: ThaiTokenizer

Package org.apache.lucene.analysis.th

# Class ThaiTokenizer

java.lang.Object
org.apache.lucene.util.AttributeSource
org.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.Tokenizer
org.apache.lucene.analysis.util.SegmentingTokenizerBase
org.apache.lucene.analysis.th.ThaiTokenizer

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`

public class ThaiTokenizer extends SegmentingTokenizerBase
Tokenizer that use `BreakIterator` to tokenize Thai text.

WARNING: this tokenizer may not be supported by all JREs. It is known to work with Sun/Oracle and Harmony JREs. If your application needs to be fully portable, consider using ICUTokenizer instead, which uses an ICU Thai BreakIterator that will always be available.

- ## Nested Class Summary

  ## Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

  `AttributeSource.State`

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `static final boolean`
  `DBBI_AVAILABLE`
  True if the JRE supports a working dictionary-based breakiterator for Thai.

  ### Fields inherited from class org.apache.lucene.analysis.util.SegmentingTokenizerBase

  `buffer, BUFFERMAX, offset`

  ### Fields inherited from class org.apache.lucene.analysis.Tokenizer

  `input`

  ### Fields inherited from class org.apache.lucene.analysis.TokenStream

  `DEFAULT_TOKEN_ATTRIBUTE_FACTORY`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `ThaiTokenizer()`
  Creates a new ThaiTokenizer
  `ThaiTokenizer(AttributeFactory factory)`
  Creates a new ThaiTokenizer, supplying the AttributeFactory

- ## Method Summary

  Modifier and Type
  Method
  Description
  `protected boolean`
  `incrementWord()`
  Returns true if another word is available
  `protected void`
  `setNextSentence(int sentenceStart, int sentenceEnd)`
  Provides the next input sentence for analysis

  ### Methods inherited from class org.apache.lucene.analysis.util.SegmentingTokenizerBase

  `end, incrementToken, isSafeEnd, reset`

  ### Methods inherited from class org.apache.lucene.analysis.Tokenizer

  `close, correctOffset, setReader, setReaderTestPoint`

  ### Methods inherited from class org.apache.lucene.util.AttributeSource

  `addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString`

  ### Methods inherited from class java.lang.Object

  `clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

- ## Field Details

  - ### DBBI_AVAILABLE

    public static final boolean DBBI_AVAILABLE
    True if the JRE supports a working dictionary-based breakiterator for Thai. If this is false, this tokenizer will not work at all!

- ## Constructor Details

  - ### ThaiTokenizer

    public ThaiTokenizer()
    Creates a new ThaiTokenizer

  - ### ThaiTokenizer

    public ThaiTokenizer(AttributeFactory factory)
    Creates a new ThaiTokenizer, supplying the AttributeFactory

- ## Method Details

  - ### setNextSentence

    protected void setNextSentence(int sentenceStart, int sentenceEnd)
    Description copied from class: `SegmentingTokenizerBase`
    Provides the next input sentence for analysis
    Specified by:
      `setNextSentence` in class `SegmentingTokenizerBase`

  - ### incrementWord

    protected boolean incrementWord()
    Description copied from class: `SegmentingTokenizerBase`
    Returns true if another word is available
    Specified by:
      `incrementWord` in class `SegmentingTokenizerBase`