DictionaryCompoundWordTokenFilter (Lucene 10.5.0 common API)
Description: declaration: package: org.apache.lucene.analysis.compound, class: DictionaryCompoundWordTokenFilter

Package org.apache.lucene.analysis.compound

# Class DictionaryCompoundWordTokenFilter

java.lang.Object
org.apache.lucene.util.AttributeSource
org.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.TokenFilter
org.apache.lucene.analysis.compound.CompoundWordTokenFilterBase
org.apache.lucene.analysis.compound.DictionaryCompoundWordTokenFilter

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`, `Unwrappable<TokenStream>`

public class DictionaryCompoundWordTokenFilter extends CompoundWordTokenFilterBase
A `TokenFilter` that decomposes compound words found in many Germanic languages.

"Donaudampfschiff" becomes Donau, dampf, schiff so that you can find "Donaudampfschiff" even when you only enter "schiff". It uses a brute-force algorithm to achieve this.

- ## Nested Class Summary

  ## Nested classes/interfaces inherited from class org.apache.lucene.analysis.compound.CompoundWordTokenFilterBase

  `CompoundWordTokenFilterBase.CompoundToken`

  ## Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

  `AttributeSource.State`

- ## Field Summary

  ### Fields inherited from class org.apache.lucene.analysis.compound.CompoundWordTokenFilterBase

  `DEFAULT_MAX_SUBWORD_SIZE, DEFAULT_MIN_SUBWORD_SIZE, DEFAULT_MIN_WORD_SIZE, dictionary, maxSubwordSize, minSubwordSize, minWordSize, offsetAtt, onlyLongestMatch, termAtt, tokens`

  ### Fields inherited from class org.apache.lucene.analysis.TokenFilter

  `input`

  ### Fields inherited from class org.apache.lucene.analysis.TokenStream

  `DEFAULT_TOKEN_ATTRIBUTE_FACTORY`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `DictionaryCompoundWordTokenFilter(TokenStream input, CharArraySet dictionary)`
  Creates a new `DictionaryCompoundWordTokenFilter`
  `DictionaryCompoundWordTokenFilter(TokenStream input, CharArraySet dictionary, int minWordSize, int minSubwordSize, int maxSubwordSize, boolean onlyLongestMatchIgnoreSubwords)`
  Creates a new `DictionaryCompoundWordTokenFilter`
  `DictionaryCompoundWordTokenFilter(TokenStream input, CharArraySet dictionary, int minWordSize, int minSubwordSize, int maxSubwordSize, boolean onlyLongestMatch, boolean onlyLongestMatchIgnoreSubwords)`
  Deprecated.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `protected void`
  `decompose()`
  Decomposes the current `CompoundWordTokenFilterBase.termAtt` and places `CompoundWordTokenFilterBase.CompoundToken` instances in the `CompoundWordTokenFilterBase.tokens` list.

  ### Methods inherited from class org.apache.lucene.analysis.compound.CompoundWordTokenFilterBase

  `incrementToken, reset`

  ### Methods inherited from class org.apache.lucene.analysis.TokenFilter

  `close, end, unwrap`

  ### Methods inherited from class org.apache.lucene.util.AttributeSource

  `addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString`

  ### Methods inherited from class java.lang.Object

  `clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

- ## Constructor Details

  - ### DictionaryCompoundWordTokenFilter

    public DictionaryCompoundWordTokenFilter(TokenStream input, CharArraySet dictionary)
    Creates a new `DictionaryCompoundWordTokenFilter`
    Parameters:
      `input` - the `TokenStream` to process
      `dictionary` - the word dictionary to match against.

  - ### DictionaryCompoundWordTokenFilter

    @Deprecated public DictionaryCompoundWordTokenFilter(TokenStream input, CharArraySet dictionary, int minWordSize, int minSubwordSize, int maxSubwordSize, boolean onlyLongestMatch, boolean onlyLongestMatchIgnoreSubwords)
    Deprecated.
    Creates a new `DictionaryCompoundWordTokenFilter`
    Parameters:
      `input` - the `TokenStream` to process
      `dictionary` - the word dictionary to match against.
      `minWordSize` - only words longer than this get processed
      `minSubwordSize` - only subwords longer than this get to the output stream
      `maxSubwordSize` - only subwords shorter than this get to the output stream
      `onlyLongestMatch` - deprecated, use parameter onlyLongestMatchIgnoreSubwords instead
      `onlyLongestMatchIgnoreSubwords` - Subwords are igored, e.g. if a word contains 'schwein', only the longer word 'schwein' will be extracted, the subword 'wein' will be ignored. Supersede parameter onlyLongestMatch

  - ### DictionaryCompoundWordTokenFilter

    public DictionaryCompoundWordTokenFilter(TokenStream input, CharArraySet dictionary, int minWordSize, int minSubwordSize, int maxSubwordSize, boolean onlyLongestMatchIgnoreSubwords)
    Creates a new `DictionaryCompoundWordTokenFilter`
    Parameters:
      `input` - the `TokenStream` to process
      `dictionary` - the word dictionary to match against.
      `minWordSize` - only words longer than this get processed
      `minSubwordSize` - only subwords longer than this get to the output stream
      `maxSubwordSize` - only subwords shorter than this get to the output stream
      `onlyLongestMatchIgnoreSubwords` - Subwords are igored, e.g. if a word contains 'schwein', only the longer word 'schwein' will be extracted, the subword 'wein' will be ignored. Supersede parameter onlyLongestMatch

- ## Method Details

  - ### decompose

    protected void decompose()
    Description copied from class: `CompoundWordTokenFilterBase`
    Decomposes the current `CompoundWordTokenFilterBase.termAtt` and places `CompoundWordTokenFilterBase.CompoundToken` instances in the `CompoundWordTokenFilterBase.tokens` list. The original token may not be placed in the list, as it is automatically passed through this filter.
    Specified by:
      `decompose` in class `CompoundWordTokenFilterBase`