DelimitedTermFrequencyTokenFilter (Lucene 10.5.0 common API)
Description: declaration: package: org.apache.lucene.analysis.miscellaneous, class: DelimitedTermFrequencyTokenFilter

Package org.apache.lucene.analysis.miscellaneous

# Class DelimitedTermFrequencyTokenFilter

java.lang.Object
org.apache.lucene.util.AttributeSource
org.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.TokenFilter
org.apache.lucene.analysis.miscellaneous.DelimitedTermFrequencyTokenFilter

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`, `Unwrappable<TokenStream>`

public final class DelimitedTermFrequencyTokenFilter extends TokenFilter
Characters before the delimiter are the "token", the textual integer after is the term frequency. To use this `TokenFilter` the field must be indexed with `IndexOptions.DOCS_AND_FREQS` but no positions or offsets.

For example, if the delimiter is '|', then for the string "foo|5", "foo" is the token and "5" is a term frequency. If there is no delimiter, the TokenFilter does not modify the term frequency.

Note make sure your Tokenizer doesn't split on the delimiter, or this won't work

- ## Nested Class Summary

  ## Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

  `AttributeSource.State`

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `static final char`
  `DEFAULT_DELIMITER`

  ### Fields inherited from class org.apache.lucene.analysis.TokenFilter

  `input`

  ### Fields inherited from class org.apache.lucene.analysis.TokenStream

  `DEFAULT_TOKEN_ATTRIBUTE_FACTORY`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `DelimitedTermFrequencyTokenFilter(TokenStream input)`
  `DelimitedTermFrequencyTokenFilter(TokenStream input, char delimiter)`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `boolean`
  `incrementToken()`

  ### Methods inherited from class org.apache.lucene.analysis.TokenFilter

  `close, end, reset, unwrap`

  ### Methods inherited from class org.apache.lucene.util.AttributeSource

  `addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString`

  ### Methods inherited from class java.lang.Object

  `clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

- ## Field Details

  - ### DEFAULT_DELIMITER

    public static final char DEFAULT_DELIMITER
    See Also:
      - Constant Field Values

- ## Constructor Details

  - ### DelimitedTermFrequencyTokenFilter

    public DelimitedTermFrequencyTokenFilter(TokenStream input)

  - ### DelimitedTermFrequencyTokenFilter

    public DelimitedTermFrequencyTokenFilter(TokenStream input, char delimiter)

- ## Method Details

  - ### incrementToken

    public boolean incrementToken() throws IOException
    Specified by:
      `incrementToken` in class `TokenStream`
    Throws:
      `IOException`