LimitTokenCountFilter (Lucene 10.5.0 common API)
Description: declaration: package: org.apache.lucene.analysis.miscellaneous, class: LimitTokenCountFilter

Package org.apache.lucene.analysis.miscellaneous

# Class LimitTokenCountFilter

java.lang.Object
org.apache.lucene.util.AttributeSource
org.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.TokenFilter
org.apache.lucene.analysis.miscellaneous.LimitTokenCountFilter

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`, `Unwrappable<TokenStream>`

public final class LimitTokenCountFilter extends TokenFilter
This TokenFilter limits the number of tokens while indexing. It is a replacement for the maximum field length setting inside `IndexWriter`.

By default, this filter ignores any tokens in the wrapped `TokenStream` once the limit has been reached, which can result in `reset()` being called prior to `incrementToken()` returning `false`. For most `TokenStream` implementations this should be acceptable, and faster then consuming the full stream. If you are wrapping a `TokenStream` which requires that the full stream of tokens be exhausted in order to function properly, use the `consumeAllTokens` option.

- ## Nested Class Summary

  ## Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

  `AttributeSource.State`

- ## Field Summary

  ### Fields inherited from class org.apache.lucene.analysis.TokenFilter

  `input`

  ### Fields inherited from class org.apache.lucene.analysis.TokenStream

  `DEFAULT_TOKEN_ATTRIBUTE_FACTORY`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `LimitTokenCountFilter(TokenStream in, int maxTokenCount)`
  Build a filter that only accepts tokens up to a maximum number.
  `LimitTokenCountFilter(TokenStream in, int maxTokenCount, boolean consumeAllTokens)`
  Build an filter that limits the maximum number of tokens per field.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `boolean`
  `incrementToken()`
  `void`
  `reset()`

  ### Methods inherited from class org.apache.lucene.analysis.TokenFilter

  `close, end, unwrap`

  ### Methods inherited from class org.apache.lucene.util.AttributeSource

  `addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString`

  ### Methods inherited from class java.lang.Object

  `clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

- ## Constructor Details

  - ### LimitTokenCountFilter

    public LimitTokenCountFilter(TokenStream in, int maxTokenCount)
    Build a filter that only accepts tokens up to a maximum number. This filter will not consume any tokens beyond the maxTokenCount limit
    See Also:
      - `LimitTokenCountFilter(TokenStream,int,boolean)`

  - ### LimitTokenCountFilter

    public LimitTokenCountFilter(TokenStream in, int maxTokenCount, boolean consumeAllTokens)
    Build an filter that limits the maximum number of tokens per field.
    Parameters:
      `in` - the stream to wrap
      `maxTokenCount` - max number of tokens to produce
      `consumeAllTokens` - whether all tokens from the input must be consumed even if maxTokenCount is reached.

- ## Method Details

  - ### incrementToken

    public boolean incrementToken() throws IOException
    Specified by:
      `incrementToken` in class `TokenStream`
    Throws:
      `IOException`

  - ### reset

    public void reset() throws IOException
    Overrides:
      `reset` in class `TokenFilter`
    Throws:
      `IOException`