Lucene99HnswVectorsReader (Lucene 10.5.0 core API)
Description: declaration: package: org.apache.lucene.codecs.lucene99, class: Lucene99HnswVectorsReader

Package org.apache.lucene.codecs.lucene99

# Class Lucene99HnswVectorsReader

java.lang.Object
org.apache.lucene.codecs.KnnVectorsReader
org.apache.lucene.codecs.lucene99.Lucene99HnswVectorsReader

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`, `HnswGraphProvider`, `Accountable`, `QuantizedVectorsReader`

public final class Lucene99HnswVectorsReader extends KnnVectorsReader implements QuantizedVectorsReader, HnswGraphProvider
Reads vectors from the index segments along with index data structures supporting KNN search.

WARNING: This API is experimental and might change in incompatible ways in the next release.

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `static final int`
  `EXHAUSTIVE_BULK_SCORE_ORDS`
  `static final List<VectorSimilarityFunction>`
  `SIMILARITY_FUNCTIONS`

  ### Fields inherited from interface org.apache.lucene.util.Accountable

  `NULL_ACCOUNTABLE`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `Lucene99HnswVectorsReader(SegmentReadState state, FlatVectorsReader flatVectorsReader)`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `void`
  `checkIntegrity()`
  Checks consistency of this reader.
  `void`
  `close()`
  `void`
  `finishMerge()`
  Optional: reset or close merge resources used in the reader
  `ByteVectorValues`
  `getByteVectorValues(String field)`
  Returns the `ByteVectorValues` for the given `field`.
  `FloatVectorValues`
  `getFloatVectorValues(String field)`
  Returns the `FloatVectorValues` for the given `field`.
  `HnswGraph`
  `getGraph(String field)`
  Return the stored HnswGraph for the given field.
  `KnnVectorsReader`
  `getMergeInstance()`
  Returns an instance optimized for merging.
  `Map<String,Long>`
  `getOffHeapByteSize(FieldInfo fieldInfo)`
  Returns the desired size of off-heap memory for the given field.
  `ScalarQuantizer`
  `getQuantizationState(String field)`
  `BaseQuantizedByteVectorValues`
  `getQuantizedVectorValues(String field)`
  `CloseableRandomVectorScorerSupplier`
  `getRandomVectorScorerSupplierForMerge(FieldInfo fieldInfo, SegmentWriteState segmentWriteState)`
  Provides a scorer for merging this quantized vector reader.
  `long`
  `ramBytesUsed()`
  Returns an estimate of the JVM heap memory used by this object in bytes.
  `static VectorSimilarityFunction`
  `readSimilarityFunction(DataInput input)`
  `static VectorEncoding`
  `readVectorEncoding(DataInput input)`
  `void`
  `search(String field, byte[] target, KnnCollector knnCollector, AcceptDocs acceptDocs)`
  Return the k nearest neighbor documents as determined by comparison of their vector values for this field, to the given vector, by the field's similarity function.
  `void`
  `search(String field, float[] target, KnnCollector knnCollector, AcceptDocs acceptDocs)`
  Return the k nearest neighbor documents as determined by comparison of their vector values for this field, to the given vector, by the field's similarity function.

  ### Methods inherited from class org.apache.lucene.codecs.KnnVectorsReader

  `mergeOffHeapByteSizeMaps`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

  ### Methods inherited from interface org.apache.lucene.util.Accountable

  `getChildResources`

- ## Field Details

  - ### EXHAUSTIVE_BULK_SCORE_ORDS

    public static final int EXHAUSTIVE_BULK_SCORE_ORDS
    See Also:
      - Constant Field Values

  - ### SIMILARITY_FUNCTIONS

    public static final List<VectorSimilarityFunction> SIMILARITY_FUNCTIONS

- ## Constructor Details

  - ### Lucene99HnswVectorsReader

    public Lucene99HnswVectorsReader(SegmentReadState state, FlatVectorsReader flatVectorsReader) throws IOException
    Throws:
      `IOException`

- ## Method Details

  - ### getMergeInstance

    public KnnVectorsReader getMergeInstance() throws IOException
    Description copied from class: `KnnVectorsReader`
    Returns an instance optimized for merging. This instance may only be consumed in the thread that called `KnnVectorsReader.getMergeInstance()`.

    The default implementation returns `this`

    Overrides:
      `getMergeInstance` in class `KnnVectorsReader`
    Throws:
      `IOException`

  - ### finishMerge

    public void finishMerge() throws IOException
    Description copied from class: `KnnVectorsReader`
    Optional: reset or close merge resources used in the reader

    The default implementation is empty

    Overrides:
      `finishMerge` in class `KnnVectorsReader`
    Throws:
      `IOException`

  - ### readSimilarityFunction

    public static VectorSimilarityFunction readSimilarityFunction(DataInput input) throws IOException
    Throws:
      `IOException`

  - ### readVectorEncoding

    public static VectorEncoding readVectorEncoding(DataInput input) throws IOException
    Throws:
      `IOException`

  - ### ramBytesUsed

    public long ramBytesUsed()
    Description copied from interface: `Accountable`
    Returns an estimate of the JVM heap memory used by this object in bytes. The method name uses "ram" for historical reasons; only JVM heap memory should be reported. Off-heap resources such as memory-mapped files or native direct buffers should not be included. Negative values are illegal.
    Specified by:
      `ramBytesUsed` in interface `Accountable`

  - ### checkIntegrity

    public void checkIntegrity() throws IOException
    Description copied from class: `KnnVectorsReader`
    Checks consistency of this reader.

    Note that this may be costly in terms of I/O, e.g. may involve computing a checksum value against large data files.

    Specified by:
      `checkIntegrity` in class `KnnVectorsReader`
    Throws:
      `IOException`

  - ### getFloatVectorValues

    public FloatVectorValues getFloatVectorValues(String field) throws IOException
    Description copied from class: `KnnVectorsReader`
    Returns the `FloatVectorValues` for the given `field`. The behavior is undefined if the given field doesn't have KNN vectors enabled on its `FieldInfo`. The return value is never `null`.
    Specified by:
      `getFloatVectorValues` in class `KnnVectorsReader`
    Throws:
      `IOException`

  - ### getByteVectorValues

    public ByteVectorValues getByteVectorValues(String field) throws IOException
    Description copied from class: `KnnVectorsReader`
    Returns the `ByteVectorValues` for the given `field`. The behavior is undefined if the given field doesn't have KNN vectors enabled on its `FieldInfo`. The return value is never `null`.
    Specified by:
      `getByteVectorValues` in class `KnnVectorsReader`
    Throws:
      `IOException`

  - ### search

    public void search(String field, float[] target, KnnCollector knnCollector, AcceptDocs acceptDocs) throws IOException
    Description copied from class: `KnnVectorsReader`
    Return the k nearest neighbor documents as determined by comparison of their vector values for this field, to the given vector, by the field's similarity function. The score of each document is derived from the vector similarity in a way that ensures scores are positive and that a larger score corresponds to a higher ranking.

    The search is allowed to be approximate, meaning the results are not guaranteed to be the true k closest neighbors. For large values of k (for example when k is close to the total number of documents), the search may also retrieve fewer than k documents.

    The returned `TopDocs` will contain a `ScoreDoc` for each nearest neighbor, in order of their similarity to the query vector (decreasing scores). The `TotalHits` contains the number of documents visited during the search. If the search stopped early because it hit `visitedLimit`, it is indicated through the relation `TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO`.

    The behavior is undefined if the given field doesn't have KNN vectors enabled on its `FieldInfo`. The return value is never `null`.

    Specified by:
      `search` in class `KnnVectorsReader`
    Parameters:
      `field` - the vector field to search
      `target` - the vector-valued query
      `knnCollector` - a KnnResults collector and relevant settings for gathering vector results
      `acceptDocs` - `Bits` that represents the allowed documents to match, or `null` if they are all allowed to match.
    Throws:
      `IOException`

  - ### search

    public void search(String field, byte[] target, KnnCollector knnCollector, AcceptDocs acceptDocs) throws IOException
    Description copied from class: `KnnVectorsReader`
    Return the k nearest neighbor documents as determined by comparison of their vector values for this field, to the given vector, by the field's similarity function. The score of each document is derived from the vector similarity in a way that ensures scores are positive and that a larger score corresponds to a higher ranking.

    The search is allowed to be approximate, meaning the results are not guaranteed to be the true k closest neighbors. For large values of k (for example when k is close to the total number of documents), the search may also retrieve fewer than k documents.

    The returned `TopDocs` will contain a `ScoreDoc` for each nearest neighbor, in order of their similarity to the query vector (decreasing scores). The `TotalHits` contains the number of documents visited during the search. If the search stopped early because it hit `visitedLimit`, it is indicated through the relation `TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO`.

    The behavior is undefined if the given field doesn't have KNN vectors enabled on its `FieldInfo`. The return value is never `null`.

    Specified by:
      `search` in class `KnnVectorsReader`
    Parameters:
      `field` - the vector field to search
      `target` - the vector-valued query
      `knnCollector` - a KnnResults collector and relevant settings for gathering vector results
      `acceptDocs` - `Bits` that represents the allowed documents to match, or `null` if they are all allowed to match.
    Throws:
      `IOException`

  - ### getGraph

    public HnswGraph getGraph(String field) throws IOException
    Description copied from interface: `HnswGraphProvider`
    Return the stored HnswGraph for the given field.
    Specified by:
      `getGraph` in interface `HnswGraphProvider`
    Parameters:
      `field` - the field containing the graph
    Returns:
      the HnswGraph for the given field if found
    Throws:
      `IOException` - when reading potentially off-heap graph fails

  - ### getOffHeapByteSize

    public Map<String,Long> getOffHeapByteSize(FieldInfo fieldInfo)
    Description copied from class: `KnnVectorsReader`
    Returns the desired size of off-heap memory for the given field. This size can be used to help determine the memory requirements for optimal search performance, which can be greatly affected by page faults when not enough memory is available.

    For reporting purposes, the size of the off-heap index structures is broken down by their file extension, which provides a logical categorization of their purpose, e.g. the `Lucene99HnswVectorsFormat` stores the HNSW graph neighbours lists in a file with the "vex" extension.

    The long value is the size in bytes of the off-heap space needed if the associated index structure were to be fully loaded in memory. While somewhat analogous to `Accountable.ramBytesUsed()` (which reports actual on-heap memory usage), the sizes reported by this method are not actual usage but rather the amount of available memory needed to fully load the index into memory, rather than an actual RAM usage requirement.

    To determine the total desired off-heap memory size for the given field:

         getOffHeapByteSize(field).values().stream().mapToLong(Long::longValue).sum();

    The default implementation returns an empty map.

    Overrides:
      `getOffHeapByteSize` in class `KnnVectorsReader`
    Parameters:
      `fieldInfo` - the fieldInfo
    Returns:
      a map of the desired off-heap memory requirements by category

  - ### close

    public void close() throws IOException
    Specified by:
      `close` in interface `AutoCloseable`
    Specified by:
      `close` in interface `Closeable`
    Throws:
      `IOException`

  - ### getQuantizedVectorValues

    public BaseQuantizedByteVectorValues getQuantizedVectorValues(String field) throws IOException
    Specified by:
      `getQuantizedVectorValues` in interface `QuantizedVectorsReader`
    Throws:
      `IOException`

  - ### getQuantizationState

    public ScalarQuantizer getQuantizationState(String field)
    Specified by:
      `getQuantizationState` in interface `QuantizedVectorsReader`

  - ### getRandomVectorScorerSupplierForMerge

    public CloseableRandomVectorScorerSupplier getRandomVectorScorerSupplierForMerge(FieldInfo fieldInfo, SegmentWriteState segmentWriteState) throws IOException
    Description copied from interface: `QuantizedVectorsReader`
    Provides a scorer for merging this quantized vector reader. This way any additional merging logic can be implemented by the user of this class.
    Specified by:
      `getRandomVectorScorerSupplierForMerge` in interface `QuantizedVectorsReader`
    Parameters:
      `fieldInfo` - fieldInfo of the field to merge
      `segmentWriteState` - the SegmentWriteState to write temporary files if needed
    Returns:
      a scorer over the newly merged flat vectors, which should be closed as it may hold temporary file handles to read over auxiliary data structures
    Throws:
      `IOException` - if an I/O error occurs when merging