NRTCachingDirectory (Lucene 10.5.0 core API)
Description: declaration: package: org.apache.lucene.store, class: NRTCachingDirectory

Package org.apache.lucene.store

# Class NRTCachingDirectory

java.lang.Object
org.apache.lucene.store.Directory
org.apache.lucene.store.FilterDirectory
org.apache.lucene.store.NRTCachingDirectory

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`, `Accountable`

public class NRTCachingDirectory extends FilterDirectory implements Accountable
Wraps a RAM-resident directory around any provided delegate directory, to be used during NRT search.

This class is likely only useful in a near-real-time context, where indexing rate is lowish but reopen rate is highish, resulting in many tiny files being written. This directory keeps such segments (as well as the segments produced by merging them, as long as they are small enough), in RAM.

This is safe to use: when your app calls {IndexWriter#commit}, all cached files will be flushed from the cached and sync'd.

Here's a simple example usage:

       Directory fsDir = FSDirectory.open(new File("/path/to/index").toPath());
       NRTCachingDirectory cachedFSDir = new NRTCachingDirectory(fsDir, 5.0, 60.0);
       IndexWriterConfig conf = new IndexWriterConfig(analyzer);
       IndexWriter writer = new IndexWriter(cachedFSDir, conf);

This will cache all newly flushed segments, all merges whose expected segment size is `<= 5 MB`, unless the net cached bytes exceeds 60 MB at which point all writes will not be cached (until the net bytes falls below 60 MB).

WARNING: This API is experimental and might change in incompatible ways in the next release.

- ## Field Summary

  ### Fields inherited from class org.apache.lucene.store.FilterDirectory

  `in`

  ### Fields inherited from interface org.apache.lucene.util.Accountable

  `NULL_ACCOUNTABLE`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `NRTCachingDirectory(Directory delegate, double maxMergeSizeMB, double maxCachedMB)`
  We will cache a newly created output if 1) it's a flush or a merge and the estimated size of the merged segment is `<= maxMergeSizeMB`, and 2) the total cached bytes is `<= maxCachedMB`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `void`
  `close()`
  Close this directory, which flushes any cached files to the delegate and then closes the delegate.
  `IndexOutput`
  `createOutput(String name, IOContext context)`
  Creates a new, empty file in the directory and returns an `IndexOutput` instance for appending data to this file.
  `IndexOutput`
  `createTempOutput(String prefix, String suffix, IOContext context)`
  Creates a new, empty, temporary file in the directory and returns an `IndexOutput` instance for appending data to this file.
  `void`
  `deleteFile(String name)`
  Removes an existing file in the directory.
  `protected boolean`
  `doCacheWrite(String name, IOContext context)`
  Subclass can override this to customize logic; return true if this file should be written to the RAM-based cache first.
  `long`
  `fileLength(String name)`
  Returns the byte length of a file in the directory.
  `String[]`
  `listAll()`
  Returns names of all files stored in this directory.
  `String[]`
  `listCachedFiles()`
  `IndexInput`
  `openInput(String name, IOContext context)`
  Opens a stream for reading an existing file.
  `long`
  `ramBytesUsed()`
  Returns an estimate of the JVM heap memory used by this object in bytes.
  `void`
  `rename(String source, String dest)`
  Renames `source` file to `dest` file where `dest` must not already exist in the directory.
  `void`
  `sync(Collection<String> fileNames)`
  Ensures that any writes to these files are moved to stable storage (made durable).
  `String`
  `toString()`

  ### Methods inherited from class org.apache.lucene.store.FilterDirectory

  `ensureOpen, getDelegate, getPendingDeletions, obtainLock, syncMetaData, unwrap`

  ### Methods inherited from class org.apache.lucene.store.Directory

  `copyFrom, getTempFileName, openChecksumInput`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

  ### Methods inherited from interface org.apache.lucene.util.Accountable

  `getChildResources`

- ## Constructor Details

  - ### NRTCachingDirectory

    public NRTCachingDirectory(Directory delegate, double maxMergeSizeMB, double maxCachedMB)
    We will cache a newly created output if 1) it's a flush or a merge and the estimated size of the merged segment is `<= maxMergeSizeMB`, and 2) the total cached bytes is `<= maxCachedMB`

- ## Method Details

  - ### toString

    public String toString()
    Overrides:
      `toString` in class `FilterDirectory`

  - ### listAll

    public String[] listAll() throws IOException
    Description copied from class: `Directory`
    Returns names of all files stored in this directory. The output must be in sorted (UTF-16, java's `String.compareTo(java.lang.String)`) order.
    Overrides:
      `listAll` in class `FilterDirectory`
    Throws:
      `IOException` - in case of I/O error

  - ### deleteFile

    public void deleteFile(String name) throws IOException
    Description copied from class: `Directory`
    Removes an existing file in the directory.

    This method must throw either `NoSuchFileException` or `FileNotFoundException` if `name` points to a non-existing file.

    Overrides:
      `deleteFile` in class `FilterDirectory`
    Parameters:
      `name` - the name of an existing file.
    Throws:
      `IOException` - in case of I/O error

  - ### fileLength

    public long fileLength(String name) throws IOException
    Description copied from class: `Directory`
    Returns the byte length of a file in the directory.

    This method must throw either `NoSuchFileException` or `FileNotFoundException` if `name` points to a non-existing file.

    Overrides:
      `fileLength` in class `FilterDirectory`
    Parameters:
      `name` - the name of an existing file.
    Throws:
      `IOException` - in case of I/O error

  - ### listCachedFiles

    public String[] listCachedFiles()

  - ### createOutput

    public IndexOutput createOutput(String name, IOContext context) throws IOException
    Description copied from class: `Directory`
    Creates a new, empty file in the directory and returns an `IndexOutput` instance for appending data to this file.

    This method must throw `FileAlreadyExistsException` if the file already exists.

    Overrides:
      `createOutput` in class `FilterDirectory`
    Parameters:
      `name` - the name of the file to create.
    Throws:
      `IOException` - in case of I/O error

  - ### sync

    public void sync(Collection<String> fileNames) throws IOException
    Description copied from class: `Directory`
    Ensures that any writes to these files are moved to stable storage (made durable).

    Lucene uses this to properly commit changes to the index, to prevent a machine/OS crash from corrupting the index.

    Overrides:
      `sync` in class `FilterDirectory`
    Throws:
      `IOException`
    See Also:
      - `Directory.syncMetaData()`

  - ### rename

    public void rename(String source, String dest) throws IOException
    Description copied from class: `Directory`
    Renames `source` file to `dest` file where `dest` must not already exist in the directory.

    It is permitted for this operation to not be truly atomic, for example both `source` and `dest` can be visible temporarily in `Directory.listAll()`. However, the implementation of this method must ensure the content of `dest` appears as the entire `source` atomically. So once `dest` is visible for readers, the entire content of previous `source` is visible.

    This method is used by IndexWriter to publish commits.

    Overrides:
      `rename` in class `FilterDirectory`
    Throws:
      `IOException`

  - ### openInput

    public IndexInput openInput(String name, IOContext context) throws IOException
    Description copied from class: `Directory`
    Opens a stream for reading an existing file.

    This method must throw either `NoSuchFileException` or `FileNotFoundException` if `name` points to a non-existing file.

    Overrides:
      `openInput` in class `FilterDirectory`
    Parameters:
      `name` - the name of an existing file.
    Throws:
      `IOException` - in case of I/O error

  - ### close

    public void close() throws IOException
    Close this directory, which flushes any cached files to the delegate and then closes the delegate.
    Specified by:
      `close` in interface `AutoCloseable`
    Specified by:
      `close` in interface `Closeable`
    Overrides:
      `close` in class `FilterDirectory`
    Throws:
      `IOException`

  - ### doCacheWrite

    protected boolean doCacheWrite(String name, IOContext context)
    Subclass can override this to customize logic; return true if this file should be written to the RAM-based cache first.

  - ### createTempOutput

    public IndexOutput createTempOutput(String prefix, String suffix, IOContext context) throws IOException
    Description copied from class: `Directory`
    Creates a new, empty, temporary file in the directory and returns an `IndexOutput` instance for appending data to this file.

    The temporary file name (accessible via `IndexOutput.getName()`) will start with `prefix`, end with `suffix` and have a reserved file extension `.tmp`.

    Overrides:
      `createTempOutput` in class `FilterDirectory`
    Throws:
      `IOException`

  - ### ramBytesUsed

    public long ramBytesUsed()
    Description copied from interface: `Accountable`
    Returns an estimate of the JVM heap memory used by this object in bytes. The method name uses "ram" for historical reasons; only JVM heap memory should be reported. Off-heap resources such as memory-mapped files or native direct buffers should not be included. Negative values are illegal.
    Specified by:
      `ramBytesUsed` in interface `Accountable`