Struct AiResultCache 

pub struct AiResultCache { /* private fields */ }

foyer-backed in-memory cache of per-row inference results.

foyer::Cache is internally sharded and lock-free on the read path, so AiResultCache is Send + Sync.

Implementations

impl AiResultCache

pub fn new(config: AiResultCacheConfig) -> Self

Create a cache with the given configuration.

pub fn with_defaults() -> Self

Create a cache with default configuration.

pub fn get(&self, key: &AiCacheKey) -> Option<CachedOutput>

Look up a cached result, recording a hit or miss.

pub fn insert(&self, key: AiCacheKey, value: CachedOutput)

Insert or update a cached result.

pub fn hit_count(&self) -> u64

Total cache hits since creation.

pub fn miss_count(&self) -> u64

Total cache misses since creation.

pub fn len(&self) -> usize

Number of entries currently cached.

pub fn is_empty(&self) -> bool

Whether the cache is empty.

Trait Implementations

impl Debug for AiResultCache

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.