Struct LocalProvider 

pub struct LocalProvider { /* private fields */ }

Local ONNX provider, backed by a model cache directory.

Implementations

impl LocalProvider

pub fn new(cache_dir: impl Into<PathBuf>) -> Self

Create a provider that resolves models under cache_dir.

Trait Implementations

impl InferenceProvider for LocalProvider

fn infer_batch<'life0, 'async_trait>( &'life0 self, request: InferenceRequest, ) -> Pin<Box<dyn Future<Output = Result<InferenceResponse, ProviderError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Run one batch of inputs through the model.

fn name(&self) -> &'static str

Stable backend-kind identity for logging and the laminar.ai_calls log (e.g. anthropic, openai, local). Constant per implementor.

fn intrinsic_labels(&self, model: &str) -> Option<Vec<String>>

Classifier labels intrinsic to a model, discovered from its own metadata. Returns None for backends that have none (remote providers, embedding models). A local classifier returns its config.json id2label once the model is on disk — the seam that lets a lazily downloaded classifier score without the labels having been known at startup. The default is None.