Skip to main content

InferenceProvider

laminar_ai::provider

Trait InferenceProvider 

Source 
pub trait InferenceProvider: Send + Sync {
    // Required methods
    fn infer_batch<'life0, 'async_trait>(
        &'life0 self,
        request: InferenceRequest,
    ) -> Pin<Box<dyn Future<Output = Result<InferenceResponse, ProviderError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;
    fn name(&self) -> &'static str;

    // Provided method
    fn intrinsic_labels(&self, _model: &str) -> Option<Vec<String>> { ... }
}
Expand description

Transport over a model backend. Implementors perform I/O only — no task framing, no result parsing. Shared as Arc<dyn InferenceProvider> and driven from the Ring 1 inference worker, never from Ring 0.

Required Methods§

Source

fn infer_batch<'life0, 'async_trait>( &'life0 self, request: InferenceRequest, ) -> Pin<Box<dyn Future<Output = Result<InferenceResponse, ProviderError>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait,

Run one batch of inputs through the model.

§Errors

Returns ProviderError on transport failure, timeout, rate limiting, a malformed response, or an unsupported task.

Source

fn name(&self) -> &'static str

Stable backend-kind identity for logging and the laminar.ai_calls log (e.g. anthropic, openai, local). Constant per implementor.

Provided Methods§

Source

fn intrinsic_labels(&self, _model: &str) -> Option<Vec<String>>

Classifier labels intrinsic to a model, discovered from its own metadata. Returns None for backends that have none (remote providers, embedding models). A local classifier returns its config.json id2label once the model is on disk — the seam that lets a lazily downloaded classifier score without the labels having been known at startup. The default is None.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§