Trait InferenceProvider 

pub trait InferenceProvider: Send + Sync {
    // Required methods
    fn infer_batch<'life0, 'async_trait>(
        &'life0 self,
        request: InferenceRequest,
    ) -> Pin<Box<dyn Future<Output = Result<InferenceResponse, ProviderError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;
    fn name(&self) -> &'static str;

    // Provided method
    fn intrinsic_labels(&self, _model: &str) -> Option<Vec<String>> { ... }
}

I/O transport over a model backend. Shared as Arc<dyn InferenceProvider>; driven from Ring 1, never Ring 0.

Required Methods

fn infer_batch<'life0, 'async_trait>( &'life0 self, request: InferenceRequest, ) -> Pin<Box<dyn Future<Output = Result<InferenceResponse, ProviderError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Run one batch of inputs through the model.

Errors

Returns ProviderError on transport failure, timeout, rate limiting, a malformed response, or an unsupported task.

fn name(&self) -> &'static str

Backend name for logging (e.g. anthropic, openai, local).

Provided Methods

fn intrinsic_labels(&self, _model: &str) -> Option<Vec<String>>

Labels intrinsic to a local classifier (config.json id2label). Returns None for remote providers and embedding models. Lets a lazily downloaded classifier return labels without them being known at startup.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl InferenceProvider for AnthropicProvider

impl InferenceProvider for LocalProvider

impl InferenceProvider for OpenAiProvider

impl InferenceProvider for RateLimitedProvider