Module cached_provider 

Cached LLM provider wrapper using Moka

Provides in-memory caching of LLM responses to reduce latency and API costs. Uses the Moka cache library for high-performance concurrent caching with TTL-based expiration.

2025 Best Practices

• Uses SHA-256 hash of request as cache key
• Configurable TTL and max entries
• Thread-safe concurrent access
• Does NOT cache streaming responses

Structs

CachedProvider

Cached LLM provider wrapper

LlmCacheConfig

Configuration for the LLM cache