GPTCache is a Python library providing a semantic cache for LLM queries, integrated with LangChain and llama_index, to reduce API calls and speed responses.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
GPTCache is a library for creating a semantic cache for LLM queries. It aims to reduce LLM API costs and improve response times by caching responses and using similarity search to handle related queries. The project is integrated with LangChain and llama_index.
How it works
GPTCache uses embeddings to convert queries into vector representations and stores them in a vector store for similarity search. It supports modular components and allows customization of the cache pipeline. It includes metrics such as Hit Ratio, Latency, and Recall to evaluate caching performance.
Getting started
Quick Install: pip install gptcache
Dev install instructions are provided in the README under the Quick Start section, including cloning the repo and installing requirements. The README also includes example usage demonstrating how to initialize the cache and set the OpenAI API key:
from gptcache import cache
from gptcache.adapter import openai
cache.init()
cache.set_openai_key()
The README contains extensive example code blocks showing exact usage for OpenAI integrations, including configuration for embedding, data managers, and post-processing.
Recent releases
Latest release: 0.1.44 (2024-08-01)
- Fix nil memory eviction when using the init_similar_cache method
- Update the version to 0.1.44 Previous releases listed include 0.1.43 (2023-11-28), 0.1.42 (2023-09-28), 0.1.41 (2023-09-14), 0.1.40 (2023-08-23).
Traction
GitHub stars: 8120 Forks: 590 Open issues: 94
Behind the repo
Not provided in the excerpt.
Caveats
License: MIT Created: 2023-03-24 Last push: 2025-07-11 Language: Python