经验池

概述与目的

Experience Pooling 作为 MetaGPT 中一个复杂的函数输出缓存层。与依赖精确匹配的简单缓存不同，Experience Pooling 可以识别语义上相似的请求并返回先前成功的响应。

来源

该系统由几个关键组件组成

经验管理器：管理经验的生命周期
经验缓存装饰器：拦截函数调用以提供缓存
存储后端：BM25 或基于向量（Chroma）的经验存储
评估系统：用于评估经验质量的评分器和判断器

核心数据模型

Experience Pooling 的核心是 Experience 数据结构，它封装了请求-响应对及其元数据。

来源

metagpt/exp_pool/schema.py16-76

The Experience model stores

req: The original request text
resp: The response content
metric: Performance metrics including time cost, money cost, and quality score
tag: A categorization label (typically function or class name)
Additional metadata like timestamps and types

Experience Manager

The ExperienceManager class manages the lifecycle of experiences, including storage, retrieval, and querying.

来源

The manager provides methods to

Create new experiences
Query existing experiences by similarity
Delete all experiences
Get the total count of stored experiences

It supports two storage backends

BM25: Text-based retrieval with strong keyword matching
Chroma: Vector-based retrieval for semantic similarity

Experience Cache Decorator

The exp_cache decorator is the primary user interface for Experience Pooling. It wraps functions to provide automatic caching and retrieval.

来源

The decorator performs several key functions

Intercepts function calls and serializes requests
Queries for similar existing experiences
Evaluates if any experience is "perfect" for the request
If a perfect experience is found, returns it immediately
Otherwise, executes the original function
Evaluates and stores the new result if writing is enabled

配置

Experience Pooling is highly configurable through the MetaGPT configuration system.

来源

关键配置选项

enabled: Master switch for the Experience Pooling system
enable_read/enable_write: Granular control over operations
retrieval_type: Choose between BM25 (text-based) or Chroma (vector-based) storage
use_llm_ranker: Whether to use LLM to rank retrieved experiences

存储机制

Experience Pooling provides two storage backends with different characteristics

功能	BM25 Storage	Chroma Vector Storage
Retrieval basis	Keyword matching	Semantic similarity
Embedding required	否	是
Storage format	Text index	Vector database
最适合	Exact/keyword queries	Semantic/conceptual queries
配置	`retrieval_type: bm25`	`retrieval_type: chroma`

来源

The system automatically creates and manages the storage based on the configured retrieval type.

Experience Evaluation

To maintain quality, Experience Pooling includes evaluation mechanisms

Scorers: Evaluate the quality of responses
Perfect Judges: Determine if an experience is good enough to use

来源

The default SimpleScorer uses an LLM to evaluate responses on a scale of 1-10, considering how well they address the request. The quality score is stored with the experience and used for future reference.

使用示例

Here's how to use the exp_cache decorator in your code

来源

You can also use more advanced options

与其他系统集成

Experience Pooling is designed to integrate with other MetaGPT systems

来源

Experience Pooling shares architectural patterns with RoleZero's long-term memory system, but serves a different purpose

Experience Pooling: Caches and reuses function outputs across the application
RoleZero Memory: Maintains conversation context for an individual role

Both systems leverage similar RAG (Retrieval-Augmented Generation) technologies for storage and retrieval.

性能考量

When implementing Experience Pooling in your applications, consider

Storage Size: Monitor the growth of your experience pool and implement cleanup strategies for old or low-quality experiences
Evaluation Overhead: The LLM-based scorer adds additional API calls, which can impact performance and costs
Retrieval Precision: BM25 is faster but less semantically aware; Chroma requires embeddings but provides better semantic matching
Cache Hit Rate: Monitor the effectiveness of your caching by tracking how often perfect experiences are found and reused

来源

To enable verbose logging of new experiences for monitoring

Experience Pooling provides a powerful way to improve efficiency and reduce costs in MetaGPT applications by intelligently reusing previous results when appropriate.

经验池