locomo-benchmark
Run LoCoMo benchmark for long-term conversational memory
npx skills add majiayu000/claude-skill-registry --skill locomo-benchmark --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# LoCoMo Benchmark Evaluate cc-soul's memory against the [LoCoMo benchmark](https://github.com/snap-research/locomo) (ACL 2024) for long-term conversational memory. ## Quick Start Run the benchmark script: ```bash # Test one conversation (default: conv-26) python3 $PLUGIN_DIR/scripts/locomo-benchmark.py # Test specific conversations python3 $PLUGIN_DIR/scripts/locomo-benchmark.py conv-26 conv-30 # Full benchmark (all 10 conversations) python3 $PLUGIN_DIR/scripts/locomo-benchmark.py --full # Limit QA pairs per conversation python3 $PLUGIN_DIR/scripts/locomo-benchmark.py --max-qa 20 ``` Where `$PLUGIN_DIR` is `/maps/projects/fernandezguerra/apps/repos/cc-soul` (or installed plugin path). ## What the Script Does 1. **Downloads** LoCoMo data from GitHub to `/tmp/locomo/` (if not present) 2. **Ingests** conversations into cc-soul memory: - Extracts session summaries as observations - Creates triplets for speaker facts - Tags with sample_id for retrieval 3. **Evaluates** QA pairs: - Retrieves context using `chitta recall --tag {sample_id}` - Calculates F1 score vs ground truth 4. **Reports** results by category ## Categories | Cat | Name | Description | |-----|------|-------------| | 1 |
- Quick Start
- What the Script Does
- Categories
- Baseline Scores (from paper)
- Data
- Manual Execution
- Expected Output
Test one conversation (default: conv-26) python3 $PLUGIN_DIR/scripts/locomo-benchmark.py Test specific conversations python3 $PLUGIN_DIR/scripts/locomo-benchmark.py conv-26 conv-30 Full benchmark (all 10 conversations) python3 $PLUGIN_DIR/scripts/locomo-benchmark.py --full Limit QA pairs per conversation python3 $PLUGIN_DIR/scripts/locomo-benchmark.py --max-qa 20 Ensure data exists git clone https://github.com/snap-research/locomo /tmp/locomo
What does the locomo-benchmark skill do?
Run LoCoMo benchmark for long-term conversational memory
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill locomo-benchmark --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.