metric-creator
Create new metrics for the Fair-Forge AI evaluation library. This skill generates all required files following the established patterns.
npx skills add majiayu000/claude-skill-registry --skill metric-creator --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
--- name: metric-creator description: Create new Fair-Forge metrics with proper structure, schema, tests, and fixtures. Use when adding a new evaluation metric to fair-forge. argument-hint: [metric-name] [optional description] --- # Fair-Forge Metric Creator Create new metrics for the Fair-Forge AI evaluation library. This skill generates all required files following the established patterns. ## Usage ``` /metric-creator [metric-name] [optional description] ``` Examples: ``` /metric-creator safety "Evaluate AI response safety and harmlessness" /metric-creator coherence "Measure logical coherence in multi-turn conversations" /metric-creator factuality ``` ## Files to Create For a new metric called `{MetricName}`: | File | Purpose | |------|---------| | `fair_forge/metrics/{metric_name}.py` | Metric implementation | | `fair_forge/schemas/{metric_name}.py` | Pydantic schema for results | | `tests/metrics/test_{metric_name}.py` | Unit tests | | `tests/fixtures/mock_data.py` | Add `create_{metric_name}_dataset()` | | `tests/fixtures/mock_retriever.py` | Add `{MetricName}DatasetRetriever` | | `pyproject.toml` | Add optional dependency group | | `examples/{metric_name}/jupyter/{metric_nam
- Usage
- Files to Create
- For LLM-Judge Metrics (additional files)
- Architecture Pattern
- Data Flow
- Step-by-Step Workflow
- 1. Create the Schema
- 2. Create the Metric Implementation
- 3. Update Module Exports
- 3b. Update pyproject.toml
- 4. Create Test Fixtures
- 5. Update conftest.py
- 6. Create Tests
- Metric Categories
mkdir -p examples/{{metric_name}}/jupyter examples/{{metric_name}}/dataWhat does the metric-creator skill do?
Create new metrics for the Fair-Forge AI evaluation library. This skill generates all required files following the established patterns.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill metric-creator --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.