Ground Truth Management

npx skills add majiayu000/claude-skill-registry --skill ground-truth-management-amnadtaowsoam-cerebraskills --agent claude-code

Same command for any agent — swap --agent for codex, cursor, copilot.

# Ground Truth Management ## What is Ground Truth? **Definition:** Correct answers for evaluation - human-verified data that serves as the gold standard for measuring AI performance. ### Example ``` Question: "What is the capital of France?" Ground Truth: "Paris" AI Answer: "Paris" → Correct ✓ AI Answer: "Lyon" → Incorrect ✗ ``` --- ## Why Ground Truth Matters ### Measure Accuracy Objectively ``` Without ground truth: "This answer seems good" (subjective) With ground truth: "Accuracy: 85%" (objective) ``` ### Train and Validate Models ``` Training: Learn from ground truth examples Validation: Measure performance on ground truth test set ``` ### Regression Testing ``` Before change: Accuracy 90% After change: Accuracy 85% → Regression detected! ``` ### Benchmarking ``` Model A: 90% accuracy on ground truth Model B: 85% accuracy on ground truth → Model A is better ``` --- ## Types of Ground Truth ### Exact Match: Single Correct Answer ```json { "question": "What is 2+2?", "answer": "4" } ``` ### Multiple Acceptable Answers ```json { "question": "What is the capital of France?", "acceptable_answers": ["Paris", "paris", "PARIS", "The capital is Paris"] } ``` ### Rubric-Based: Quality S

Git for dataset versioning
git init
git add dataset.jsonl
git commit -m "Initial dataset v1.0"
Tag versions
git tag v1.0
Update dataset
git commit -m "Added 100 new examples"
git tag v1.1
git add dataset/