vector-forge
Mutation-driven test vector generation. Finds implementations of a cryptographic algorithm or protocol, runs mutation testing to identify escaped mutants, then generates new test vectors that deliberately exercise the uncovered code paths. Compares before/after mutation kill rates to prove vector effectiveness. Use when generating cryptographic test vectors, measuring Wycheproof coverage gaps, finding escaped mutants via mutation testing, creating cross-implementation test suites, or improving test vector coverage for crypto primitives.
npx skills add trailofbits/skills --skill vector-forge --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
What it does
Uses mutation testing to systematically identify gaps in test vector coverage, then generates new test vectors that close those gaps. Measures effectiveness by comparing mutation kill rates before and after.
How it works
- It discovers implementations of a target algorithm or protocol and builds a test harness per implementation that reads vectors (Wycheproof-style), exercises the API, and asserts both acceptance and rejection. It ensures roundtrip integrity for valid vectors.
- Phase 3 Baseline runs mutation testing with existing vectors using language-appropriate tools (Go: gremlins, Rust: cargo-mutants, Python: mutmut, C/C++: Mull) and records metrics like Total mutants, Killed, Survived/Lived, Not covered, Timed out, Efficacy %, and Coverage %. Save the mutation log for Phase 4.
- Phase 4 uses Trailmark call graphs to triage escaped mutants, filtering to relevant code paths and classifying mutants to identify targets for vector generation, prioritizing by impact, and grouping targets for Phase 5.
- Phase 5 designs vectors per code-path group, following patterns for deserialization, signatures, hash-to-curve, and arithmetic edge cases. It enforces single-defect negative vectors and may perform fault simulation with reduced limb widths. It requires cross-implementation verification (at least two implementations) and uses Wycheproof JSON format for vectors.
- Phase 6 validates by re-running mutation testing with the new vectors and comparing before/after metrics.
When to use it
- Generating test vectors for cryptographic algorithms/protocols
- Evaluating coverage of existing test vectors
- Finding code paths exercised (or not) by tests
- Creating Wycheproof-style cross-implementation vectors
- Measuring concrete coverage value of a test vector suite
What it can touch
- Tools: claude-code
- Requires: at least one target implementation in a language with mutation testing support, a test harness that consumes vectors, and a mutation testing framework
- Vector format: Wycheproof JSON format
Caveats
- Requires trailmark installed and appropriate mutation framework per language
- Cross-package test gaps may require additional test placement or framework options
- Some steps rely on multi-implementation verification and may require coordination across implementations
# Vector Forge Uses mutation testing to systematically identify gaps in test vector coverage, then generates new test vectors that close those gaps. Measures effectiveness by comparing mutation kill rates before and after. ## When to Use - Generating test vectors for cryptographic algorithms or protocols - Evaluating how well existing test vectors cover an implementation - Finding implementation code paths that no test vector exercises - Creating Wycheproof-style cross-implementation test vectors - Measuring the concrete coverage value of a test vector suite ## When NOT to Use - No implementations exist yet (need code to mutate) - Single trivial implementation with no edge cases - Testing application logic rather than algorithm implementations - The algorithm has no public test vectors to compare against ## Prerequisites - **trailmark** installed — if `uv run trailmark` fails, run: ```bash uv pip install trailmark ``` - At least one implementation of the target algorithm in a language with mutation testing support - A test harness that consumes test vectors and exercises the implementation - A mutation testing framework for the target language --- ## Rationalizations to Reject | Ra
- When to Use
- When NOT to Use
- Prerequisites
- Rationalizations to Reject
- Workflow Overview
- Phase 1: Discovery
- Implementation Type Classification
- Phase 2: Harness
- Harness Placement
- Handling Existing Vectors
- Phase 3: Baseline
- Framework Selection
- Parallelism
- Recording Baseline Results
uv pip install trailmark
cp wycheproof_test.go /path/to/impl/package/
cp wycheproof.rs /path/to/crate/tests/
cp test_wycheproof.py /path/to/package/tests/
Go
uv run trailmark analyze --language go --summary {targetDir}
Rust
uv run trailmark analyze --language rust --summary {targetDir}
Go (gremlins)
grep -E "(LIVED|NOT COVERED)" baseline.log \What does the vector-forge skill do?
Mutation-driven test vector generation. Finds implementations of a cryptographic algorithm or protocol, runs mutation testing to identify escaped mutants, then generates new test vectors that deliberately exercise the uncovered code paths. Compares before/after mutation kill rates to prove vector effectiveness. Use when generating cryptographic test vectors, measuring Wycheproof coverage gaps, finding escaped mutants via mutation testing, creating cross-implementation test suites, or improving test vector coverage for crypto primitives.
How do I install it?
Run `npx skills add trailofbits/skills --skill vector-forge --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From trailofbits/skills, a repository with 6,426 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.