quantization
Model quantization for efficient inference and training. Covers precision types (FP32, FP16, BF16, INT8, INT4), BitsAndBytes configuration, memory estimation, and performance tradeoffs.
npx skills add majiayu000/claude-skill-registry --skill quantization --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Model Quantization ## Overview Quantization reduces model precision to save memory and speed up inference. A 7B model at FP32 requires ~28GB, but at 4-bit only ~4GB. ## Quick Reference | Precision | Bits | Memory | Quality | Speed | |-----------|------|--------|---------|-------| | FP32 | 32 | 4x | Best | Slowest | | FP16 | 16 | 2x | Excellent | Fast | | BF16 | 16 | 2x | Excellent | Fast | | INT8 | 8 | 1x | Good | Faster | | INT4 | 4 | 0.5x | Acceptable | Fastest | ## Memory Estimation ```python def estimate_memory(params_billions, precision_bits): """Estimate model memory in GB.""" bytes_per_param = precision_bits / 8 return params_billions * bytes_per_param # Example: 7B model model_size = 7 # billion parameters print(f"FP32: {estimate_memory(7, 32):.1f} GB") # 28 GB print(f"FP16: {estimate_memory(7, 16):.1f} GB") # 14 GB print(f"INT8: {estimate_memory(7, 8):.1f} GB") # 7 GB print(f"INT4: {estimate_memory(7, 4):.1f} GB") # 3.5 GB ``` ## Measure Model Size ```python def get_model_size(model): """Get model size in GB including buffers.""" param_size = sum(p.numel() * p.element_size() for p in model.parameters()) buffer_size = sum(b.numel() * b.element_size() for b in model.buffer
- Overview
- Quick Reference
- Memory Estimation
- Measure Model Size
- Load Model at Different Precisions
- FP32 (Default)
- FP16 / BF16
- 8-bit Quantization
- 4-bit Quantization (Recommended)
- BitsAndBytesConfig Options
- 4-bit Configuration
- Options Explained
- Compare Precision Performance
- Quantization for Training
What does the quantization skill do?
Model quantization for efficient inference and training. Covers precision types (FP32, FP16, BF16, INT8, INT4), BitsAndBytes configuration, memory estimation, and performance tradeoffs.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill quantization --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.