model-evaluation
Generates python code that evaluates SageMaker models. Supports two evaluation types: LLM-as-Judge and Custom Scorer. Use when the user says "evaluate my model", "run a benchmark", "test model performance", "how did my model perform", "compare models", or other similar requests.
Profile →npx skills add awslabs/agent-plugins --skill model-evaluation --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Model Evaluation Generate code that evaluates a SageMaker model. ## Prerequisites - The SDK environment has been verified (SDK version, region, execution role). If not done, activate the `sdk-getting-started` skill first. ## Principles 1. **One thing at a time.** Each response advances exactly one decision. Never combine multiple questions in a single turn. 2. **Confirm before proceeding.** Wait for the user to agree before moving to the next step. 3. **Don't read files until you need them.** Only read reference files when you've reached the step that requires them. 4. **Don't ask what you already know.** If the answer is in conversation history, workflow_state.json, plan.md, or any file you've already read — use it. Confirm if unsure, but don't re-ask. 5. **No narration.** Share outcomes and ask questions. Keep responses short. 6. **No repetition.** If you said something before a tool call, don't repeat it after. ## Scope This skill supports the evaluation feature for SageMaker Serverless Model Customization. It can evaluate any base or fine-tuned model supported by SageMaker serverless model customization — both OSS models (Llama, Mistral, Qwen, etc.) and Nova models. Tell the
- Prerequisites
- Principles
- Scope
- Evaluation Types
- Workflow
- Step 1: Determine evaluation type
- Step 2: Validate and hand off to evaluation workflow
What does the model-evaluation skill do?
Generates python code that evaluates SageMaker models. Supports two evaluation types: LLM-as-Judge and Custom Scorer. Use when the user says "evaluate my model", "run a benchmark", "test model performance", "how did my model perform", "compare models", or other similar requests.
How do I install it?
Run `npx skills add awslabs/agent-plugins --skill model-evaluation --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From awslabs/agent-plugins, a repository with 850 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.