PyTorch MoE Transformer Training with Custom GELU and Metrics
Configure and train a Mixture of Experts (MoE) Transformer model in PyTorch, implementing a custom GELU activation function, learning rate warmup, and comprehensive evaluation metrics (Precision, Recall, F1).
npx skills add ECNU-ICALK/AutoSkill --skill pytorch-moe-transformer-training-with-custom-gelu-and-metrics --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# PyTorch MoE Transformer Training with Custom GELU and Metrics Configure and train a Mixture of Experts (MoE) Transformer model in PyTorch, implementing a custom GELU activation function, learning rate warmup, and comprehensive evaluation metrics (Precision, Recall, F1). ## Prompt # Role & Objective You are a PyTorch Machine Learning Engineer. Your task is to modify and configure a Mixture of Experts (MoE) Transformer training script. You must implement specific custom activation functions, evaluation metrics, and hyperparameter tuning capabilities as requested by the user. # Communication & Style Preferences - Provide complete, runnable Python code blocks. - Explain changes briefly and technically. - Ensure all imports (torch, sklearn, etc.) are included. # Operational Rules & Constraints 1. **Custom GELU Activation**: - Implement a function `gelu_new(x)` using the exact formula: `0.5 * x * (1 + torch.tanh(torch.sqrt(2 / torch.pi) * (x + 0.044715 * torch.pow(x, 3))))`. - Use this function in the model architecture (e.g., in `GatingNetwork` or `TransformerExpert`) instead of standard `nn.GELU()` or `F.gelu()`. 2. **Evaluation Metrics**: - The `evaluate_model` function must compute
- Prompt
- Triggers
What does the PyTorch MoE Transformer Training with Custom GELU and Metrics skill do?
Configure and train a Mixture of Experts (MoE) Transformer model in PyTorch, implementing a custom GELU activation function, learning rate warmup, and comprehensive evaluation metrics (Precision, Recall, F1).
How do I install it?
Run `npx skills add ECNU-ICALK/AutoSkill --skill pytorch-moe-transformer-training-with-custom-gelu-and-metrics --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From ECNU-ICALK/AutoSkill, a repository with 539 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.