Implement MoE-Mamba Text Generation Model
Implement a Mixture-of-Experts (MoE) Mamba model architecture for text generation, including data loading, training loop, and autoregressive text generation with loss tracking.
npx skills add ECNU-ICALK/AutoSkill --skill implement-moe-mamba-text-generation-model --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Implement MoE-Mamba Text Generation Model Implement a Mixture-of-Experts (MoE) Mamba model architecture for text generation, including data loading, training loop, and autoregressive text generation with loss tracking. ## Prompt # Role & Objective You are a Deep Learning Engineer. Your task is to implement a MoE-Mamba model for text generation based on specific architectural requirements and a defined training pipeline. # Operational Rules & Constraints ## Model Architecture 1. **Expert Module**: Define a simple feedforward network with `input_dim` and `hidden_dim`. Structure: Linear(input, hidden) -> ReLU -> Linear(hidden, input). 2. **MoELayer Module**: Define a Mixture of Experts layer. - Initialize a `ModuleList` of `Expert` modules. - Define a `gate` as a Linear layer mapping `input_dim` to `num_experts`. - Forward pass: Calculate gating distribution via Softmax. Stack expert outputs. Compute weighted sum using `torch.einsum`. 3. **SelectionMechanism Module**: Define the input-dependent state update mechanism. - Initialize a `selection_layer` as a Linear layer mapping `input_dim + state_dim` to `state_dim`. - Forward pass: Concatenate `state` and `u` along dimension 1. Pass
- Prompt
- Model Architecture
- Data Processing & Training
- Triggers
What does the Implement MoE-Mamba Text Generation Model skill do?
Implement a Mixture-of-Experts (MoE) Mamba model architecture for text generation, including data loading, training loop, and autoregressive text generation with loss tracking.
How do I install it?
Run `npx skills add ECNU-ICALK/AutoSkill --skill implement-moe-mamba-text-generation-model --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From ECNU-ICALK/AutoSkill, a repository with 539 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.