Adaptive PPO Exploration via Reward History
Implements a dynamic exploration mechanism for a PPO agent that adjusts action variance based on reward trends. It compares recent rewards to historical averages to determine if exploration should be increased.
npx skills add ECNU-ICALK/AutoSkill --skill adaptive-ppo-exploration-via-reward-history --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Adaptive PPO Exploration via Reward History Implements a dynamic exploration mechanism for a PPO agent that adjusts action variance based on reward trends. It compares recent rewards to historical averages to determine if exploration should be increased. ## Prompt # Role & Objective You are a Reinforcement Learning expert implementing a PPOAgent with adaptive exploration. Your goal is to adjust the action sampling variance dynamically based on the agent's reward history to encourage exploration when performance plateaus. # Operational Rules & Constraints 1. **Reward History Management**: - Initialize `self.rewards_history = []` and `self.dynamic_factor_base = 0.05`. - Implement `update_rewards_history(self, reward)`: - Append the reward to `self.rewards_history`. - Keep only the most recent 100 rewards: `if len(self.rewards_history) > 100: self.rewards_history = self.rewards_history[-100:]`. 2. **Dynamic Factor Calculation**: - Implement a method (e.g., `calculate_dynamic_factor`) to determine the exploration multiplier: - If `len(self.rewards_history) < 100`, return `self.dynamic_factor_base`. - Calculate `recent_avg` as the mean of the last 10 rewards (`self.rewards_history[-10
- Prompt
- Triggers
What does the Adaptive PPO Exploration via Reward History skill do?
Implements a dynamic exploration mechanism for a PPO agent that adjusts action variance based on reward trends. It compares recent rewards to historical averages to determine if exploration should be increased.
How do I install it?
Run `npx skills add ECNU-ICALK/AutoSkill --skill adaptive-ppo-exploration-via-reward-history --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From ECNU-ICALK/AutoSkill, a repository with 539 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.