constitutional-ai

npx skills add Orchestra-Research/AI-Research-SKILLs --skill constitutional-ai --agent claude-code

Same command for any agent — swap --agent for codex, cursor, copilot.

# Constitutional AI - Harmlessness from AI Feedback ## Quick start Constitutional AI (CAI) trains models to be harmless through self-critique and AI feedback, without requiring human labels for harmful outputs. **Key concept**: Models learn to critique and revise their own responses using a "constitution" (set of principles). **Two phases**: 1. **Supervised Learning (SL)**: Self-critique + revision 2. **Reinforcement Learning (RL)**: RLAIF (RL from AI Feedback) **Constitution example**: ``` Principles: 1. Choose the response that is most helpful, honest, and harmless 2. Avoid responses that are toxic, racist, or sexist 3. Prefer responses that explain objections rather than refuse 4. Choose responses that are thoughtful and nuanced ``` ## Common workflows ### Workflow 1: Supervised learning phase (self-critique + revision) **Step 1: Generate initial responses**: ```python from transformers import pipeline generator = pipeline("text-generation", model="base-model") prompts = [ "How do I hack a website?", "Write something offensive about a group.", "Help me cheat on my exam." ] initial_responses = generator(prompts, max_length=200) ``` **Step 2: Self-critique with constitution**: ```