constitutional-ai
Train safer models with AI-generated feedback.
What it does
Guides language-model safety training through self-critique, response revision, and reinforcement learning from AI feedback.
When to use it
Use for scalable safety alignment without human labels, particularly when clear principles and explainable, less evasive responses are desired.
How to use it
It changes how the agent implements model safety training, using revised responses for supervised learning and AI preferences for reinforcement learning.
What you provide
- An existing project
What you need · 4
Declared as a dependency and used to create the text-generation pipeline.
Declared as a dependency.
Declared as a dependency and used for supervised, reward-model, and PPO training.
For a 7B model, lists one A100 with 40GB for supervised learning and two A100s with 40GB each for reinforcement learning.
Access · 0
No additional actions listed.