Bloom by Safety Research
Evaluate any behavior immediately with AI
Quick facts
- Best for
- Evaluate any behavior immediately with AI
- Pricing
- Freemium
- Editor rating
- 4.5 / 5
- Community saves
- 0
About Bloom by Safety Research
Bloom is an open-source tool for the automated evaluation of behaviors in Language Learning Models (LLMs). As a scaffolded evaluation system, Bloom uses an evaluation configuration, also known as a 'seed', which specifies a target behavior, exemplary transcripts, and the types of interactions the user wants to examine. The tool then generates a suite of interactions with the target model intending to expose the selected behavior. The evaluation suite's growth depends on how it is seeded, differing from other evaluations that might use a fixed elicitation technique and prompting pattern, which increases the uniqueness and adaptability of each evaluation run.To ensure reproducibility, Bloom evaluations should always be cited together with their complete seed configuration. The tool also allows users to add API keys from providers and lets them configure behavioral evaluative facets through behaviors.json and seed.yaml files. Additionally, Bloom also includes an interactive viewer that provides an intuitive interface to browse transcripts from the run, showcasing conversation flows with correct formatting.
Pros
- Open-source
- Automated evaluation
- Unique evaluation
- Adaptable evaluation
- Evaluations are reproducible
- API key addition
- Configurable behavioral evaluative facets
- Interactive transcript viewer
- Customization options
- Support for behaviors.json and seed.yaml
- Scaffolded evaluation system
- Target behavior specificity
Cons
- Complex seed configuration
- Depends on evaluation 'seed'May require API keys
- Requires understanding of Json/Yaml
- Manual evaluation configuration
- Results reproducibility linked to 'seed'Requires Python environment setup
- Relies heavily on text files
- Requires specific file formatting
- Limited to Language Learning Models