​
Integrating with Braintrust

This guide explains how to bridge Reward Kit with Braintrust. You can log Reward Kit evaluations to Braintrust or reuse Braintrust scorers as Reward Kit reward functions.

​
Installation

Install the Braintrust SDK in your environment: 
pip install braintrust

​
Using a Braintrust scorer in Reward Kit

Convert a Braintrust-style scorer to a Reward Kit reward function using scorer_to_reward_fn: 
from braintrust import Eval
from reward_kit.integrations.braintrust import scorer_to_reward_fn


def equality_scorer(input: str, output: str, expected: str) -> float:
    return 1.0 if output == expected else 0.0

reward_fn = scorer_to_reward_fn(equality_scorer)


def hi_bot_task(name: str) -> str:
    return "Hi " + name


Eval(
    "Reward Kit Braintrust Example",
    data=lambda: [
        {"input": "Foo", "expected": "Hi Foo"},
        {"input": "Bar", "expected": "Hello Bar"},
    ],
    task=hi_bot_task,
    scores=[reward_fn],
)
Run the script with your Braintrust API key: 
BRAINTRUST_API_KEY=<your key> braintrust eval examples/braintrust_integration.py
This will create an experiment in Braintrust where you can inspect the scores, outputs and metadata.