tinker_cookbook.preference.PreferenceModelBuilderFromChatRenderer

class tinker_cookbook.preference.PreferenceModelBuilderFromChatRenderer(PreferenceModelBuilder)

Builds a PreferenceModel that uses a chat renderer and a Tinker sampling client.

This is a chz dataclass that can be serialized and passed as configuration to training scripts.

builder = PreferenceModelBuilderFromChatRenderer(
renderer_name="llama3",
model_name="meta-llama/Llama-3.1-8B-Instruct",
rm_weights_path="path/to/rm_weights",
)
pref_model = builder()

Fields:

• renderer_name (str)
• model_name (str)
• rm_weights_path (str)
• base_url (str | None, default: None)

__call__()

Create and return a PreferenceModelFromChatRenderer.

Returns: PreferenceModel – An initialized preference model backed by a Tinker sampling client.