tinker.types.SampleResponse
Generated from tinker 0.30.4 at commit 1e5777e. Source links point at that snapshot.
class tinker.types.SampleResponse()
Response from a sampling request.
Contains generated sequences and optional prompt-level log probabilities. Numpy fields provide direct array access without format conversion. The corresponding Python-list properties convert lazily on first access.
Fields:
- sequences (Sequence[SampledSequence]) – Generated sequences. Each contains token IDs, optional logprobs, and stop reason.
- prompt_logprobs_np (Optional[np.ndarray]) – Per-token log probabilities for the prompt as a 1-D float32 numpy array,
shape
(prompt_length,).NaNat positions where logprobs were not computed (e.g. the first prompt token). None if prompt logprobs were not requested. - topk_prompt_logprobs_np (Optional[TopkLogprobs]) – Top-k prompt logprobs as a pair of dense matrices
(see
TopkLogprobs). None if top-k was not requested. - target_prompt_logprobs (Optional[TensorData]) – Logprobs of the ids in
SampleRequest.target_prompt_logprobs: a float32 tensor of the same shape and layout, 0.0 where the request had-1. None if not requested. -
prompt_cache_hit_tokens (int) – Number of prompt tokens billed as prefix-cache hits.
Counted on the prompt itself: for
num_samples > 1the prompt is shared, so this is not multiplied across samples. Prefill on the shared prompts samples (the remainingnum_samples - 1) is billed as cache hits.
prompt_logprobs()
Per-token log probabilities for the prompt as a Python list.
If prompt_logprobs was set to true in the request, logprobs are
computed for every token in the prompt. Each entry is a float, or
None for positions where logprobs were not computed (e.g. the
first prompt token). Returns None if prompt logprobs were not
requested.
Converted from prompt_logprobs_np on first access (cached afterwards).
Returns: Optional[List[Optional[float]]]
topk_prompt_logprobs()
Top-k prompt logprobs as nested Python lists.
If topk_prompt_logprobs was set to a positive integer k in the request,
the top-k logprobs are computed for every token in the prompt.
For each prompt position: a list of up to k (token_id, logprob)
tuples, or None for positions where logprobs were not computed.
Returns None if top-k was not requested.
Converted from topk_prompt_logprobs_np on first access (cached afterwards).