Skip to content

ServiceClient

class tinker.ServiceClient(user_metadata=None, project_id=None, **kwargs)

The ServiceClient is the main entry point for the Tinker API. It provides methods to:

  • Query server capabilities and health status
  • Generate TrainingClient instances for model training workflows
  • Generate SamplingClient instances for text generation and inference
  • Generate RestClient instances for REST API operations like listing weights
# Near instant
client = ServiceClient()

# Takes a moment as we initialize the model and assign resources
training_client = client.create_lora_training_client(base_model="Qwen/Qwen3-8B")

# Near-instant
sampling_client = client.create_sampling_client(base_model="Qwen/Qwen3-8B")

# Near-instant
rest_client = client.create_rest_client()

Parameters:

  • user_metadata (dict[str, str] | None, default: None) – Optional metadata attached to the created session.
  • project_id (str | None, default: None) – Optional project ID to attach to the created session.
  • **kwargs (Any) – advanced options passed to the underlying HTTP client, including API keys, headers, and connection settings.

get_server_capabilities()

Query the server's supported features and capabilities.

Returns: GetServerCapabilitiesResponse with available models, features, and limits

capabilities = service_client.get_server_capabilities()
print(f"Supported models: {capabilities.supported_models}")
print(f"Max batch size: {capabilities.max_batch_size}")

Async variant: get_server_capabilities_async()

create_lora_training_client(base_model, rank=32, seed=None, train_mlp=True, train_attn=True, train_unembed=True, user_metadata=None)

Create a TrainingClient for LoRA fine-tuning.

Parameters:

  • base_model (str) – Name of the base model to fine-tune (e.g., "Qwen/Qwen3-8B")
  • rank (int, default: 32) – LoRA rank controlling the size of adaptation matrices (default 32)
  • seed (int | None, default: None) – Random seed for initialization. None means random seed.
  • train_mlp (bool, default: True) – Whether to train MLP layers (default True)
  • train_attn (bool, default: True) – Whether to train attention layers (default True)
  • train_unembed (bool, default: True) – Whether to train unembedding layers (default True)
  • user_metadata (dict[str, str] | None, default: None) – Optional metadata to attach to the training run

Returns: TrainingClient configured for LoRA training

training_client = service_client.create_lora_training_client(
base_model="Qwen/Qwen3-8B",
rank=16,
train_mlp=True,
train_attn=True
)
# Now use training_client.forward_backward() to train

Async variant: create_lora_training_client_async()

create_training_client_from_state(path, user_metadata=None, weights_access_token=None)

Create a TrainingClient from saved model weights.

This loads only the model weights, not optimizer state. To also restore optimizer state (e.g., Adam momentum), use create_training_client_from_state_with_optimizer.

Parameters:

  • path (str) – Tinker path to saved weights (e.g., "tinker://run-id/weights/checkpoint-001")
  • user_metadata (dict[str, str] | None, default: None) – Optional metadata to attach to the new training run
  • weights_access_token (str | None, default: None) – Optional access token for loading checkpoints under a different account.

Returns: TrainingClient loaded with the specified weights

# Resume training from a checkpoint (weights only, optimizer resets)
training_client = service_client.create_training_client_from_state(
"tinker://run-id/weights/checkpoint-001"
)
# Continue training from the loaded state

Async variant: create_training_client_from_state_async()

create_training_client_from_state_with_optimizer(path, user_metadata=None, weights_access_token=None)

Create a TrainingClient from saved model weights and optimizer state.

This is similar to create_training_client_from_state but also restores optimizer state (e.g., Adam momentum), which is useful for resuming training exactly where it left off.

Parameters:

  • path (str) – Tinker path to saved weights (e.g., "tinker://run-id/weights/checkpoint-001")
  • user_metadata (dict[str, str] | None, default: None) – Optional metadata to attach to the new training run
  • weights_access_token (str | None, default: None) – Optional access token for loading checkpoints under a different account.

Returns: TrainingClient loaded with the specified weights and optimizer state

# Resume training from a checkpoint with optimizer state
training_client = service_client.create_training_client_from_state_with_optimizer(
"tinker://run-id/weights/checkpoint-001"
)
# Continue training with restored optimizer momentum

Async variant: create_training_client_from_state_with_optimizer_async()

create_sampling_client(model_path=None, base_model=None, retry_config=None)

Create a SamplingClient for text generation.

Parameters:

  • model_path (str | None, default: None) – Path to saved model weights (e.g., "tinker://run-id/weights/checkpoint-001")
  • base_model (str | None, default: None) – Name of base model to use (e.g., "Qwen/Qwen3-8B")
  • retry_config (RetryConfig | None, default: None) – Optional configuration for retrying failed requests

Returns: SamplingClient configured for text generation

# Use a base model
sampling_client = service_client.create_sampling_client(
base_model="Qwen/Qwen3-8B"
)

# Or use saved weights
sampling_client = service_client.create_sampling_client(
model_path="tinker://run-id/weights/checkpoint-001"
)

Async variant: create_sampling_client_async()

create_rest_client()

Create a RestClient for REST API operations.

The RestClient provides access to various REST endpoints for querying model information, checkpoints, sessions, and managing checkpoint visibility.

Returns: RestClient for accessing REST API endpoints

rest_client = service_client.create_rest_client()

# List checkpoints for a training run
checkpoints = rest_client.list_checkpoints("run-id").result()

# Get training run info
training_run = rest_client.get_training_run("run-id").result()

# Publish a checkpoint
rest_client.publish_checkpoint_from_tinker_path(
"tinker://run-id/weights/checkpoint-001"
).result()