tinker.types.LoraConfig

class tinker.types.LoraConfig(StrictBase)

Fields:

• rank (int) – LoRA rank (dimension of low-rank matrices)

• seed (Optional[int]) – Seed used for initialization of LoRA weights.

  Useful if you need deterministic or reproducible initialization of weights.

• train_unembed (bool) – Whether to add lora to the unembedding layer

• train_mlp (bool) – Whether to add loras to the MLP layers (including MoE layers)
• train_attn (bool) – Whether to add loras to the attention layers