fine-tuning

Adapting a pretrained model to a specific task or domain by continuing training on a smaller, task-specific dataset. More efficient than training from scratch.

Syntax

ai-fundamentals

1. Start with pretrained weights
2. Train on domain-specific data with lower learning rate
3. Evaluate on task-specific metrics

Example

ai-fundamentals

# Fine-tuning with HuggingFace Trainer:
from transformers import Trainer, TrainingArguments

training_args = TrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    learning_rate=2e-5,
    per_device_train_batch_size=16,
    weight_decay=0.01
)

trainer = Trainer(model=model, args=training_args, train_dataset=dataset)
trainer.train()