SFT Trainer

Dataset

dataset_text_field="text": Specifies the field in the dataset containing the text.
max_seq_length=512: Maximum sequence length for input texts.
per_device_train_batch_size=2: Batch size per GPU.
gradient_accumulation_steps=4: Number of steps to accumulate gradients before updating.
optim="paged_adamw_32bit": Specifies the optimizer.
save_steps=50: Save a checkpoint every 50 steps.
logging_steps=5: Log training metrics every 5 steps.
learning_rate=2e-4: The learning rate for training.
fp16=True: Use mixed precision training.
max_grad_norm=0.3: Maximum gradient norm for gradient clipping.
max_steps=200: Total number of training steps.
warmup_ratio=0.03: Portion of training steps used for learning rate warmup.
lr_scheduler_type="linear": Type of learning rate scheduler.
gradient_checkpointing=True: Use gradient checkpointing to save memory.

用llama.cpp 来merge

这里因为 lora是根据fp16 train的而在finetone过程中模型先被dequantized到了nf4(normalize后将weight划分到16个相邻最近的值减小模型大小 16bit → 4bit 每个para)

这里我们选择先将两个fp16的merge 然后再quant 来做inference

这样我们从本来运行不了这个大小的模型现在不光可以finetune 还可以做inference

我们将原始模型(fp16) 转换到 gguf (convert_hf_to_gguf)

python3 llama.cpp/convert_hf_to_gguf.py --outfile ./Meta-Llama-3.1-8B-Instruct-f16.gguf ./llama31-ft/Meta-Llama-3.1-8B-Instruct/

然后lora weight 到gguf (convert_lora_to_gguf)

python3 llama.cpp/convert_lora_to_gguf.py --base ./Meta-Llama-3.1-8B-Instruct --outfile ./lora_adaptor.gguf --outtype f16 ./llama3.1-ft-lora-adaptor

merge(这里需要把 llama.cpp/bin/ 加到path里)

llama-export-lora -m ./Meta-Llama-3.1-8B-Instruct-f16.gguf --lora ./lora_adaptor.gguf -o Llama3.1-FT-merged-F16.gguf

quant

llama.cpp/bin/llama-quantize ./Llama3.1-FT-merged-F16.gguf ./Meta-Llama-3.1-8B-Instruct-FT-Q4_0.gguf 2

Why left padding?

下面这篇文章写的很好

Reference: