网站设计的软件_微信里的小程序找不到了_榆林市网站seo_湖南专业关键词优化

参考地址：
https://github.com/hiyouga/LLaMA-Factory/blob/main/examples/README_zh.md#lora-微调

使用 Adam-mini 进行全参数训练
llamafactory-cli train examples/extras/adam_mini/qwen2_full_sft.yaml

qwen2_full_sft.yaml配置：

### model
model_name_or_path: /data/model/Qwen2.5-0.5B-Instruct### method
stage: sft
do_train: true
finetuning_type: full
use_adam_mini: true### dataset
dataset: alpaca_my_data
template: qwen
cutoff_len: 1024
max_samples: 1000
overwrite_cache: true
preprocessing_num_workers: 1### output
output_dir: saves/qwen2_5-0_5b/full/sft
logging_steps: 10
save_steps: 500
plot_loss: true
overwrite_output_dir: true### train
per_device_train_batch_size: 1
gradient_accumulation_steps: 8
learning_rate: 1.0e-5
num_train_epochs: 100.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true
ddp_timeout: 180000000### eval
val_size: 0.1
per_device_eval_batch_size: 1
eval_strategy: steps
eval_steps: 500

opencompass模型评测工具
opencompass$ python run.py --models vllm_merged_sft --datasets demo_gsm8k_chat_gen demo_math_chat_gen --debug

dataset	version	metric	mode	qwen2.5-0.5b-instruct-hf
demo_gsm8k	1d7fe4	accuracy	gen	39.06
demo_math	393424	accuracy	gen	17.19

dataset	version	metric	mode	qwen2.5-0.5b-instruct-vllm-local
demo_gsm8k	1d7fe4	accuracy	gen	43.75
demo_math	393424	accuracy	gen	20.31

dataset	version	metric	mode	qwen2.5-0.5b-instruct-vllm-sft-merged
demo_gsm8k	1d7fe4	accuracy	gen	37.50
demo_math	393424	accuracy	gen	21.88

dataset	version	metric	mode	internlm2-chat-1.8b-hf	qwen2-1.5b-instruct-hf
demo_gsm8k	1d7fe4	accuracy	gen	32.81	57.81
demo_math	393424	accuracy	gen	14.06	20.31

dataset	version	metric	mode	qwen2.5-0.5b-instruct-vllm-sft
demo_gsm8k	1d7fe4	accuracy	gen	4.69
demo_math	393424	accuracy	gen	1.56