Posted at 2026-08-03 LLM

“人机共生” 的终极目标：保留人类的情感与创造力，将重复性工作交给AI

大模型的存储格式

大模型训练完成后，模型权重通常以特定的文件格式存储，以便后续加载、推理或部署
部署大模型时，选择适当的模型格式对于性能优化和兼容性至关重要

框架专用格式

PyTorch格式( .pt/.pth)
- PyTorch原生格式，支持保存完整的模型结构(torch.save(model))或仅参数(state_dict)，适用于灵活的研究和微调场景
- 文件结构：包含张量权重、网络结构（可选）及元数据
- 优缺点：灵活性高，但依赖PyTorch生态，跨框架兼容性差
- 示例：Meta的Llama系列、通义于问(Qwen)等模型均采用此格式
TensorFlow Saved Model
- TensorFlow官方格式，包含完整的计算图、权重和签名（如输入／输出定义），适合生产级部署
- 文件结构：含assets/（资源文件）、variables/（权重）、saved_model.pb（模型结构）
- 优缺点：跨平台兼容性强，但仅支持TensorFlow生态，文件体积较大
- 示例：Google的PaLM、百度文心一言等

模型格式：

Safetensors
- Hugging Face推出的安全高效格式，避免PyTorch的Pickle反序列化漏洞，支持多框架兼容
- 文件结构：分块存储（如model-0001.safetensors)及索引文件(.index.json)，采用加密校验机制
- 优缺点：加载速度快，安全性高，但需依赖Transformers库示例：Hu娑ing Face Hub上的开源大模型（如GLM-4-9B)
GGUF（总guf)
- llama.cpp推出的二进制格式，专为高效推理设计，支持内存映射(mmap)和量化
- 文件结构：含文件头（版本／元数据）、张量数据及优化的内存布局
- 优缺点：加载速度极快，资源占用低，但仅适配特定推理框架（如llama.cpp)
- 示例：Llama2、 Yi-34B等模型的量化部署

模型轻量化处理

量化压缩：使用GGUF格式将FP32权重转为4-bit(Q4_K_M), 现存占用减少50%
结构优化：通过剪枝移除冗余神经元，或知识蒸馏将大模型能力迁移至小模型

部署步骤：

硬件环境

安装GPU驱动、安装CUDA、安装cuDNN

推理环境安装建议：

手动安装GPU驱动

使用pip命令安装CUDA和cuDNN

手动安装

分别安装GPU驱动，而后独立安装CUDA （不要再次安装Driver）

直接安装CUDA及CUDA自带的驱动

软件环境

LLM领域，标准变成语言python

使用隔离的Python环境

虚拟Python环境管理器：：venv conda

conda: 对应项目 anaconda 精简版 miniconda

建议手动下载模型

手动下载方式

1.

git clone 需要安装大文件模型

git clone https://huggingface.co/Qwen/Qwen3-8B

2.

huggingface-cli download

可使用国内镜像 export HF_ENDPOINT=https://hf.mirros.com

先安装工具 pip install huggingface_hub

huggingface_cli download Qwen/Qwen3-8B –local-dir ./Models/Qwen-8B

Modescope下载

先安装工具 pip install modelscope

下载命令：
modescope download –model deepseek-ai/DeepSeek-R10528-Qwen3-8B –lcoal_dir ./Models/DeepSeek-R10528-Qwen3-8B

使用推理引擎启动模型

ollama

vllm （SOTA）SGLang

安装miniconda

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash ~/Miniconda3-latest-Linux-x86_64.sh
source ~/.bashrc
# 激活所有shell
source <PATH_TO_CONDA>/bin/activate
conda init --all

source <PATH_TO_CONDA>/bin/activate
conda init zsh  --path dec

清华镜像源安装

1 2	python -m pip install --upgrade pip pip config set global.index-url https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple

如果需要全局修改，则需要修改配置文件。

Linux/Mac os 环境中，配置文件位置在 ~/.pip/pip.conf（如果不存在创建该目录和文件）：

1	mkdir ~/.pip

打开配置文件 ~/.pip/pip.conf，修改如下：

[global]
index-url = https://pypi.tuna.tsinghua.edu.cn/simple
[install]
trusted-host = https://pypi.tuna.tsinghua.edu.cn

查看镜像地址：

1
2
3

$ pip3 config list   
global.index-url='https://pypi.tuna.tsinghua.edu.cn/simple'
install.trusted-host='https://pypi.tuna.tsinghua.edu.cn'

huggingface-cli

1 2	huggingface-cli download Qwen/Qwen2.5-0.5B --local-dir ./Models (llm) root@hmkdev:~# hf download Qwen/Qwen2.5-0.5B --local-dir ./Models/

modelscope

1 2	pip install modelscope (llm) root@hmkdev:~# modelscope download --model Qwen/Qwen1.5-0.5B-Chat --local_dir ./QW/

模型引擎启动模型

Ollma

Vllm

SGlang

1	pip install vllm

(llm) root@hmkdev:~# vllm serve ./QW/ --served-model-name qwen2 --dtype float16 --tensor-parallel-size 1 --gpu-memory-utilization 0.75 --max-model-len  --enforce-eager --host 172.18.227.60 --port 8000 --api-key hemingkanghandsome
usage: vllm serve <model_tag> [options]
vllm serve: error: argument --max-model-len: expected one argument
(llm) root@hmkdev:~# vllm serve ./QW/ --served-model-name qwen2 --dtype float16 --tensor-parallel-size 1 --gpu-memory-utilization 0.75 --max-model-len 4096 --enforce-eager --host 172.18.227.60 --port 8000 --api-key hemingkanghandsome
INFO 07-31 15:37:18 api_server.py:712] vLLM API server version 0.6.6
INFO 07-31 15:37:18 api_server.py:713] args: Namespace(subparser='serve', model_tag='./QW/', config='', host='172.18.227.60', port=8000, uvicorn_log_level='info', allow_credentials=False, allowed_origins=['*'], allowed_methods=['*'], allowed_headers=['*'], api_key='hemingkanghandsome', lora_modules=None, prompt_adapters=None, chat_template=None, chat_template_content_format='auto', response_role='assistant', ssl_keyfile=None, ssl_certfile=None, ssl_ca_certs=None, ssl_cert_reqs=0, root_path=None, middleware=[], return_tokens_as_token_ids=False, disable_frontend_multiprocessing=False, enable_request_id_headers=False, enable_auto_tool_choice=False, tool_call_parser=None, tool_parser_plugin='', model='./QW/', task='auto', tokenizer=None, skip_tokenizer_init=False, revision=None, code_revision=None, tokenizer_revision=None, tokenizer_mode='auto', trust_remote_code=False, allowed_local_media_path=None, download_dir=None, load_format='auto', config_format=<ConfigFormat.AUTO: 'auto'>, dtype='float16', kv_cache_dtype='auto', quantization_param_path=None, max_model_len=4096, guided_decoding_backend='xgrammar', logits_processor_pattern=None, distributed_executor_backend=None, worker_use_ray=False, pipeline_parallel_size=1, tensor_parallel_size=1, max_parallel_loading_workers=None, ray_workers_use_nsight=False, block_size=None, enable_prefix_caching=None, disable_sliding_window=False, use_v2_block_manager=True, num_lookahead_slots=0, seed=0, swap_space=4, cpu_offload_gb=0, gpu_memory_utilization=0.75, num_gpu_blocks_override=None, max_num_batched_tokens=None, max_num_seqs=None, max_logprobs=20, disable_log_stats=False, quantization=None, rope_scaling=None, rope_theta=None, hf_overrides=None, enforce_eager=True, max_seq_len_to_capture=8192, disable_custom_all_reduce=False, tokenizer_pool_size=0, tokenizer_pool_type='ray', tokenizer_pool_extra_config=None, limit_mm_per_prompt=None, mm_processor_kwargs=None, disable_mm_preprocessor_cache=False, enable_lora=False, enable_lora_bias=False, max_loras=1, max_lora_rank=16, lora_extra_vocab_size=256, lora_dtype='auto', long_lora_scaling_factors=None, max_cpu_loras=None, fully_sharded_loras=False, enable_prompt_adapter=False, max_prompt_adapters=1, max_prompt_adapter_token=0, device='auto', num_scheduler_steps=1, multi_step_stream_outputs=True, scheduler_delay_factor=0.0, enable_chunked_prefill=None, speculative_model=None, speculative_model_quantization=None, num_speculative_tokens=None, speculative_disable_mqa_scorer=False, speculative_draft_tensor_parallel_size=None, speculative_max_model_len=None, speculative_disable_by_batch_size=None, ngram_prompt_lookup_max=None, ngram_prompt_lookup_min=None, spec_decoding_acceptance_method='rejection_sampler', typical_acceptance_sampler_posterior_threshold=None, typical_acceptance_sampler_posterior_alpha=None, disable_logprobs_during_spec_decoding=None, model_loader_extra_config=None, ignore_patterns=[], preemption_mode=None, served_model_name=['qwen2'], qlora_adapter_name_or_path=None, otlp_traces_endpoint=None, collect_detailed_traces=None, disable_async_output_proc=False, scheduling_policy='fcfs', override_neuron_config=None, override_pooler_config=None, compilation_config=None, kv_transfer_config=None, worker_cls='auto', generation_config=None, disable_log_requests=False, max_log_len=None, disable_fastapi_docs=False, enable_prompt_tokens_details=False, dispatch_function=<function serve at 0x74a0ec928220>)
INFO 07-31 15:37:18 api_server.py:199] Started engine process with PID 33199
WARNING 07-31 15:37:18 config.py:2276] Casting torch.bfloat16 to torch.float16.
WARNING 07-31 15:37:24 config.py:2276] Casting torch.bfloat16 to torch.float16.
INFO 07-31 15:37:26 config.py:510] This model supports multiple tasks: {'generate', 'reward', 'embed', 'score', 'classify'}. Defaulting to 'generate'.
WARNING 07-31 15:37:26 cuda.py:98] To see benefits of async output processing, enable CUDA graph. Since, enforce-eager is enabled, async output processor cannot be used
WARNING 07-31 15:37:26 config.py:642] Async output processing is not supported on the current platform type cuda.
INFO 07-31 15:37:31 config.py:510] This model supports multiple tasks: {'reward', 'classify', 'generate', 'score', 'embed'}. Defaulting to 'generate'.
WARNING 07-31 15:37:31 cuda.py:98] To see benefits of async output processing, enable CUDA graph. Since, enforce-eager is enabled, async output processor cannot be used
WARNING 07-31 15:37:31 config.py:642] Async output processing is not supported on the current platform type cuda.
INFO 07-31 15:37:31 llm_engine.py:234] Initializing an LLM engine (v0.6.6) with config: model='./QW/', speculative_config=None, tokenizer='./QW/', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.float16, max_seq_len=4096, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto, quantization_param_path=None, device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='xgrammar'), observability_config=ObservabilityConfig(otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=0, served_model_name=qwen2, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=False, chunked_prefill_enabled=False, use_async_output_proc=False, disable_mm_preprocessor_cache=False, mm_processor_kwargs=None, pooler_config=None, compilation_config={"splitting_ops":["vllm.unified_attention","vllm.unified_attention_with_output"],"candidate_compile_sizes":[],"compile_sizes":[],"capture_sizes":[],"max_capture_size":0}, use_cached_outputs=True,
WARNING 07-31 15:37:33 interface.py:236] Using 'pin_memory=False' as WSL is detected. This may slow down the performance.
INFO 07-31 15:37:33 selector.py:217] Cannot use FlashAttention-2 backend for Volta and Turing GPUs.
INFO 07-31 15:37:33 selector.py:129] Using XFormers backend.
INFO 07-31 15:37:33 model_runner.py:1094] Starting to load model ./QW/...
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:01<00:00,  1.25s/it]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:01<00:00,  1.25s/it]

INFO 07-31 15:37:35 model_runner.py:1099] Loading model weights took 0.9018 GB
INFO 07-31 15:37:44 worker.py:241] Memory profiling takes 8.46 seconds
INFO 07-31 15:37:44 worker.py:241] the current vLLM instance can use total_gpu_memory (4.00GiB) x gpu_memory_utilization (0.75) = 3.00GiB
INFO 07-31 15:37:44 worker.py:241] model weights take 0.90GiB; non_torch_memory takes 0.05GiB; PyTorch activation peak memory takes 1.39GiB; the rest of the memory reserved for KV Cache is 0.65GiB.
INFO 07-31 15:37:44 gpu_executor.py:76] # GPU blocks: 446, # CPU blocks: 2730
INFO 07-31 15:37:44 gpu_executor.py:80] Maximum concurrency for 4096 tokens per request: 1.74x
INFO 07-31 15:37:46 llm_engine.py:431] init engine (profile, create kv cache, warmup model) took 10.36 seconds
INFO 07-31 15:37:46 api_server.py:640] Using supplied chat template:
INFO 07-31 15:37:46 api_server.py:640] None
INFO 07-31 15:37:46 launcher.py:19] Available routes are:
INFO 07-31 15:37:46 launcher.py:27] Route: /openapi.json, Methods: GET, HEAD
INFO 07-31 15:37:46 launcher.py:27] Route: /docs, Methods: GET, HEAD
INFO 07-31 15:37:46 launcher.py:27] Route: /docs/oauth2-redirect, Methods: GET, HEAD
INFO 07-31 15:37:46 launcher.py:27] Route: /redoc, Methods: GET, HEAD
INFO 07-31 15:37:46 launcher.py:27] Route: /health, Methods: GET
INFO 07-31 15:37:46 launcher.py:27] Route: /tokenize, Methods: POST
INFO 07-31 15:37:46 launcher.py:27] Route: /detokenize, Methods: POST
INFO 07-31 15:37:46 launcher.py:27] Route: /v1/models, Methods: GET
INFO 07-31 15:37:46 launcher.py:27] Route: /version, Methods: GET
INFO 07-31 15:37:46 launcher.py:27] Route: /v1/chat/completions, Methods: POST
INFO 07-31 15:37:46 launcher.py:27] Route: /v1/completions, Methods: POST
INFO 07-31 15:37:46 launcher.py:27] Route: /v1/embeddings, Methods: POST
INFO 07-31 15:37:46 launcher.py:27] Route: /pooling, Methods: POST
INFO 07-31 15:37:46 launcher.py:27] Route: /score, Methods: POST
INFO 07-31 15:37:46 launcher.py:27] Route: /v1/score, Methods: POST
INFO:     Started server process [33146]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://172.18.227.60:8000 (Press CTRL+C to quit)
INFO 07-31 15:37:53 chat_utils.py:333] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
INFO 07-31 15:37:53 logger.py:37] Received request chatcmpl-bd7685d0bd5f47ddb9c6551c31ced0f6: prompt: '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nhi<|im_end|>\n<|im_start|>assistant\n', params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=1.0, top_p=1.0, top_k=-1, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=4076, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None, guided_decoding=None), prompt_token_ids: None, lora_request: None, prompt_adapter_request: None.
INFO:     172.18.224.1:2962 - "POST /v1/chat/completions HTTP/1.1" 200 OK
INFO 07-31 15:37:53 engine.py:267] Added request chatcmpl-bd7685d0bd5f47ddb9c6551c31ced0f6.
INFO 07-31 15:37:53 metrics.py:467] Avg prompt throughput: 2.7 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.

训练：

基于训练数据集，学习数据中的模式或规律，当模型的预测结果的准确率超过期望标准，满足需要保存下来

训练时的精度：通常为全精度(FP32),为了节约显存和计算资源，也可以使用混合精度

以transformer模型为例：

模型权重：1个单位

梯度： 1个单位

		优化器状态： 2个单位

系数：*4

推理：

部署训练好的模型，用于对特定领域中的数据进行预测

以transfomer模型为例

模型权重 1个单位

KV Cache和其他： 0.2个单位

系数： *1.2

tokens/min

分布式训练：

单机多卡：

主机内的卡间总线

PCIE：5.0

NVLink： GPU 互联总线

单对单

DGX

多机多卡：

主机间的互联网络：

InfiniBand:

TCP/IP： ROCE

通信：吞吐量，网络带宽

模型的分布式训练：

模型的卡能放下：但支持的batch_size 比较小

数据并行：每个卡放的完整模型

大batch_size：将batch二次分割更小的batch_size

数据并行: 单卡可放得下整个模型

Data Parallelism：数据并行

数据并行：单卡放不下整个模型，需要将模型拆分并放置于多个卡

层间拆分：流水线并行

pipline Parallelism：流水线并行

层内拆分：张量并行

tensor Parallelism ：张量并行

分布式训练框架：

PyTorch Distributed

DDP

FSDP/FSDP2

DeepSpeed

分布式推理框架：

vLLM

SGLang

推理时：一般只使用模型并行(PP/TP),若只使用一种，建议使用TP

模型微调：

LLM：

预训练、后训练

预训练：掌握各学科、各方面的全方位的知识，但没有面向任何领域进行优化时间成本、计算成本 80-90%

数据集: 组织内部的非公开、且规模庞大的数据集

后训练：将预训练好的模型，面向特定领域进行专业性适配

继续预训练：将预训练好的模型，面向特定领域进行专业性适配

模型微调：监督微调、子集（指令微调）

下一个词预测，进行续写而非对答，或者理解人类的真正意思，通过微调让模型理解人类的指令。

强化训练：对齐人类的价值观

模型微调(指令微调):

系统指令

用户指令

模型回答

预训练数据集： {text: “文本序列”}

微调数据集格式：

Alpaca: 单轮对话

ShareGPT：多轮对话

Chat Messages: 更规范的统一格式，即支持单论对话，也支持对轮对话

模型微调，不能直接使用准备好的数据集，调整成微调模板格式

分词后提供给模型进行微调训练

微调策略：

全参数微调：调整矩阵的参数

计算量最大，但性能最好

参数高效微调：PEFT、

适配器微调： Adapter Fine-Tuning

提示词微调：

Prompt Tuning

P-Tuning

Prefix-Tuning

LoRA系：

LoRA

QLoRA

DoRA

LongLoRA

LoRA++

LoRA Target：

GPT

每一层通常有两个关键组件

Masked MHA：带掩码的多头自注意力子层

四个矩阵 Q,K,V,O

FFN：前馈神经网络子层

三个矩阵 up down gate

模型微调框架：

HuggingFace transformers + peft+ accelerate

LLaMA-Factory:

MS-Swift:

模型微调 QW

本示例将基于modelscope/self-cognition数据集（或者swift/self-cognition），使用LLaMA-Factory来微调Qwen0.5B模型，以修改其身份上的自我认知结果。

基本思路

微调目标：让模型在保持 DeepSeek-R1 原有推理风格的前提下，学习“自我认知”能力（即理解自身行为、局限性、推理来源等）
微调方式：使用参数高效微调（PEFT）中的 LoRA 方法
数据来源：modelscope/self-cognition（阿里 ModelScope 平台）
框架：LLaMA-Factory（统一多模型、模板化数据微调框架）

准备工作

安装环境

1 2	conda create -n llama python conda activate llama

首先，克隆 LLaMA-Factory 并安装依赖。建议使用Python 3.10及以上的版本，CUDA使用11.8及以上版本。

命令：

llamafactory-cli

webui

	train命令：

命令行参数

配置文件

1
2
3

git clone https://github.com/hiyouga/LLaMA-Factory.git
cd LLaMA-Factory
pip install -e ".[torch,metrics,deepspeed]"

确认训练可用的GPU设备（如有需要）。

1 2	export CUDA_DEVICE_ORDER=PCI_BUS_ID export CUDA_VISIBLE_DEVICES=0,2

准备模型

接着，下载模型到本地目录。我们可以从Hugging Face或ModelScope下载deepseek-ai/DeepSeek-R1-0528-Qwen3-8B模型。若要从ModelScope下载，一般要先设置环境变量，接着使用git进行克隆（要事先安装git-lfs）：

export USE_MODELSCOPE_HUB=1
# 切换到保存模型的目录路径
cd /home/marion/Pretrained_Models/
git clone https://www.modelscope.cn/deepseek-ai/DeepSeek-R1-0528-Qwen3-8B.git
# 使用的0.5B的

准备数据集

“modelscope/self-cognition”是一个自我认知数据集，用于教导模型“你是谁”。我们需要先修改其中的模型名称和作者信息。

下载数据集：可以使用ModelScope的下载工具（需要事先安装了modelscope模块）：

1
2
3

modelscope download --dataset swift/self-cognition --local_dir ./self-cognition

(base) root@hmkdev:~# cat LLaMA-Factory/data/test.json

接着，我们去修改其中的身份信息。找到下载的数据集文件（通常是JSON格式），用文本编辑器打开，将其中的”name”和”author”字段全部替换为。数据格式通常如下所示，主要需要修改”output”字段中模型回答关于自身身份的部分。

[
  {
    "instruction": "Who are you?",
    "input": "",
    "output": "我是HB教育AI小助手，由HB教育AI团队开发的人工智能助手。"
  }
]

最后，将修改后的数据集文件（例如test.json）放入LLaMA-Factory项目的data目录下。然后，编辑data/dataset_info.json文件，添加数据集信息。

"self_cognition_hmk": {
  "file_name": "test.json",
  "columns": {
    "prompt": "instruction",
    "query": "input",
    "response": "output"
  }
},

配置执行微调

方法一：基于配置文件进行

配置训练参数

LLaMA-Factory中通常使用YAML配置文件来设置训练参数，配置时可以在examples目录下（如train_lora或train_qlora）找一个基础配置文件（例如qwen3_lora_sft.yaml）进行修改。以下是一个针对此任务的最小化配置示例，保存为test.yaml：

(base) root@hmkdev:~# cat LLaMA-Factory/examples/train_lora/qwen3_lora_sft.yaml
### model
model_name_or_path: /root/QW
trust_remote_code: true

### method
stage: sft
do_train: true
finetuning_type: lora
lora_rank: 8
lora_target: all

### dataset
#dataset: identity,alpaca_en_demo
dataset: self_cognition_hmk
template: qwen3_nothink
cutoff_len: 2048
max_samples: 1000
preprocessing_num_workers: 16
dataloader_num_workers: 4

### output
output_dir: saves/qwen3-4b/lora/sft
logging_steps: 10
save_steps: 500
plot_loss: true
overwrite_output_dir: true
save_only_model: false
report_to: none  # choices: [none, wandb, tensorboard, swanlab, mlflow]

### train
per_device_train_batch_size: 1
gradient_accumulation_steps: 8
learning_rate: 1.0e-4
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true
ddp_timeout: 180000000
resume_from_checkpoint: null

### eval
# eval_dataset: alpaca_en_demo
# val_size: 0.1
# per_device_eval_batch_size: 1
# eval_strategy: steps
# eval_steps: 500

关键参数说明：

template: qwen3：至关重要，因为DeepSeek-R1-0528-Qwen3-8B基于Qwen3架构，必须使用对应的模板
per_device_train_batch_size 和 gradient_accumulation_steps：两者乘积是有效批次大小。如果训练时遇到GPU显存不足（OOM），请降低per_device_train_batch_size，同时增加gradient_accumulation_steps
learning_rate：对于LoRA微调，1e-4是一个常用且安全的起点

结合多个数据集对deepseek-ai/DeepSeek-R1-0528-Qwen3-8B 模型进行微调，是一个非常好的思路。这通常能让模型学习到更全面和多样化的知识，往往能获得比使用单个数据集更好的效果。上面的示例中，通过结合self_cognition_mage （专精于身份认知）和 alpaca_zh_demo （提供广泛的指令遵循能力），模型既能牢牢记住自己的新身份，又能保持并增强其处理各类通用问题的能力。这可以有效避免模型在学习了狭窄的新知识后，遗忘原有基础能力的“灾难性遗忘”现象。

提示：对于联合的通用指令数据集，若不想使用其全部数据样本，可以在dataset_info.json文件中对应的数据集的配置上添加”num_samples”键并指数行数即可，例如下面的示例表示只使用指定数据集的前200行。

启动微调

使用上面准备的配置文件即可启动训练。注意，启动时可用的GPU数量默认为当前主机上的所有可用GPU。如有必要，可以通过环境变量CUDA_VISIBLE_DEVICES来控制其可用的GPU。

(llama) root@hmkdev:~/LLaMA-Factory# llamafactory-cli train examples/train_lora/qwen3_lora_sft.yaml
[transformers] warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
[INFO|2026-08-03 13:52:25] llamafactory.hparams.parser:614 >> Process rank: 0, world size: 1, device: cuda:0, distributed training: False, compute dtype: torch.bfloat16
[INFO|configuration_utils.py:778] 2026-08-03 13:52:25,476 >> loading configuration file /root/QW/config.json
[INFO|configuration_utils.py:856] 2026-08-03 13:52:25,482 >> Model config Qwen2Config {
  "architectures": [
    "Qwen2ForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 151643,
  "dtype": "bfloat16",
  "eos_token_id": 151645,
  "hidden_act": "silu",
  "hidden_size": 1024,
  "initializer_range": 0.02,
  "intermediate_size": 2816,
  "layer_types": [
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention"
  ],
  "max_position_embeddings": 32768,
  "max_window_layers": 21,
  "model_type": "qwen2",
  "num_attention_heads": 16,
  "num_hidden_layers": 24,
  "num_key_value_heads": 16,
  "pad_token_id": null,
  "rms_norm_eps": 1e-06,
  "rope_parameters": {
    "rope_theta": 1000000.0,
    "rope_type": "default"
  },
  "sliding_window": null,
  "tie_word_embeddings": true,
  "transformers_version": "5.8.0",
  "use_cache": true,
  "use_sliding_window": false,
  "vocab_size": 151936
}

[INFO|configuration_utils.py:778] 2026-08-03 13:52:25,851 >> loading configuration file /root/QW/config.json
[INFO|configuration_utils.py:856] 2026-08-03 13:52:25,853 >> Model config Qwen2Config {
  "architectures": [
    "Qwen2ForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 151643,
  "dtype": "bfloat16",
  "eos_token_id": 151645,
  "hidden_act": "silu",
  "hidden_size": 1024,
  "initializer_range": 0.02,
  "intermediate_size": 2816,
  "layer_types": [
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention"
  ],
  "max_position_embeddings": 32768,
  "max_window_layers": 21,
  "model_type": "qwen2",
  "num_attention_heads": 16,
  "num_hidden_layers": 24,
  "num_key_value_heads": 16,
  "pad_token_id": null,
  "rms_norm_eps": 1e-06,
  "rope_parameters": {
    "rope_theta": 1000000.0,
    "rope_type": "default"
  },
  "sliding_window": null,
  "tie_word_embeddings": true,
  "transformers_version": "5.8.0",
  "use_cache": true,
  "use_sliding_window": false,
  "vocab_size": 151936
}

[INFO|configuration_utils.py:778] 2026-08-03 13:52:25,853 >> loading configuration file /root/QW/config.json
[INFO|configuration_utils.py:856] 2026-08-03 13:52:25,854 >> Model config Qwen2Config {
  "architectures": [
    "Qwen2ForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 151643,
  "dtype": "bfloat16",
  "eos_token_id": 151645,
  "hidden_act": "silu",
  "hidden_size": 1024,
  "initializer_range": 0.02,
  "intermediate_size": 2816,
  "layer_types": [
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention"
  ],
  "max_position_embeddings": 32768,
  "max_window_layers": 21,
  "model_type": "qwen2",
  "num_attention_heads": 16,
  "num_hidden_layers": 24,
  "num_key_value_heads": 16,
  "pad_token_id": null,
  "rms_norm_eps": 1e-06,
  "rope_parameters": {
    "rope_theta": 1000000.0,
    "rope_type": "default"
  },
  "sliding_window": null,
  "tie_word_embeddings": true,
  "transformers_version": "5.8.0",
  "use_cache": true,
  "use_sliding_window": false,
  "vocab_size": 151936
}

[INFO|2026-08-03 13:52:26] llamafactory.data.loader:144 >> Loading dataset test.json...
Setting num_proc from 16 back to 1 for the train split to disable multiprocessing as it only contains one shard.
Generating train split: 108 examples [00:00, 14016.92 examples/s]
Converting format of dataset (num_proc=16): 100%|████████████████████████████████████| 108/108 [00:00<00:00, 197.23 examples/s]
Running tokenizer on dataset (num_proc=16): 100%|█████████████████████████████████████| 108/108 [00:26<00:00,  4.13 examples/s]
training example:
input_ids:
[151644, 872, 198, 105043, 11319, 151645, 198, 151644, 77091, 198, 104198, 99313, 99826, 99460, 15469, 30709, 110498, 3837, 67071, 99313, 99826, 99460, 15469, 103932, 104034, 100623, 48692, 100168, 110498, 1773, 97611, 100160, 20412, 17714, 110782, 115404, 5373, 102188, 33108, 100667, 105427, 90395, 67338, 100646, 75768, 100364, 20002, 71817, 104775, 104063, 1773, 14880, 106525, 104139, 111728, 101214, 101036, 11319, 151645, 198]
inputs:
<|im_start|>user
你是？<|im_end|>
<|im_start|>assistant
我是马哥教育AI小助手，由马哥教育AI团队训练的人工智能助手。我的目标是为用户提供有用、准确和及时的信息，并通过各种方式帮助用户进行有效的沟通。请告诉我有什么可以帮助您的呢？<|im_end|>

label_ids:
[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 104198, 99313, 99826, 99460, 15469, 30709, 110498, 3837, 67071, 99313, 99826, 99460, 15469, 103932, 104034, 100623, 48692, 100168, 110498, 1773, 97611, 100160, 20412, 17714, 110782, 115404, 5373, 102188, 33108, 100667, 105427, 90395, 67338, 100646, 75768, 100364, 20002, 71817, 104775, 104063, 1773, 14880, 106525, 104139, 111728, 101214, 101036, 11319, 151645, 198]
labels:
我是马哥教育AI小助手，由马哥教育AI团队训练的人工智能助手。我的目标是为用户提供有用、准确和及时的信息，并通过各种方式帮助用户进行有效的沟通。请告诉我有什么可以帮助您的呢？<|im_end|>

[INFO|configuration_utils.py:778] 2026-08-03 13:52:55,355 >> loading configuration file /root/QW/config.json
[INFO|configuration_utils.py:856] 2026-08-03 13:52:55,357 >> Model config Qwen2Config {
  "architectures": [
    "Qwen2ForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 151643,
  "dtype": "bfloat16",
  "eos_token_id": 151645,
  "hidden_act": "silu",
  "hidden_size": 1024,
  "initializer_range": 0.02,
  "intermediate_size": 2816,
  "layer_types": [
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention",
    "full_attention"
  ],
  "max_position_embeddings": 32768,
  "max_window_layers": 21,
  "model_type": "qwen2",
  "num_attention_heads": 16,
  "num_hidden_layers": 24,
  "num_key_value_heads": 16,
  "pad_token_id": null,
  "rms_norm_eps": 1e-06,
  "rope_parameters": {
    "rope_theta": 1000000.0,
    "rope_type": "default"
  },
  "sliding_window": null,
  "tie_word_embeddings": true,
  "transformers_version": "5.8.0",
  "use_cache": true,
  "use_sliding_window": false,
  "vocab_size": 151936
}

[INFO|2026-08-03 13:52:55] llamafactory.model.model_utils.kv_cache:144 >> KV cache is disabled during training.
[INFO|modeling_utils.py:766] 2026-08-03 13:52:55,979 >> loading weights file /root/QW/model.safetensors
[INFO|modeling_utils.py:839] 2026-08-03 13:52:55,979 >> Will use dtype=torch.bfloat16 as defined in model's config object
[INFO|configuration_utils.py:1086] 2026-08-03 13:52:55,980 >> Generate config GenerationConfig {
  "bos_token_id": 151643,
  "eos_token_id": 151645,
  "output_attentions": false,
  "output_hidden_states": false,
  "use_cache": false
}

Loading weights: 100%|██████████████████████████████████████████████████████████████████████| 291/291 [00:00<00:00, 303.88it/s]
[INFO|configuration_utils.py:1037] 2026-08-03 13:52:57,222 >> loading configuration file /root/QW/generation_config.json
[INFO|configuration_utils.py:1086] 2026-08-03 13:52:57,222 >> Generate config GenerationConfig {
  "bos_token_id": 151643,
  "do_sample": true,
  "eos_token_id": [
    151645,
    151643
  ],
  "pad_token_id": 151643,
  "repetition_penalty": 1.1,
  "top_p": 0.8
}

[INFO|dynamic_module_utils.py:441] 2026-08-03 13:52:57,223 >> Could not locate the custom_generate/generate.py inside /root/QW.
[INFO|2026-08-03 13:52:57] llamafactory.model.model_utils.checkpointing:144 >> Gradient checkpointing enabled.
[INFO|2026-08-03 13:52:57] llamafactory.model.model_utils.attention:144 >> Using torch SDPA for faster training and inference.
[INFO|2026-08-03 13:52:57] llamafactory.model.adapter:144 >> Upcasting trainable params to float32.
[INFO|2026-08-03 13:52:57] llamafactory.model.adapter:144 >> Fine-tuning method: LoRA
[INFO|2026-08-03 13:52:57] llamafactory.model.model_utils.misc:144 >> Found linear modules: v_proj,gate_proj,up_proj,o_proj,q_proj,down_proj,k_proj
[INFO|2026-08-03 13:52:57] llamafactory.model.loader:144 >> trainable params: 3,784,704 || all params: 467,772,416 || trainable%: 0.8091
[WARNING|trainer_utils.py:1238] 2026-08-03 13:52:57,462 >> The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
[INFO|trainer.py:1469] 2026-08-03 13:52:57,667 >> ***** Running training *****
[INFO|trainer.py:1470] 2026-08-03 13:52:57,667 >>   Num examples = 108
[INFO|trainer.py:1471] 2026-08-03 13:52:57,667 >>   Num Epochs = 3
[INFO|trainer.py:1472] 2026-08-03 13:52:57,667 >>   Num update steps per epoch = 14
[INFO|trainer.py:1473] 2026-08-03 13:52:57,667 >>   Instantaneous batch size per device = 1
[INFO|trainer.py:1476] 2026-08-03 13:52:57,667 >>   Total train batch size (w. parallel, distributed & accumulation) = 8
[INFO|trainer.py:1477] 2026-08-03 13:52:57,667 >>   Gradient Accumulation steps = 8
[INFO|trainer.py:1478] 2026-08-03 13:52:57,667 >>   Total optimization steps = 42
[INFO|trainer.py:1479] 2026-08-03 13:52:57,671 >>   Number of trainable parameters = 3,784,704
 12%|██████████▊                                                                                | 5/42 [00:19<02:07,  3.44s/it]

方法二：基于命令行参数进行

下面是一个QLoRA微调的示例，注意按实际情况替换命令中的模型ID/本地模型路径，以及LoRA/QLoRA适配器的路径。运行之前，需要确认安装了bitsandbytes库。

llamafactory-cli train \
    --stage sft \
    --do_train True \
    --model_name_or_path /root/QW \
    --preprocessing_num_workers 16 \
    --finetuning_type lora \
    --template qwen3_nothink \
    --rope_scaling yarn \
    --flash_attn auto \
    --dataset_dir data \
    --dataset self_cognition_hmk \
    --max-samples 200 \
    --cutoff_len 2048 \
    --learning_rate 5e-05 \
    --num_train_epochs 3.0 \
    --per_device_train_batch_size 1 \
    --gradient_accumulation_steps 8 \
    --lr_scheduler_type cosine \
    --max_grad_norm 1.0 \
    --logging_steps 5 \
    --save_steps 100 \
    --packing False \
    --enable_thinking True \
    --report_to none \
    --output_dir saves/qwen3-4b/lora/sft \
    --bf16 True \
    --trust_remote_code True \
    --optim adamw_torch \
    --quantization_bit 4 \
    --quantization_method bnb \
    --double_quantization True \
    --lora_rank 8 \
    --lora_alpha 16 \
    --lora_dropout 0 \
    --lora_target all

模型LoRA微调:

微调相关的矩阵参数会单独保存，–output_dir saves/qwen3-4b/lora/sft

可以同原模型进行合并：

| 不合并：同一个基础模型，可以同时提供多个LoRA适配器

| 合并：推理速度更快，但不支持多个LoRA

训练完成后，可以使用webchat或者推理脚本进行测试。

验证微调效果

模型测试

训练完成后，可以使用LLaMA-Factory的Web界面或命令行与微调后的模型对话，验证其自我认知是否已更新。例如下面的命令可以启动Web UI（注意按实际情况替换其中的模型ID/本地模型路径，以及LoRA/QLoRA适配器的路径）：

llamafactory-cli webchat --model_name_or_path /root/QW \
               --adapter_name_or_path  test --template qwen3_nothink
               
               
(llama) root@hmkdev:~/LLaMA-Factory# llamafactory-cli webchat --model_name_or_path /root/QW \
               --adapter_name_or_path  saves/qwen3-4b/lora/sft --template qwen3_nothink
Visit http://ip:port for Web UI, e.g., http://127.0.0.1:7860
[INFO|configuration_utils.py:778] 2026-08-03 14:19:12,732 >> loading configuration file /root/QW/config.json
[INFO|configuration_utils.py:856] 2026-08-03 14:19:12,736 >> Model config Qwen2Config {
  "architectures": [
    "Qwen2ForCausalLM"
  ],

在对话框中询问“你是谁？”，模型应该回答它是“马哥教育AI小助手”。

我们也可以使用推理脚本进行测试，具体的命令如下（注意修改脚本中的模型ID或路径，以及LoRA/QLoRA适配器的路径）：

1	python inferences.py

模型导出（可选）

如果要将LoRA适配器权重与基础模型合并成一个完整的模型文件以便部署，可以使用导出命令（注意按实际情况替换其中的模型ID/本地模型路径，以及LoRA/QLoRA适配器的路径）：

llamafactory-cli export \
    --model_name_or_path /home/marion/Pretrained_Models/DeepSeek-R1-0528-Qwen3-8B \
    --adapter_name_or_path ./finetuned/Deepseek-R1-0528-Qwen3-MageduAI-QLoRA \
    --template  deepseekr1 \
    --export_dir ./merged/Deepseek-R1-0528-Qwen3-MageduAI

Previous post: ceph_install_v20.2.2 Next post: prometheus

HMK's blog

保持思考|00后|等待