玩转 Gemma 2，模型的部署与 Fine-Tuning_AI阅读总结 — 包阅AI

Google 近期发布了最新开放模型 Gemma 2，目前与同等规模的开放模型相比，取得了明显的优势，同时在安全可控性上得到了显著的增强。

from transformers import AutoTokenizer, AutoModelForCausalLM

import transformers

import torch

#加载模型

model_id = “google/gemma-2-27b”
dtype = torch.bfloat16
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map=”cuda”,
torch_dtype=dtype,
)

#准备prompt（对话格式）

prompt = “””### Human: I can’t cancel the subscription to your corporate newsletter### Assistant: “””

#将prompt转换成token id并输入model生成内容

inputs = tokenizer.encode(prompt, add_special_tokens=False, return_tensors=”pt”)

outputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=100)

#解码token id，打印输出

print(tokenizer.decode(outputs[0]))

!git clone https://github.com/huggingface/peft.git

!cd examples/sft/

Gemma 2 与开源的 Hugging Face 的 Transformer library 完全兼容，因此用于训练其它模型的脚本在更改模型名称后，便可以开始训练 Gemma 模型。但是需要注意的是，不同模型多轮对话支持的特殊 token 可能不一样，因此与之相对应的 tokenzier 里面 chat 模板需要提前进行处理。本次我们使用 chatml 格式的模板，只需要在示例脚本上设置对应参数，代码对会自动对 tokenizer 进行处理。

!echo ‘export HUGGINGFACE_TOKEN=YOUR_HF_API_TOKEN’ >> ~/.bashrc

!echo ‘export WANDB_API_KEY=YOUR_WANDB_API_KEY’ >> ~/.bashrc

!source ~/.bashrc

!pip install accelerate==0.32.1

!pip install deepspeed==0.14.4

!pip install transformers==4.42.4

!pip install peft==0.11.1

!pip install trl==0.9.6

!pip install bitsandbytes==0.43.1

!pip install wandb==0.17.4

# 任务训练任务启动命令

!accelerate launch –config_file “configs/deepspeed_config.yaml” train.py \
–seed 100 \
–model_name_or_path “google/gemma-2-27b” \
–dataset_name “smangrul/ultrachat-10k-chatml” \
–chat_template_format “chatml” \
–add_special_tokens False \
–append_concat_token False \
–splits “train,test” \
–max_seq_len 2048 \
–num_train_epochs 1 \
–logging_steps 5 \
–log_level “info” \
–logging_strategy “steps” \
–evaluation_strategy “epoch” \
–save_strategy “epoch” \
–bf16 True \
–learning_rate 1e-4 \
–lr_scheduler_type “cosine” \
–weight_decay 1e-4 \
–warmup_ratio 0.0 \
–max_grad_norm 1.0 \
–output_dir “gemma2-27b-sft-deepspeed” \
–per_device_train_batch_size 1 \
–per_device_eval_batch_size 1 \
–gradient_accumulation_steps 4 \
–gradient_checkpointing True \
–use_reentrant False \
–dataset_text_field “content”

# Deespeed配置脚本

compute_environment: LOCAL_MACHINE
debug: false
deepspeed_config:
deepspeed_multinode_launcher: standard
gradient_accumulation_steps: 4
offload_optimizer_device: none
offload_param_device: none
zero3_init_flag: true
zero3_save_16bit_model: true
zero_stage: 3
distributed_type: DEEPSPEED
downcast_bf16: ‘no’
machine_rank: 0
main_training_function: main
mixed_precision: bf16
num_machines: 1
num_processes: 8
rdzv_backend: static
same_network: true
tpu_env: []
tpu_use_cluster: false
tpu_use_sudo: false
use_cpu: false

from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch

model_id = “./gemma2-27b-sft-deepspeed”
dtype = torch.bfloat16

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map=”cuda”,
torch_dtype=dtype,
)

prompt = “””### Human: I can’t cancel the subscription to your corporate newsletter### Assistant: “””

inputs = tokenizer.encode(prompt, add_special_tokens=False, return_tensors=”pt”)
outputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=100)
print(tokenizer.decode(outputs[0]))