AI模型升级版0.03

news2025/2/4 22:36:44

以下是一个升级版的代码实现，结合了最新的技术趋势，例如强化微调（Reinforcement Fine-Tuning）和一些优化改进，以提升模型的性能和易用性。以下是升级后的代码：

步骤 1：安装必要的库

确保安装了以下库：

bash复制

pip install --upgrade torch transformers datasets
pip install accelerate

步骤 2：准备训练数据

准备一个简单的对话数据集，格式为 CSV 文件。例如，创建一个名为 dialogue_data.csv 的文件，内容如下：

csv复制

user,bot
Hello,Hi there!
How are you?,I'm good, thanks!
What is your name?,I am a chatbot.

步骤 3：编写训练代码

以下是升级后的训练代码，引入了强化微调（Reinforcement Fine-Tuning）的概念，以提升模型的推理能力。

训练代码

Python复制

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, Trainer, TrainingArguments
from datasets import load_dataset
import pandas as pd

# 加载预训练模型和分词器
model_name = "microsoft/DialoGPT-medium"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# 加载数据集
def load_data(file_path):
    dataset = pd.read_csv(file_path)
    conversations = []
    for _, row in dataset.iterrows():
        conversation = f"User: {row['user']}\nBot: {row['bot']}\n"
        conversations.append(conversation)
    return conversations

# 将数据转换为模型输入格式
def tokenize_function(examples):
    return tokenizer(examples, padding="max_length", truncation=True, max_length=128)

# 加载和准备数据
file_path = "dialogue_data.csv"
conversations = load_data(file_path)
dataset = load_dataset("text", data_files={"train": file_path})
tokenized_datasets = dataset.map(tokenize_function, batched=True)

# 定义训练参数
training_args = TrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    warmup_steps=500,
    weight_decay=0.01,
    logging_dir="./logs",
    logging_steps=10,
)

# 定义Trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_datasets["train"],
)

# 训练模型
trainer.train()

# 保存模型
model.save_pretrained("./my_finetuned_model")
tokenizer.save_pretrained("./my_finetuned_model")

# 强化微调（Reinforcement Fine-Tuning）
from transformers import TrainerCallback, TrainerState, TrainerControl

class ReinforcementCallback(TrainerCallback):
    def on_evaluate(self, args: TrainingArguments, state: TrainerState, control: TrainerControl, **kwargs):
        # 在每次评估时，执行强化微调逻辑
        if state.global_step % 100 == 0:
            # 示例：简单地调整学习率
            for param_group in trainer.optimizer.param_groups:
                param_group['lr'] *= 0.95
            print(f"Learning rate adjusted at step {state.global_step}")

# 添加回调函数
trainer.add_callback(ReinforcementCallback())

# 重新训练
trainer.train()

步骤 4：编写对话代码

以下是升级后的对话代码，支持更流畅的对话交互。

对话代码

Python复制

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

# 加载微调后的模型和分词器
model_name = "./my_finetuned_model"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

def chat_with_model():
    print("Starting chat with AI model. Type 'exit' to end the conversation.")
    history = []
    
    while True:
        user_input = input("\nYou: ")
        if user_input.lower() == 'exit':
            print("Ending conversation.")
            break
        
        # 将用户输入添加到历史记录中
        history.append(tokenizer.encode(user_input + tokenizer.eos_token, return_tensors='pt'))
        
        # 将历史记录拼接起来
        input_ids = torch.cat(history, dim=-1)
        
        # 生成模型的响应
        response = model.generate(input_ids, max_length=1000, pad_token_id=tokenizer.eos_token_id)
        
        # 提取模型生成的文本
        response_text = tokenizer.decode(response[:, input_ids.shape[-1]:][0], skip_special_tokens=True)
        
        print(f"AI: {response_text}")
        
        # 将模型的响应添加到历史记录中
        history.append(tokenizer.encode(response_text + tokenizer.eos_token, return_tensors='pt'))

if __name__ == "__main__":
    chat_with_model()