shibu-phys/gemma-2-27b-it-241217-2epoch-Llama_lora

Sample Use

Google Colab 上でモデルを読み込み、elyza-tasks-100-TV_0.jsonl のinput からoutput を出力し、jsonl 形式で保存するコードは以下の通りである。

# 必要なライブラリをインストール
%%capture
!pip install unsloth
!pip uninstall unsloth -y && pip install --upgrade --no-cache-dir "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
!pip install -U torch
!pip install -U peft

# 必要なライブラリを読み込み
from unsloth import FastLanguageModel
from peft import PeftModel
import torch
import json
from tqdm import tqdm
import re
import pandas as pd
from datasets import load_dataset
import time

from google.colab import userdata
HF_TOKEN=userdata.get('HF_TOKEN')

# ベースとなるモデルと学習したLoRAのアダプタ（Hugging FaceのIDを指定）。
model_id = "./gemma-2-27b"
adapter_id = "LLMstudy/gemma-2-27b-it-241217-2epoch-Llama_lora"

# 入出力ファイルの設定
test_file_path = './elyza-tasks-100-TV_0.jsonl'
output_file_path = f'./answer_gemma-2-27b-it-241217.jsonl'

# プロンプトフォーマットの定義
prompt = """以下は、タスクを説明する指示です。指示を適切に満たす回答を書いてください。


### 指示:
{}


### 回答:
{}"""

# モデルの読み込み
!huggingface-cli login --token $HF_TOKEN
!huggingface-cli download google/gemma-2-27b --local-dir gemma-2-27b/

# model parameters
max_seq_length = 2048
dtype = None
load_in_4bit = True

# FastLanguageModel インスタンスを作成
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name=model_id,
    dtype=dtype,
    load_in_4bit=load_in_4bit,
    trust_remote_code=True,
)

# 元のモデルにLoRAのアダプタを統合。
model = PeftModel.from_pretrained(model, adapter_id, token = HF_TOKEN)

# 推論
# データセットの読み込み
datasets = []
with open(test_file_path, "r") as f:
    item = ""
    for line in f:
      line = line.strip()
      item += line
      if item.endswith("}"):
        datasets.append(json.loads(item))
        item = ""

# 学習したモデルを用いてタスクを実行
FastLanguageModel.for_inference(model)
start_time = time.time()
results = []
for dt in tqdm(datasets):
  input = dt["input"]
  instruction = prompt.format(input, "")
  inputs = tokenizer([instruction], return_tensors = "pt").to(model.device)

  outputs = model.generate(**inputs, max_new_tokens = 2048, use_cache = True, do_sample=False, repetition_penalty=1.2)
  prediction = tokenizer.decode(outputs[0], skip_special_tokens=True).split('\n### 回答:\n')[-1]

  results.append({"task_id": dt["task_id"], "input": input, "output": prediction})
  print(f"task_id: {dt['task_id']}")
  print(f"prompt: {instruction}")
  print(f"output: {prediction}")
  print("-" * 50)
end_time = time.time()
print(f"Execution Time: {end_time - start_time} seconds")

# jsonlで保存
with open(output_file_path, 'w', encoding='utf-8') as f:
    for result in results:
        json.dump(result, f, ensure_ascii=False)
        f.write('\n')

License

This model is distributed under the following terms:

1. Gemma Terms of Use (see attached Gemma_Terms_of_Use.txt or https://ai.google.dev/gemma/terms)
2. Additional Restrictions:
   • Use of this model, its derivatives, or its outputs is strictly prohibited for any purpose, including research, commercial, or educational purposes, without the explicit permission of the creator.
   • Redistribution of this model or its derivatives in any form is prohibited without the explicit permission of the creator.

---

Model Card for Model ID

Model Details

Model Description

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

• Developed by: [More Information Needed]
• Funded by [optional]: [More Information Needed]
• Shared by [optional]: [More Information Needed]
• Model type: [More Information Needed]
• Language(s) (NLP): [More Information Needed]
• License: [More Information Needed]
• Finetuned from model [optional]: [More Information Needed]

Model Sources [optional]

• Repository: [More Information Needed]
• Paper [optional]: [More Information Needed]
• Demo [optional]: [More Information Needed]

Uses

Direct Use

[More Information Needed]

Downstream Use [optional]

[More Information Needed]

Out-of-Scope Use

[More Information Needed]

Bias, Risks, and Limitations

[More Information Needed]

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

Training Details

Training Data

[More Information Needed]

Training Procedure

Preprocessing [optional]

[More Information Needed]

Training Hyperparameters

• Training regime: [More Information Needed]

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

[More Information Needed]

Factors

[More Information Needed]

Metrics

[More Information Needed]

Results

[More Information Needed]

Summary

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: [More Information Needed]
• Hours used: [More Information Needed]
• Cloud Provider: [More Information Needed]
• Compute Region: [More Information Needed]
• Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

[More Information Needed]

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]

Citation [optional]

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

[More Information Needed]

Model Card Contact

[More Information Needed]