juliensimon/flan-t5-large-billsum-qlora

Flan-T5 Large — BillSum QLoRA

A Flan-T5 Large model fine-tuned with QLoRA (8-bit quantization) for summarizing US Congressional bills using the BillSum dataset.

Video walkthrough: Parameter-efficient fine-tuning with QLoRA and Hugging Face

Model Details

Detail	Value
Base model	google/flan-t5-large
Task	Text summarization
Dataset	BillSum (US Congressional bills)
Method	QLoRA with 8-bit quantization (bitsandbytes)
Framework	PEFT 0.5.0

Quantization Config

Parameter	Value
quant_method	bitsandbytes
load_in_8bit	True
llm_int8_threshold	6.0

Usage

from peft import PeftModel
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

base_model = AutoModelForSeq2SeqLM.from_pretrained("google/flan-t5-large", load_in_8bit=True)
model = PeftModel.from_pretrained(base_model, "juliensimon/flan-t5-large-billsum-qlora")
tokenizer = AutoTokenizer.from_pretrained("google/flan-t5-large")

text = "Summarize: " + bill_text
inputs = tokenizer(text, return_tensors="pt", max_length=512, truncation=True)
outputs = model.generate(**inputs, max_new_tokens=150)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))