Gemma-3 270M-IT /resonate/ LoRA

LoRA adapter that teaches Gemma-3 270M-IT the /resonate/ reasoning format — stream-of-consciousness thinking followed by a clean answer.

What is /resonate/?

/resonate/
[free-form thinking — cynical, multilingual, associative, honest]
/resonated/
[clean, structured answer]

The model learns to THINK before answering. The /resonate/ block is raw reasoning — it can switch languages, use metaphors, be irreverent. The /resonated/ block is the distilled answer.

Architecture


Base	`unsloth/gemma-3-270m-it` (268.1M params)
Frozen	`embed_tokens` = 167.8M (63%) — all 140 languages preserved
LoRA	R=16, alpha=32, q_proj + v_proj only
Trainable	0.74M (0.3% of total)
Training	3 epochs, 6445 examples, 32 min on A100
Best val loss	2.9241

Key insight

Freezing embed_tokens (63% of the model) preserves the multilingual embedding space. The LoRA adapter only modifies attention projections — teaching the model HOW to think, not WHAT languages to know.

Languages verified working

English, French, German, Russian, Hebrew, Arabic, Japanese, Chinese — all generate coherent text with /resonate/ format after fine-tuning.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

tokenizer = AutoTokenizer.from_pretrained("ataeff/g")
base = AutoModelForCausalLM.from_pretrained("unsloth/gemma-3-270m-it", dtype=torch.bfloat16)
model = PeftModel.from_pretrained(base, "ataeff/g")

prompt = "<start_of_turn>user\nWhat is the meaning of life?<end_of_turn>\n<start_of_turn>model\n"
ids = tokenizer(prompt, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=200, temperature=0.7, do_sample=True)
print(tokenizer.decode(out[0], skip_special_tokens=True))

Training data

resonance_yent_full.jsonl — 6435 examples of /resonate/ format dialogues
resonance_gold_10.jsonl — 10 hand-crafted gold examples (math, philosophy, code, multilingual)