Efficient LLM Fine-Tuning on Intel AI PCs

Most approaches to fine-tuning large language models on a PC require compromises between model size, training speed, and final performance, forcing developers to choose between sophisticated techniques and practical workarounds. Intel's Panther Lake architecture, featuring the Intel Arc GPU, enables previously impractical local reinforcement learning (RL) fine-tuning on Windows laptops. Developers can execute complex fine-tuning workloads through enhanced hardware optimizations and seemless Hugging Face ecosystem integration. We tried out a couple of use cases and are sharing the results and lessons learned so you can get started quickly with your own fine-tuning projects on Panther Lake.

Use Case Configurations

We evaluated two scenarios: a Math QA task using a small model and a Biomedical QA task using a larger, quantized model. Both utilized Group Relative Optimization (GRPO) to improve reasoning and formatting.

Table 1: Use Case Configurations & Strategies

	Math QA	Biomedical QA
Model	Qwen2.5-1.5B-Instruct	Llama3-8B-Instruct
Methods	LoRA + GRPO	QLoRA (NF4) + GRPO
Dataset	gsm8k	pubmedqa
Learning Rate (LR)	5×10-5	6×10-5
LR Scheduler Type	cosine	cosine
LoRA Rank	64	64
Num Generations	10	4
Max Sequence Length	2048	2048
Strategies	Small model, High LR with limited steps, High generation count	4-bit quantization, High LR with limited steps, Small generation count

Software Stack & Environment

Efficient local fine-tuning is achieved through a specialized stack optimized for Intel’s XPU architecture:

• TRL (Transformers Reinforcement Learning): Implements GRPO, a rewards-based technique that optimizes multiple objectives.
• Unsloth: Delivers fast training speeds and reduced memory requirements through a combination of advanced kernel optimizations (we used Triton) and efficient memory management.
• PEFT (Parameter-Efficient Fine-Tuning):
  • LoRA: Updates low-rank matrices to minimize trainable parameters.
  • QLoRA: Uses 4-bit quantization, enabling models to run in laptop VRAM

This combination transforms what was once possible only in data centers into practical edge computing reality.

Requirements:

• Intel oneAPI Base Toolkit 2025.2.1
• Intel GPU Driver
• PyTorch 2.9 for Intel GPU
• System: Panther Lake, 32GB RAM, 12Xe Arc GPU

For starter code samples, we recommend:

• Unsloth’s fine-tuning notebooks
• Hugging Face TRL’s example notebooks

Tips for Efficient Local Fine-Tuning

1. Strategic Model Selection
   Balance model capabilities against memory constraints. For rapid prototyping, the 1.5B Qwen model allowed for higher batch sizes and faster iterations. For larger models like Llama3-8B, 4-bit quantization is essential to maintain stable GPU memory usage.

2. LoRA/QLoRA Rank Optimization
   Higher rank increases trainable parameters and facilitates faster adaptation to new data. However, it usually requires more GPU memory. In our short-train-time experiments, we found a rank of 64 utilized the Intel Arc GPU’s memory well while allowing the model to learn quickly without overfitting.

   Figure 1: Average GPU memory utilization monitored using xpu-smi over 25 training steps for Llama3-8B-instruct (batch size = 4) and 85 steps for Qwen2.5-5B-instruct (batch size = 10).

3. Tuning GRPO num_generations
   The num_generations parameter (completions sampled per prompt) should scale with dataset complexity:

   • Simple/Binary outputs (e.g. Biomedical QA’s “yes/no/maybe” output): We lowered num_generations to 4 to reduce compute.
   • Complex reasoning (e.g. Math QA’s multiple XML-formatted key-value pairs): We increased num_generations to 10 to increase the probability of obtaining rewardable correct reasoning paths.

4. Higher Learning Rate, Lower Step Count
   In resource-constrained environments, prioritize faster convergence. In the Math QA use case, increasing the learning rate from 5×10^-6 to 5×10^-5 resulted in a 21% increase in Exact-Match accuracy after 30 minutes of training compared to a 15% increase using the smaller value. Combining the higher learning rate with an increased number of generations provided an even bigger boost.

   Figure 2: When combined, the increase in num_generations and LR produced an even bigger boost to accuracy within 30 minutes (more than 2x).

5. Reward Functions & System Prompt
   Good reward signals make every step count when fine-tuning on an AI PC. Use a combination of:

   • Format Rewards: Try to cover areas where the base model seems weak and encourage specific structured outputs via system prompts. With Qwen2.5-1.5B-Instruct, we were able to overcome persistent formatting issues by explicitly including “XML” in the system prompt. In the Biomedical QA use case, we added an extra reward for the end </answer> tag that was consistently missed when evaluating the base model. In Figure 3, you can see that in less than 10 training steps, the model has learned to include the end tag.
   • Correctness Rewards: Use lenient matching (i.e. reward correct answers even if formatting is slightly off) to provide a stronger signal for the model. For math reasoning, parsing and evaluating the last number appearing in the response, in addition to ones inside <answer></answer> tags, was crucial to reaching the target performance. Be sure to consider a variety of different factors in a response such as formatting, correctness, politeness, and reasoning, and assign appropriate reward weights based on what’s most important for your use case.

6. Monitor Key Metrics and Fail Fast
   Use xpu-smi to monitor GPU and memory utilization. If you are not using the full capacity of the GPU, you might be able to increase the batch size or generation count for faster training.

   Also, monitor the training metrics, and if you don't start to see the reward metrics trending upward after ~20 steps, consider stopping early, and adjust your hyperparameters or reward functions.

   Figure 3: Training reward metrics over 25 steps when fine tuning Llama3-8B-Instruct on PubMedQA using GRPO and QLoRA (batch size = 4).

Benefits of Panther Lake

Beyond its technical capabilities, Panther Lake's Intel Arc GPU delivers three key advantages:

Performance That Fits Your Workflow
Our experiments consistently showed noticeable improvements in evaluation accuracy within 30 minutes of training. The ability to iterate quickly using a well-configured laptop, while not revolutionary, is genuinely useful and enables experimentation, adjustment, and results in real-time.

Seamless Ecosystem Integration
Panther Lake's compatibility with the Hugging Face and Unsloth ecosystems means developers can leverage the same advanced fine-tuning techniques used in production AI systems, without sacrificing ease-of-use or disrupting familiar development workflows. You can work with the same tools and techniques, just scaled to fit your laptop’s capabilities.

Accessibility and Privacy
Panther Lake enables AI development that prioritizes data privacy and accessibility. Fine-tuning sensitive datasets, like medical records, proprietary business data, or personal information, can now happen entirely on local hardware, eliminating many concerns inherent in cloud-based training. This also helps democratize access to sophisticated fine-tuning capabilities, removing barriers related to cloud costs or specialized infrastructure requirements.

Conclusion

These experiments show that, by leveraging Unsloth and TRL, engineers can achieve measurable accuracy gains in 30 minutes of local training on a Panther Lake AI PC. We hope the combination of accessible performance, seamless ecosystem integration, and practical tips we’ve shared can help you accomplish some interesting fine-tuning workflows right from your laptop.

Citations

@misc{llama3herd,
  title={The Llama 3 Herd of Models},
  author={Grattafiori, Aaron and 558 other authors},
  year={2024},
  eprint={2407.21783},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2407.21783}}

@misc{qwen2.5,
    title = {Qwen2.5: A Party of Foundation Models},
    url = {https://qwenlm.github.io/blog/qwen2.5/},
    author = {Qwen Team},
    month = {September},
    year = {2024}
}

@article{qwen2,
      title={Qwen2 Technical Report}, 
      author={An Yang and Baosong Yang and Binyuan Hui and Bo Zheng and Bowen Yu and Chang Zhou and Chengpeng Li and Chengyuan Li and Dayiheng Liu and Fei Huang and Guanting Dong and Haoran Wei and Huan Lin and Jialong Tang and Jialin Wang and Jian Yang and Jianhong Tu and Jianwei Zhang and Jianxin Ma and Jin Xu and Jingren Zhou and Jinze Bai and Jinzheng He and Junyang Lin and Kai Dang and Keming Lu and Keqin Chen and Kexin Yang and Mei Li and Mingfeng Xue and Na Ni and Pei Zhang and Peng Wang and Ru Peng and Rui Men and Ruize Gao and Runji Lin and Shijie Wang and Shuai Bai and Sinan Tan and Tianhang Zhu and Tianhao Li and Tianyu Liu and Wenbin Ge and Xiaodong Deng and Xiaohuan Zhou and Xingzhang Ren and Xinyu Zhang and Xipin Wei and Xuancheng Ren and Yang Fan and Yang Yao and Yichang Zhang and Yu Wan and Yunfei Chu and Yuqiong Liu and Zeyu Cui and Zhenru Zhang and Zhihao Fan},
      journal={arXiv preprint arXiv:2407.10671},
      year={2024}
}

@inproceedings{jin2019pubmedqa,
  title={PubMedQA: A Dataset for Biomedical Research Question Answering},
  author={Jin, Qiao and Dhingra, Bhuwan and Liu, Zhengping and Cohen, William and Lu, Xinghua},
  booktitle={Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)},
  pages={2567--2577},
  year={2019}
}

@article{cobbe2021gsm8k,
  title={Training Verifiers to Solve Math Word Problems},
  author={Cobbe, Karl and Kosaraju, Vineet and Bavarian, Mohammad and Chen, Mark and Jun, Heewoo and Kaiser, Lukasz and Plappert, Matthias and Tworek, Jerry and Hilton, Jacob and Nakano, Reiichiro and Hesse, Christopher and Schulman, John},
  journal={arXiv preprint arXiv:2110.14168},
  year={2021}
}

@software{unsloth,
  author = {Daniel Han, Michael Han and Unsloth team},
  title = {Unsloth},
  url = {http://github.com/unslothai/unsloth},
  year = {2023}
}

@misc{vonwerra2022trl,
  author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
  title = {TRL: Transformer Reinforcement Learning},
  year = {2020},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/huggingface/trl}}
}