Spaces:
Sleeping
Sleeping
| title: RAFT-QA | |
| sdk: gradio | |
| emoji: 💻 | |
| colorFrom: purple | |
| colorTo: gray | |
| pinned: true | |
| thumbnail: >- | |
| https://cdn-uploads.huggingface.co/production/uploads/67c714e90b99a2332e310979/TrWTI_vofjfGgx4PILFY3.jpeg | |
| short_description: Retrieval-Augmented Fine-Tuning for Question Answering | |
| sdk_version: 5.34.0 | |
| language: | |
| - en | |
| tags: | |
| - retrieval-augmented-learning | |
| - question-answering | |
| - fine-tuning | |
| - transformers | |
| - llm | |
| license: mit | |
| datasets: | |
| - pubmedqa | |
| - hotpotqa | |
| - gorilla | |
| library_name: transformers | |
| model-index: | |
| - name: RAFT-QA | |
| results: | |
| - task: | |
| type: question-answering | |
| name: Open-Domain Question Answering | |
| dataset: | |
| name: PubMedQA | |
| type: question-answering | |
| metrics: | |
| - name: Exact Match (EM) | |
| type: exact_match | |
| value: 79.3 | |
| - name: F1 Score | |
| type: f1 | |
| value: 87.1 | |
| # RAFT-QA: Retrieval-Augmented Fine-Tuning for Question Answering | |
| ## Model Overview | |
| RAFT-QA is a sophisticated **retrieval-augmented** question-answering model designed to significantly enhance answer accuracy through the integration of **retrieved documents** during the fine-tuning process. By utilizing retrieval-enhanced training, it advances traditional fine-tuning techniques. | |
| ## Model Details | |
| - **Base Model Options:** `mistral-7b`, `falcon-40b-instruct`, or other leading large language models (LLMs) | |
| - **Fine-Tuning Technique:** RAFT (Retrieval-Augmented Fine-Tuning) | |
| - **Retrieval Strategy:** FAISS-based document embedding retrieval | |
| - **Training Datasets Included:** PubMedQA, HotpotQA, Gorilla | |
| ## How It Works | |
| 1. **Retrieve Relevant Documents:** FAISS efficiently retrieves the most pertinent documents in response to a query. | |
| 2. **Augment Input with Retrieved Context:** Incorporates the retrieved documents into the input data. | |
| 3. **Fine-Tune the Model:** The model learns to effectively weigh the retrieved context to produce improved answers. | |
| ## Performance Comparison | |
| | Model | Exact Match (EM) | F1 Score | | |
| |------------------------|------------------|----------| | |
| | GPT-3.5 (baseline) | 74.8 | 84.5 | | |
| | Standard Fine-Tuning | 76.2 | 85.6 | | |
| | **RAFT-QA (ours)** | **79.3** | **87.1** | | |
| ## Usage | |
| To load the model using the `transformers` library: | |
| ```python | |
| from transformers import AutoModelForQuestionAnswering, AutoTokenizer | |
| model_name = "your-hf-username/raft-qa" | |
| tokenizer = AutoTokenizer.from_pretrained(model_name) | |
| model = AutoModelForQuestionAnswering.from_pretrained(model_name) | |
| ``` | |
| ## Limitations | |
| - The model's performance is contingent on the quality of the retrieved documents. | |
| - For optimal results, domain-specific tuning may be necessary. | |
| ## Citation | |
| If you utilize this model in your work, please cite it as follows: | |
| ``` | |
| @article{raft2025, | |
| title={Retrieval-Augmented Fine-Tuning (RAFT) for Enhanced Question Answering}, | |
| author={Your Name et al.}, | |
| journal={ArXiv}, | |
| year={2025} | |
| } | |
| ``` | |
| ## License | |
| This model is released under the Apache 2.0 License. | |
| --- | |
| This version provides clarity and conciseness, ensuring all sections are clear and correctly formatted according to the Hugging Face repository standards. Make sure the dataset type (`question-answering`) matches your intended use case. |