| # LM-Combiner |
| All the code and model are released [link](https://github.com/wyxstriker/LM-Combiner). Thank you for your patience! |
|
|
| # Model Weight |
| - cbart_large.zip |
| - Weight of Bart baseline model. |
| |
| |
| - lm_combiner.zip |
| - Weight of LM-Combiner for Bart baseline on FCGEC dataset. |
| |
|
|
| # Requirements |
|
|
| The part of the model is implemented using the huggingface framework and the required environment is as follows: |
| - Python |
| - torch |
| - transformers |
| - datasets |
| - tqdm |
|
|
| For the evaluation, we refer to the relevant environment configurations of [ChERRANT](https://github.com/HillZhang1999/MuCGEC/tree/main/scorers/ChERRANT). |
|
|
| # Training Stage |
| ## Preprocessing |
| ### Baseline Model |
| - Firstly, we train a baseline model (Chinese-Bart-large) for LM-Combiner on the FCGEC dataset using the Seq2Seq format. |
| ```bash |
| sh ./script/run_bart_baseline.sh |
| ``` |
| ### Candidate Datasets |
| 1. Candidate Sentence Generation |
| - We use the baseline model to generate candidate sentences for the training and test sets |
| - On tasks where the model fits better (spelling correction, etc.), we recommend using the K-fold cross-inference from the paper to generate candidate sentences separately. |
| ```bash |
| python ./src/predict_bl_tsv.py |
| ``` |
| 2. Golden Labels Merging |
| - We use the ChERRANT tool to fully decouple the error correction task and the rewriting task by merging the correct labels. |
| ```bash |
| python ./scorer_wapper/golden_label_merging.py |
| ``` |
| ## LM-combiner (gpt2) |
| - Subsequently, we train LM-Combiner on the constructed candidate dataset |
| - In particular, we supplement the gpt2 vocab (mainly **double quotes**) to better fit the FCGEC dataset, see ```./pt_model/gpt2-base/vocab.txt``` for details. |
| ```bash |
| sh ./script/run_lm_combiner.py |
| ``` |
|
|
| # Evaluation |
| - We use the official ChERRANT script to evaluate the model on the FCGEC-dev. |
| ```shell |
| sh ./script/compute_score.sh |
| ``` |
| |method|Prec|Rec|F0.5| |
| |-|-|-|-| |
| | bart_baseline|28.88|**38.95**|40.46| |
| |+lm_combiner|**52.15**|37.41|**48.34**| |
| # Citation |
|
|
| If you find this work is useful for your research, please cite our paper: |
|
|
| ``` |
| @inproceedings{wang-etal-2024-lm-combiner, |
| title = "{LM}-Combiner: A Contextual Rewriting Model for {C}hinese Grammatical Error Correction", |
| author = "Wang, Yixuan and |
| Wang, Baoxin and |
| Liu, Yijun and |
| Wu, Dayong and |
| Che, Wanxiang", |
| editor = "Calzolari, Nicoletta and |
| Kan, Min-Yen and |
| Hoste, Veronique and |
| Lenci, Alessandro and |
| Sakti, Sakriani and |
| Xue, Nianwen", |
| booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)", |
| month = may, |
| year = "2024", |
| address = "Torino, Italia", |
| publisher = "ELRA and ICCL", |
| url = "https://aclanthology.org/2024.lrec-main.934", |
| pages = "10675--10685", |
| } |
| ``` |
