distilbert-finetuned-ag-news

This model is a fine-tuned version of distilbert-base-uncased on the AG News dataset. It achieves the following results on the evaluation set:

• Loss: 0.1799
• Accuracy: 0.9470
• Precision: 0.9472
• Recall: 0.9470
• F1: 0.9470

Model description

This is a DistilBERT model fine-tuned on the AG news dataset to determine whether a news article is within any of the four categories: 'World', 'Sports', 'Business' and 'Sci/Tech'. The Trainer API was used to train the model.

Intended uses & limitations

Training and evaluation data

Data is from HuggingFace's datasets package, source: AG news dataset

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	Recall	F1
0.1888	1.0	3750	0.1786	0.9418	0.9422	0.9418	0.9419
0.1289	2.0	7500	0.1741	0.9459	0.9462	0.9459	0.9460
0.0989	3.0	11250	0.1799	0.9470	0.9472	0.9470	0.9470

Framework versions

• Transformers 4.33.0
• Pytorch 2.0.0
• Datasets 2.1.0
• Tokenizers 0.13.3