README.md

metadata

library_name: transformers
base_model: AlexeySorokin/ossbert-onc-unlab-bs64-5epochs
tags:
  - generated_from_trainer
metrics:
  - accuracy
model-index:
  - name: trainer_output
    results: []

trainer_output

This model is a fine-tuned version of AlexeySorokin/ossbert-onc-unlab-bs64-5epochs on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4048
• Accuracy: 95.0828
• Sentence accuracy: 57.6147

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Sentence accuracy
1.1201	0.9158	500	0.4242	91.5286	40.9174
0.3523	1.8315	1000	0.3102	92.9717	48.6239
0.2212	2.7473	1500	0.2753	93.8402	52.6606
0.1566	3.6630	2000	0.2708	93.9872	52.1101
0.1149	4.5788	2500	0.2717	94.4148	52.4771
0.0859	5.4945	3000	0.2636	95.0561	57.7982
0.0632	6.4103	3500	0.2898	94.8691	56.8807
0.0471	7.3260	4000	0.2927	94.9759	57.2477
0.0351	8.2418	4500	0.3040	94.8824	57.2477
0.0267	9.1575	5000	0.3026	95.0160	57.9817
0.0206	10.0733	5500	0.3088	95.2165	58.8991
0.0158	10.9890	6000	0.3278	94.8824	57.7982
0.0118	11.9048	6500	0.3241	95.1229	58.7156
0.0095	12.8205	7000	0.3374	95.2699	58.3486
0.0088	13.7363	7500	0.3647	94.8557	56.5138
0.0076	14.6520	8000	0.3639	94.9225	57.0642
0.0077	15.5678	8500	0.3574	95.2432	58.8991
0.006	16.4835	9000	0.3778	95.1630	57.6147
0.0045	17.3993	9500	0.3738	95.3634	59.0826
0.0049	18.3150	10000	0.3824	95.1897	57.4312
0.0029	19.2308	10500	0.3893	94.9626	57.7982
0.0033	20.1465	11000	0.4039	95.1096	58.1651
0.0038	21.0623	11500	0.4056	95.1096	58.7156
0.0032	21.9780	12000	0.4048	95.0828	57.6147

Framework versions

• Transformers 4.57.3
• Pytorch 2.10.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2