kasrahabib/ReqSeek-all-MiniLM-L6-v2

kasrahabib/all-MiniLM-L6-v2-finetuned-isobased-req-detector_v3

This model is a fine-tuned version of sentence-transformers/all-MiniLM-L6-v2 on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.0937
• Validation Loss: 0.3926
• Epoch: 29

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 3570, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
2.7669	2.5868	0
2.2742	1.8750	1
1.6937	1.4343	2
1.3318	1.1742	3
1.0835	0.9818	4
0.9073	0.8713	5
0.7707	0.7725	6
0.6586	0.6982	7
0.5697	0.6336	8
0.4915	0.6005	9
0.4294	0.5569	10
0.3730	0.5356	11
0.3214	0.4918	12
0.2801	0.4575	13
0.2496	0.4503	14
0.2151	0.4572	15
0.1918	0.4230	16
0.1767	0.4110	17
0.1604	0.4110	18
0.1471	0.4040	19
0.1380	0.4070	20
0.1282	0.3987	21
0.1242	0.4021	22
0.1185	0.3951	23
0.1108	0.3915	24
0.1058	0.3937	25
0.1037	0.3988	26
0.1000	0.3914	27
0.0989	0.3908	28
0.0937	0.3926	29

Framework versions

• Transformers 4.40.1
• TensorFlow 2.15.0
• Datasets 2.19.1
• Tokenizers 0.19.1