neoyipeng/ModernFinBERT-base

ModernFinBERT: Financial Sentiment Analysis

ModernFinBERT is a financial sentiment analysis model based on the ModernBERT architecture (149M parameters). It classifies financial text into three sentiment categories: NEGATIVE, NEUTRAL, and POSITIVE.

Key Features

• Built on ModernBERT-base with rotary positional embeddings, Flash Attention, and GeGLU activations
• Full fine-tuning on a diverse financial corpus spanning earnings calls, press releases, financial tweets, and analyst communications
• Augmented with DataBoost (targeted paraphrases of difficult examples via Verbalized Sampling)
• Fast inference: suitable for high-volume production use

Usage

from transformers import pipeline

classifier = pipeline("text-classification", model="neoyipeng/ModernFinBERT-base")

result = classifier("The company reported strong quarterly earnings, beating analyst expectations.")
print(result)
# [{'label': 'POSITIVE', 'score': 0.95}]

Batch inference

texts = [
    "Revenue declined 15% year-over-year due to market headwinds.",
    "The board approved a new share buyback program.",
    "The company maintained its quarterly dividend at $0.50 per share.",
]

results = classifier(texts)
for text, result in zip(texts, results):
    print(f"{result['label']} ({result['score']:.2f}): {text[:60]}...")

Training Data

The model was trained on the complete neoyipeng/financial_reasoning_aggregated dataset including FinancialPhraseBank, augmented with 410 DataBoost samples generated via Verbalized Sampling.

Source	Domain	Samples
Earnings calls (narrative)	Corporate transcripts	462
Press releases / news	Financial news	1,557
Earnings calls (Q&A)	Analyst Q&A	2,440
Financial tweets	Social media	4,184
FinancialPhraseBank	Press releases	4,846
DataBoost augmentation	Synthetic paraphrases	410
Total		~13,900

Evaluation

5-Fold Cross-Validation (Full Dataset)

Metric	Score
Accuracy	81.41% ± 1.09%
Macro F1	77.66% ± 0.79%

Held-Out Evaluation (FPB excluded from training)

Results from the research paper using LoRA fine-tuning on non-FPB data only:

Evaluation Set	Accuracy	Macro F1
FPB 50agree	82.56%	80.52%
FPB allAgree	95.14%	94.17%

Training Configuration

• Architecture: ModernBERT-base (149M parameters, all unfrozen)
• Learning rate: 2e-5
• Weight decay: 0.01
• Batch size: 16 effective (8 per device, gradient accumulation 2)
• Epochs: 10 with early stopping
• Precision: FP16 mixed precision
• Hardware: NVIDIA Tesla T4 (16GB)

Limitations

• English only: Trained and evaluated exclusively on English financial text.
• Three-class taxonomy: Classifies as NEGATIVE/NEUTRAL/POSITIVE only. Does not provide fine-grained sentiment or aspect-level analysis.
• Domain bias: Training data includes 68% Canadian mining press releases in the news subset, which may affect performance on other financial sub-domains.
• Not domain-pre-trained: Unlike FinBERT variants, this model does not include continued MLM pre-training on financial text. It is fine-tuned directly from the general-purpose ModernBERT-base checkpoint.

Citation

@article{neo2026modernfinbert,
  title={ModernFinBERT: A Systematic Study of ModernBERT for Financial Sentiment Analysis},
  author={Neo, Yipeng},
  journal={arXiv preprint},
  year={2026}
}

Paper

For full experimental details, evaluation protocols, and analysis, see the ModernFinBERT paper.