inclusionAI
/

LLaDA2.1-mini

Text Generation

text_generation

Model card Files Files and versions

fixes for transforemrs v5

#4

by kashif HF Staff - opened 5 days ago

base: refs/heads/main

←

from: refs/pr/4

Discussion Files changed

The "default" rope_type was removed in recent transformers versions. Using "linear" (with no scaling factor) is functionally equivalent.

fix: use linear rope_type instead of removed default for transformers compat52747ba6

fixes https://huggingface.co/inclusionAI/LLaDA2.1-mini/discussions/3

fix: remap default rope_type to linear for newer transformers compat0435edb8

fix: add default factor=1.0 for linear rope compat with newer transformers220496f1

fix: align RotaryEmbedding with Qwen2Moe pattern for transformers compata0f450fc

fix: call super()._init_weights() to match Qwen2Moe convention1f0d143f

fix: align _init_weights with Qwen2Moe using nn.init API92a5aa65

Use create_bidirectional_mask for backend-agnostic attention mask handling (SDPA, FA2, flex)90c63c74

kashif changed pull request title from fix: use linear rope_type instead of removed default for transformers compat to fixes for transforemrs v5 2 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Ready to merge

This branch is ready to get merged automatically.

· Sign up or log in to comment