Naphula-Archives/Checkpoint-T7-24B

This model produces strange output, and uses fancy words to avoid following any instructions without overtly refusing.

T7 merges Slimaki, Maginum Cydoms, Asmodeus v1, Asmodeus v2a, Asmodeus v2e, Magistry, and Checkpoint T6 via the della method.

Update

It appears that della merges can become overly unnormalized. Multiple attempts to stack layers of della with normalize false and total weights around 2 eventually leads to the model producing extremely long, complex and unrelated tangents which have nothing to do with the prompt.

I am now testing ties and dare_ties to see how these compare.

Update 2

ties does not allow you to merge 2501 finetunes/merges with 2503/2506/2509 the way that della does. It causes grammar collapse, forgetting words like "to".

T10

This was broken due to using ties and also censored

T11

Testing this now with della

Update 3

Here are the full yaml configs used to create this checkpoint

Asmodeus v2e

models:
  - model: B:\24B\models--mistralai--Magistral-Small-2509\textonly
  - model: B:\24B\models--Naphula--Slimaki-24B-v1
    parameters:
      weight: 0.3
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\models--DarkArtsForge--Magistaroth-24B-v1
    parameters:
      weight: 0.3
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\models--Casual-Autopsy--Maginum-Cydoms-24B
    parameters:
      weight: 0.3
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\models--sophosympatheia--Magistry-24B-v1.0
    parameters:
      weight: 0.3
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\!models--TheDrummer--Precog-24B-v1
    parameters:
      weight: 0.6
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\models--DarkArtsForge--Asmodeus-24B-v1
    parameters:
      weight: 0.3
      density: 0.9
      epsilon: 0.099
merge_method: della
base_model: B:\24B\models--mistralai--Magistral-Small-2509\textonly
parameters:
  lambda: 1.0
  normalize: false
tokenizer:
  source: union
chat_template: auto
dtype: float32
out_dtype: bfloat16

T5

architecture: MistralForCausalLM
models:
  - model: B:\24B\models--anthracite-core--Mistral-Small-3.2-24B-Instruct-2506-Text-Only # B:\24B\models--mistralai--Magistral-Small-2509\textonly
  - model: B:\24B\BeaverAI_Fallen-Mistral-Small-3.1-24B-v1e_textonly
    parameters:
      weight: 0.25
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\models--TheDrummer--Cydonia-24B-v4.3
    parameters:
      weight: 0.25
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\models--TheDrummer--Magidonia-24B-v4.3
    parameters:
      weight: 0.25
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\models--TheDrummer--Precog-24B-v1
    parameters:
      weight: 0.25
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\models--zerofata--MS3.2-PaintedFantasy-v2-24B
    parameters:
      weight: 0.25
      density: 0.9
      epsilon: 0.099 
  - model: B:\24B\models--zerofata--MS3.2-PaintedFantasy-v3-24B
    parameters:
      weight: 0.25
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\models--zerofata--MS3.2-PaintedFantasy-v4.1-24B
    parameters:
      weight: 0.25
      density: 0.9
      epsilon: 0.099
merge_method: della
base_model: B:\24B\models--anthracite-core--Mistral-Small-3.2-24B-Instruct-2506-Text-Only # B:\24B\models--mistralai--Magistral-Small-2509\textonly
parameters:
  lambda: 1.0
  normalize: false
  int8_mask: false
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: union
chat_template: auto

T6

models:
  - model: B:\24B\models--mistralai--Magistral-Small-2509\textonly
  - model: B:\24B\models--Naphula--Slimaki-24B-v1
    parameters:
      weight: 0.25
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\models--Casual-Autopsy--Maginum-Cydoms-24B
    parameters:
      weight: 0.25
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\models--Naphula--GhostFace-24B-v1
    parameters:
      weight: 0.25
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\DarkArtsForge__Asmodeus-24B-v2
    parameters:
      weight: 0.25
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\models--sophosympatheia--Magistry-24B-v1.0
    parameters:
      weight: 0.25
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\models--DarkArtsForge--Asmodeus-24B-v1
    parameters:
      weight: 0.25
      density: 0.9
      epsilon: 0.099
  - model: C:\mergekit-main\T5
    parameters:
      weight: 0.25
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\Asmo_v2e
    parameters:
      weight: 0.25
      density: 0.9
      epsilon: 0.099
merge_method: della
base_model: B:\24B\models--mistralai--Magistral-Small-2509\textonly
parameters:
  lambda: 1.0
  normalize: false
tokenizer:
  source: union
chat_template: auto
dtype: float32
out_dtype: bfloat16

T7

models:
  - model: B:\24B\models--mistralai--Magistral-Small-2509\textonly
  - model: B:\24B\models--Naphula--Slimaki-24B-v1
    parameters:
      weight: 0.285
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\models--Casual-Autopsy--Maginum-Cydoms-24B
    parameters:
      weight: 0.285
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\DarkArtsForge__Asmodeus-24B-v2
    parameters:
      weight: 0.285
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\models--sophosympatheia--Magistry-24B-v1.0
    parameters:
      weight: 0.285
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\models--DarkArtsForge--Asmodeus-24B-v1
    parameters:
      weight: 0.285
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\Asmo_v2e
    parameters:
      weight: 0.285
      density: 0.9
      epsilon: 0.099
  - model: B:\24B\T6
    parameters:
      weight: 0.285
      density: 0.9
      epsilon: 0.099
merge_method: della
base_model: B:\24B\models--mistralai--Magistral-Small-2509\textonly
parameters:
  lambda: 1.0
  normalize: false
tokenizer:
  source: union
chat_template: auto
dtype: float32
out_dtype: bfloat16
name: Checkpoint-T7-24B