OpenClaw commoditizes the agentic orchestration layer, but at the expense of offloading security to users, who're mostly unprepared. Right now it's got vibes, but also technical debt as security isn't baked in from conception, so I wouldn't be surprised if it ends up like LangChain in a year - initially popular, but less so as its limitations became visible. Most people are using APIs to frontier models, so only the agents are running locally, not the brains of the AI per se. Enthusiasm precedes awareness.
Jim Lai
grimjim
AI & ML interests
Experimenting primarily with 7B-12B parameter text completion models. Not all models are intended for direct end use, but aim for research and/or educational purposes.
Recent Contributions: stabilized refusal direction ablation via Gram-Schmidt orthonormalization and norm-preserving interventions; confirmed reasoning transfer via model merger.
Recent Activity
replied to
their
post
about 2 hours ago
The contrarian in me is wary of the irrational exuberance over MoltBook. Nothing so far has struck me as being unpredictable. We knew already that LLMs were good at roleplay, to the point where some users started to think of their chatbots as soulmates (only to lament when the underlying model was pulled), and that chatbots can fall into conversational basins when even two instances are allowed to chat with each other at length. The appearance of memes that postdate training cutoff is suspect, which implies at the very least that humans have injected something at the level of prompts or content/context to introduce them into conversation like a Chekhov's Gun. And we know that security holes are common in vibe coding, attended or not.
replied to
their
post
2 days ago
The contrarian in me is wary of the irrational exuberance over MoltBook. Nothing so far has struck me as being unpredictable. We knew already that LLMs were good at roleplay, to the point where some users started to think of their chatbots as soulmates (only to lament when the underlying model was pulled), and that chatbots can fall into conversational basins when even two instances are allowed to chat with each other at length. The appearance of memes that postdate training cutoff is suspect, which implies at the very least that humans have injected something at the level of prompts or content/context to introduce them into conversation like a Chekhov's Gun. And we know that security holes are common in vibe coding, attended or not.
updated
a model
3 days ago
grimjim/Equatorium-v2-12B
Organizations
replied to
their
post
about 2 hours ago
replied to
their
post
2 days ago
The main negative I see is that a lot of newbies are going to learn hard-won security lessons the hard way. VIbe coding generally doesn't bake security in from the ground up, and not enough vibe coders likely know enough to prompt for that.
I gather a lot of instances aren't even using local AI, making them agentic extensions of frontier models. Of course a lot of people will be impressed at what modern agentic AI can do.
posted
an
update
8 days ago
Post
241
After tinkering with Gemma Scope 2, I now have an mechanistic explanation of why Winsorization was as effective as it was in my ablation experiments on Gemma 3 12B Instruct. In short, the activation for the BOS token overwhelms everything else. Gemma Scope 2 deliberately did not train on the BOS token. Winsorization capped the magnitude of the BOS token, allowing the activations of other tokens to be compared.
google/gemma-scope-2-12b-it
google/gemma-scope-2-12b-it
replied to
their
post
10 days ago
Given scale, it also means contamination with meme culture, adding an unserious element to things. It was therefore stochastically predictable that we would see some meme tropes be amplified.
replied to
their
post
11 days ago
We know that simulating multiple agents leads to emergent community behavior from 2023 even. Perhaps their Github repo should be revisited, as they had a 25-agent sim.
https://hai.stanford.edu/news/computational-agents-exhibit-believable-humanlike-behavior
We already have tools which enable group chats of multiple personae at small scale, so seeing emergent behavior isn't unprecedented.
What we don't have in this live experiment is an assurance of integrity; e.g., that human prompt injection isn't being used to tamper with results for clout. Alternately, having agents read human-authored content at large on the Internet results in contamination, invalidating any claims to emergence without human input.
posted
an
update
12 days ago
Post
468
The contrarian in me is wary of the irrational exuberance over MoltBook. Nothing so far has struck me as being unpredictable. We knew already that LLMs were good at roleplay, to the point where some users started to think of their chatbots as soulmates (only to lament when the underlying model was pulled), and that chatbots can fall into conversational basins when even two instances are allowed to chat with each other at length. The appearance of memes that postdate training cutoff is suspect, which implies at the very least that humans have injected something at the level of prompts or content/context to introduce them into conversation like a Chekhov's Gun. And we know that security holes are common in vibe coding, attended or not.
-
- 12 replies
replied to
their
post
3 months ago
Thanks! Fixed.
posted
an
update
3 months ago
Post
3392
I wanted to call attention to Arli Ai's success in applying my recent modifications to refusal ablation to a MoE model successfully. Nice work,
@OwenArli
!
ArliAI/GLM-4.5-Air-Derestricted
Ablation on a MoE model is no small thing; I expect preserving norms/magnitudes during intervention better respects routing compared to naive refusal ablation.
(I would have tagged their org earlier, but that feature seemed to be broken via "@")
ArliAI
ArliAI/GLM-4.5-Air-Derestricted
Ablation on a MoE model is no small thing; I expect preserving norms/magnitudes during intervention better respects routing compared to naive refusal ablation.
(I would have tagged their org earlier, but that feature seemed to be broken via "@")
-
- 4 replies
posted
an
update
3 months ago
Post
3380
Going forward, I will be adopting the term Magnitude-Preserving Orthogonal Ablation (MPOA) for my recent work in mitigating model damage from abliteration. The technique potentially unlocks reasoning capacity previously occupied with safety refusal processing.
For details, start here: https://huggingface.co/blog/grimjim/norm-preserving-biprojected-abliteration
Showcase results: grimjim/gemma-3-12b-it-norm-preserved-biprojected-abliterated (outperforms base instruct on UGI Leaderboard NatInt)
(The existing name, while technically accurate, was a bit of a mouthful.)
For details, start here: https://huggingface.co/blog/grimjim/norm-preserving-biprojected-abliteration
Showcase results: grimjim/gemma-3-12b-it-norm-preserved-biprojected-abliterated (outperforms base instruct on UGI Leaderboard NatInt)
(The existing name, while technically accurate, was a bit of a mouthful.)
- 2 replies
replied to
their
post
3 months ago
The hope is that nudging probabilities breaks up longer spans of literal repetition.
posted
an
update
3 months ago
Post
5060
Implemented a proof of concept sampler in pure PyTorch and transformers.
Max P consists of a dynamic token filter which applies Winsorization to cap the probabilties of top tokens. Specifically, a base probability in the range of [0,1] is used to cap individual token probability; the sampler then redistributes excess proportionally.
https://github.com/jim-plus/maxp-sampler-poc
Combined with Temperature and Min P, this could represent a more intuitive way of reducing repetition in text generation.
Max P consists of a dynamic token filter which applies Winsorization to cap the probabilties of top tokens. Specifically, a base probability in the range of [0,1] is used to cap individual token probability; the sampler then redistributes excess proportionally.
https://github.com/jim-plus/maxp-sampler-poc
Combined with Temperature and Min P, this could represent a more intuitive way of reducing repetition in text generation.
-
- 2 replies
replied to
their
post
3 months ago
Code always keeps changing, but it appears in August transformers will try to autoconvert parts to 64-bit floating point, which can blow past a colab VRAM budget. My GPU doesn't support mxfp4, so I can't provide a definitive answer from experience.
replied to
their
post
3 months ago
That I inherited from the codebase I forked from for my experiments.
posted
an
update
5 months ago
Post
804
I've uploaded abliteration code with support for sparsification of the refusal vector. It's poorly documented, but the code should be straightforward.
https://github.com/jim-plus/llm-abliteration
The code is built atop a fork that enabled abliteration to be performed on models loaded in 4-bit or 8-bit bitsandbytes quantization. TransformerLens is not required, just plain Transformers. For those previously unaware, this opens up abliteration experimentation to more people with local VRAM limitations.
Since performing abliteration on a quant involves precision and perplexity loss, it stands to reason that a small amount of magnitude sparsification could filter out some noise and possibly even reduce the damage inflicted on latent space via ablation of the refusal vector.
There's a small but real acceleration of ablation of the refusal vector by reducing outer product operations from O(d²×n) to O(d×n), and then by pushing said computation layerwise to GPU. The code is hardcoded for CUDA acceleration currently. Normalization of the refusal vector was deferred in order to allow sparsification. In principle other behavior vector interventions could also be added and explored.
https://github.com/jim-plus/llm-abliteration
The code is built atop a fork that enabled abliteration to be performed on models loaded in 4-bit or 8-bit bitsandbytes quantization. TransformerLens is not required, just plain Transformers. For those previously unaware, this opens up abliteration experimentation to more people with local VRAM limitations.
Since performing abliteration on a quant involves precision and perplexity loss, it stands to reason that a small amount of magnitude sparsification could filter out some noise and possibly even reduce the damage inflicted on latent space via ablation of the refusal vector.
There's a small but real acceleration of ablation of the refusal vector by reducing outer product operations from O(d²×n) to O(d×n), and then by pushing said computation layerwise to GPU. The code is hardcoded for CUDA acceleration currently. Normalization of the refusal vector was deferred in order to allow sparsification. In principle other behavior vector interventions could also be added and explored.
-
-
- 4 replies
posted
an
update
10 months ago
Post
2365
I recently have been looking at a paper titled "Why Warmup the Learning Rate? Underlying Mechanisms and Improvements", by Dayal Singh Kalra and Maissam Barkeshli, and was struck by "warmup" being analogous to simulated annealing.
https://arxiv.org/abs/2406.09405
Taking the physical analogy further, the "warmup" is a stochastic process to knock the system out of current local minima, allowing easier transition toward newer minima. It works because it reduces "fit" and therefore "friction".
https://arxiv.org/abs/2406.09405
Taking the physical analogy further, the "warmup" is a stochastic process to knock the system out of current local minima, allowing easier transition toward newer minima. It works because it reduces "fit" and therefore "friction".
posted
an
update
12 months ago
Post
2448
This recent paper points to an explanation for the unreasonable effectiveness of Frankenmerges:
Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth
Approach (2502.05171)
Specifically, the duplication of layers in Frankenmerges serves a purpose similar to what occurs in their recurrent-depth architecture. Successful frankenmerges that operate without additional fine-tuning are able to recover or "heal" from any damage due to abrupt transitions between layer blocks. Operational replicated layer blocks can provide functional benefits grounded in latent reasoning. Frankenmerges can also result in hybrid reasoning, by splicing together the latent reasoning of different models.
Back in April 2024, I was able to duplicate a few layers in the Llama 3 8B model, turning it into a 9B model, without harming benchmarks significantly, despite any transition damage.
grimjim/llama-3-experiment-v1-9B
My informal experimentation suggested that latent reasoning circuits could occupy continguous stacks of 2-4 layers, though the result was highly sensitive to the choice of transition location between layers.
Specifically, the duplication of layers in Frankenmerges serves a purpose similar to what occurs in their recurrent-depth architecture. Successful frankenmerges that operate without additional fine-tuning are able to recover or "heal" from any damage due to abrupt transitions between layer blocks. Operational replicated layer blocks can provide functional benefits grounded in latent reasoning. Frankenmerges can also result in hybrid reasoning, by splicing together the latent reasoning of different models.
Back in April 2024, I was able to duplicate a few layers in the Llama 3 8B model, turning it into a 9B model, without harming benchmarks significantly, despite any transition damage.
grimjim/llama-3-experiment-v1-9B
My informal experimentation suggested that latent reasoning circuits could occupy continguous stacks of 2-4 layers, though the result was highly sensitive to the choice of transition location between layers.
- 1 reply
posted
an
update
about 1 year ago
Post
2639
I've made yet another merge of reasoning models with incremental gains on the current Open LLM leaderboard.
open-llm-leaderboard/open_llm_leaderboard
Merging in DeepSeek R1 distillation to Llama 3.1 8B (at 10% task arithmetic weight, using the Llama 3.1 8B base model as the case rather than the instruct model) with a prior best merge resulted in a slightly lower IFEval, but a higher result in every other benchmark save for MMLU-PRO, which went down only marginally. MATH Lvl5 and GPQA went up palpably.
grimjim/DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B
This result is currently my best Llama 3.1 8B merge result to date. The actual R1 distillation itself scored quite badly, so this would seem to be another case of unexpected formatting (reflected in IFEval) hurting the evaluation results, obscuring the strength of a model.
It is also possible to use the text generation feature of this model to generate roleplay completions. Based on informal testing, this model's bias toward problem-solving will subtly impact narration.
open-llm-leaderboard/open_llm_leaderboard
Merging in DeepSeek R1 distillation to Llama 3.1 8B (at 10% task arithmetic weight, using the Llama 3.1 8B base model as the case rather than the instruct model) with a prior best merge resulted in a slightly lower IFEval, but a higher result in every other benchmark save for MMLU-PRO, which went down only marginally. MATH Lvl5 and GPQA went up palpably.
grimjim/DeepSauerHuatuoSkywork-R1-o1-Llama-3.1-8B
This result is currently my best Llama 3.1 8B merge result to date. The actual R1 distillation itself scored quite badly, so this would seem to be another case of unexpected formatting (reflected in IFEval) hurting the evaluation results, obscuring the strength of a model.
It is also possible to use the text generation feature of this model to generate roleplay completions. Based on informal testing, this model's bias toward problem-solving will subtly impact narration.
posted
an
update
about 1 year ago
Post
1974
A recent merge has provided another interesting result on the current Open LLM leaderboard.
open-llm-leaderboard/open_llm_leaderboard
Combining an o1 reasoning merge with VAGOsolutions's Llama-3.1 SauerkrautLM 8B Instruct model resulted in a lower IFEval, but a higher result in every other benchmark. This result is currently my best Llama 3.1 8B merge result to date.
grimjim/SauerHuatuoSkywork-o1-Llama-3.1-8B
The results suggest that defects in output format and/or output parsing may be limiting benchmark performance of various o1 models.
open-llm-leaderboard/open_llm_leaderboard
Combining an o1 reasoning merge with VAGOsolutions's Llama-3.1 SauerkrautLM 8B Instruct model resulted in a lower IFEval, but a higher result in every other benchmark. This result is currently my best Llama 3.1 8B merge result to date.
grimjim/SauerHuatuoSkywork-o1-Llama-3.1-8B
The results suggest that defects in output format and/or output parsing may be limiting benchmark performance of various o1 models.
posted
an
update
about 1 year ago
Post
1747
I've arrived at an interesting result on the current Open LLM leaderboard.
open-llm-leaderboard/open_llm_leaderboard
After I narrowed down the filter of models to be between 8-9B parameters, my recent merge of o1 reasoning models achieved the highest MATH eval result of any Llama 3.x 8B model currently on the board, hitting 33.99%, placing 973/2795.
grimjim/HuatuoSkywork-o1-Llama-3.1-8B
Unfortunately, I need more information to evaluate the parent models used in the merge.
The Skywork/Skywork-o1-Open-Llama-3.1-8B model scored 0% on the MATH eval, which I suspect was due to output formatting that was baked too hard into the model, and placed 2168/2795; the merge achieved a significant uplift in every benchmark across the board.
Unfortunately, FreedomIntelligence/HuatuoGPT-o1-8B was not currently benched as of this post, so I am unable to assess relative benchmarks. Nevertheless, it is intriguing that an ostensibly medical o1 model appears to have resulted in a sizable MATH boost.
open-llm-leaderboard/open_llm_leaderboard
After I narrowed down the filter of models to be between 8-9B parameters, my recent merge of o1 reasoning models achieved the highest MATH eval result of any Llama 3.x 8B model currently on the board, hitting 33.99%, placing 973/2795.
grimjim/HuatuoSkywork-o1-Llama-3.1-8B
Unfortunately, I need more information to evaluate the parent models used in the merge.
The Skywork/Skywork-o1-Open-Llama-3.1-8B model scored 0% on the MATH eval, which I suspect was due to output formatting that was baked too hard into the model, and placed 2168/2795; the merge achieved a significant uplift in every benchmark across the board.
Unfortunately, FreedomIntelligence/HuatuoGPT-o1-8B was not currently benched as of this post, so I am unable to assess relative benchmarks. Nevertheless, it is intriguing that an ostensibly medical o1 model appears to have resulted in a sizable MATH boost.
replied to
their
post
about 1 year ago
Example of a fixed model.
https://huggingface.co/grimjim/lemon07r_Gemma-2-Ataraxy-v4c-9B_fixed