Do you not just need a memory layer to share that context across, no matter what medium is used for the memory layer?
Daniel Leong
daniel-ltw
AI & ML interests
None yet
Recent Activity
upvoted an article 20 days ago
The Future of the Global Open-Source AI Ecosystem: From DeepSeek to AI+ upvoted an article about 1 month ago
One Year Since the “DeepSeek Moment” Organizations
None yet
upvoted an article 20 days ago
Article
The Future of the Global Open-Source AI Ecosystem: From DeepSeek to AI+
•
51
upvoted an article about 1 month ago
Article
One Year Since the “DeepSeek Moment”
•
60
reacted to melvindave's post with 🚀 3 months ago
Post
2585
Currently having a blast learning the transformers library.
I noticed that model cards usually have Transformers code as usage examples.
So I tried to figure out how to load a model just using the transformers library without using ollama, lmstudio, or llamacpp.
Learned how to install dependencies required to make it work like pytorch and CUDA. I also used Conda for python environment dependencies.
Once I got the model loaded and sample inference working, I made an API to serve it.
I know it's very basic stuff for machine learning experts here in HF but I'm completely new to this so I'm happy to get it working!
Model used: Qwen/Qwen3-VL-8B-Instruct
GPU: NVIDIA GeForce RTX 3090
Here's the result of my experimentation
I noticed that model cards usually have Transformers code as usage examples.
So I tried to figure out how to load a model just using the transformers library without using ollama, lmstudio, or llamacpp.
Learned how to install dependencies required to make it work like pytorch and CUDA. I also used Conda for python environment dependencies.
Once I got the model loaded and sample inference working, I made an API to serve it.
I know it's very basic stuff for machine learning experts here in HF but I'm completely new to this so I'm happy to get it working!
Model used: Qwen/Qwen3-VL-8B-Instruct
GPU: NVIDIA GeForce RTX 3090
Here's the result of my experimentation
- 6 replies
liked a Space 4 months ago
reacted to abidlabs's post with 👍 4 months ago
Post
9951
Why I think local, open-source models will eventually win.
The most useful AI applications are moving toward multi-turn agentic behavior: systems that take hundreds or even thousands of iterative steps to complete a task, e.g. Claude Code, computer-control agents that click, type, and test repeatedly.
In these cases, the power of the model is not how smart it is per token, but in how quickly it can interact with its environment and tools across many steps. In that regime, model quality becomes secondary to latency.
An open-source model that can call tools quickly, check that the right thing was clicked, or verify that a code change actually passes tests can easily outperform a slightly “smarter” closed model that has to make remote API calls for every move.
Eventually, the balance tips: it becomes impractical for an agent to rely on remote inference for every micro-action. Just as no one would tolerate a keyboard that required a network request per keystroke, users won’t accept agent workflows bottlenecked by latency. All devices will ship with local, open-source models that are “good enough” and the expectation will shift toward everything running locally. It’ll happen sooner than most people think.
The most useful AI applications are moving toward multi-turn agentic behavior: systems that take hundreds or even thousands of iterative steps to complete a task, e.g. Claude Code, computer-control agents that click, type, and test repeatedly.
In these cases, the power of the model is not how smart it is per token, but in how quickly it can interact with its environment and tools across many steps. In that regime, model quality becomes secondary to latency.
An open-source model that can call tools quickly, check that the right thing was clicked, or verify that a code change actually passes tests can easily outperform a slightly “smarter” closed model that has to make remote API calls for every move.
Eventually, the balance tips: it becomes impractical for an agent to rely on remote inference for every micro-action. Just as no one would tolerate a keyboard that required a network request per keystroke, users won’t accept agent workflows bottlenecked by latency. All devices will ship with local, open-source models that are “good enough” and the expectation will shift toward everything running locally. It’ll happen sooner than most people think.
- 8 replies
reacted to prithivMLmods's post with ❤️ 5 months ago
Post
2813
Have built the new Image Studio with the Gemini Image Gen models for the following multiple tasks:
⭐ Gemini-Image-Studio: prithivMLmods/Gemini-Image-Studio (Latest)
🤞 Old-App: prithivMLmods/Nano-Banana-AIO
🥊 GitHub: https://github.com/prithivsakthiur/gemini-image-studio-hf
To proceed, you need to add your Gemini API key. Your API key is stored only for the duration of your session and will be lost when you reload or exit the page. It will not be shared or exposed anywhere.
imagen-4.0-fast-generate-001 model for Image Generation (Text-to-Image) and Multi-Image Editing (Image-to-Image), and Draw-to-Image powered by gemini-2.5-flash-image (aka Nano Banana).⭐ Gemini-Image-Studio: prithivMLmods/Gemini-Image-Studio (Latest)
🤞 Old-App: prithivMLmods/Nano-Banana-AIO
🥊 GitHub: https://github.com/prithivsakthiur/gemini-image-studio-hf
To proceed, you need to add your Gemini API key. Your API key is stored only for the duration of your session and will be lost when you reload or exit the page. It will not be shared or exposed anywhere.
reacted to mitkox's post with 🚀 7 months ago
Post
2123
I run Qwen3-Coder 480B locally on my Z8, with a 1-million token context window. It’s the equivalent of parallel-parking a Nimitz-class carrier in a kiddie pool. Thanks to whatever dark pact the llama.cpp, CUDA, and kernel folks signed, hybrid inferencing + VRAM↔RAM offload let me stream the model’s synapses across Xeon, RAM, and four lonely A6000s without summoning either the OOM killer or a small house fire.
reacted to jasoncorkill's post with 🔥 8 months ago
Post
3284
"Why did the bee get married?"
"Because he found his honey!"
This was the "funniest" joke out of 10'000 jokes we generated with LLMs. With 68% of respondents rating it as "funny".
Original jokes are particularly hard for LLMs, as jokes are very nuanced and a lot of context is needed to understand if something is "funny". Something that can only reliably be measured using humans.
LLMs are not equally good at generating jokes in every language. Generated English jokes turned out to be way funnier than the Japanese ones. 46% of English-speaking voters on average found the generated joke funny. The same statistic for other languages:
Vietnamese: 44%
Portuguese: 40%
Arabic: 37%
Japanese: 28%
There is not much variance in generation quality among models for any fixed language. But still Claude Sonnet 4 slightly outperforms others in Vietnamese, Arabic and Japanese and Gemini 2.5 Flash in Portuguese and English
We have release the 1 Million (!) native speaker ratings and the 10'000 jokes as a dataset for anyone to use:
Rapidata/multilingual-llm-jokes-4o-claude-gemini
"Because he found his honey!"
This was the "funniest" joke out of 10'000 jokes we generated with LLMs. With 68% of respondents rating it as "funny".
Original jokes are particularly hard for LLMs, as jokes are very nuanced and a lot of context is needed to understand if something is "funny". Something that can only reliably be measured using humans.
LLMs are not equally good at generating jokes in every language. Generated English jokes turned out to be way funnier than the Japanese ones. 46% of English-speaking voters on average found the generated joke funny. The same statistic for other languages:
Vietnamese: 44%
Portuguese: 40%
Arabic: 37%
Japanese: 28%
There is not much variance in generation quality among models for any fixed language. But still Claude Sonnet 4 slightly outperforms others in Vietnamese, Arabic and Japanese and Gemini 2.5 Flash in Portuguese and English
We have release the 1 Million (!) native speaker ratings and the 10'000 jokes as a dataset for anyone to use:
Rapidata/multilingual-llm-jokes-4o-claude-gemini
- 7 replies
upvoted an article 8 months ago
Article
AI is Just Math: Building Neural Networks on Your Calculator
•
3
reacted to MJannik's post with 🤝 9 months ago
Post
1641
Hi everyone, we’ve got big news! Starting today, all Langfuse product features are available as free OSS (MIT license).
You can now upgrade your self-hosted Langfuse to access features like:
- Managed LLM-as-a-Judge evaluations
- Annotation queues
- Prompt experiments
- LLM playground
We’re incredibly grateful for the support of this amazing community and can’t wait to hear your feedback on the new features!
More on this change here: https://langfuse.com/blog/2025-06-04-open-sourcing-langfuse-product
You can now upgrade your self-hosted Langfuse to access features like:
- Managed LLM-as-a-Judge evaluations
- Annotation queues
- Prompt experiments
- LLM playground
We’re incredibly grateful for the support of this amazing community and can’t wait to hear your feedback on the new features!
More on this change here: https://langfuse.com/blog/2025-06-04-open-sourcing-langfuse-product
reacted to yeonseok-zeticai's post with 🚀🔥 9 months ago
Post
2066
🚀 NEW DROP: run your own on-device LLM—in minutes, on any phone
Today we’re open-sourcing everything you need to put Qwen3-0.6B straight into a production-ready mobile app:
🎥 Watch Qwen3-0.6B chat in real time on any smartphones!
📊 TPS benchmarks – slides comparing token-per-second across heterogeneous mobile devices
💻 Plug-and-play source – Just Copy & Run the source to your project for Android (Kotlin & Java) and iOS (Swift).
🤞 Cross-platform, one pipeline – ZETIC.MLange auto-tunes kernels for every different devices, we’ve tested.
👨💻 Ready for production – swap in your own model, re-benchmark with one command, publish.
Get started
Just Sign-up and check the playground project, QWEN-0.6B
- https://mlange.zetic.ai/p/zetic-example/Qwen3-0.6B
We built this to show that cloud-free LLMs are ready today. Dive in, fork it, and tag ZETIC.ai when you launch your own on-device assistant, game NPC, or offline content generator—we’ll spotlight the best projects.
Today we’re open-sourcing everything you need to put Qwen3-0.6B straight into a production-ready mobile app:
🎥 Watch Qwen3-0.6B chat in real time on any smartphones!
📊 TPS benchmarks – slides comparing token-per-second across heterogeneous mobile devices
💻 Plug-and-play source – Just Copy & Run the source to your project for Android (Kotlin & Java) and iOS (Swift).
🤞 Cross-platform, one pipeline – ZETIC.MLange auto-tunes kernels for every different devices, we’ve tested.
👨💻 Ready for production – swap in your own model, re-benchmark with one command, publish.
Get started
Just Sign-up and check the playground project, QWEN-0.6B
- https://mlange.zetic.ai/p/zetic-example/Qwen3-0.6B
We built this to show that cloud-free LLMs are ready today. Dive in, fork it, and tag ZETIC.ai when you launch your own on-device assistant, game NPC, or offline content generator—we’ll spotlight the best projects.
reacted to celinah's post with 🚀 9 months ago
Post
2741
✨ Today we’re releasing Tiny Agents in Python — an MCP-powered Agent in ~70 lines of code 🐍
Inspired by Tiny Agents in JS from @julien-c , we ported the idea to Python and integrated it directly into
TL;DR: With MCP (Model Context Protocol), you can expose tools like web search or image generation and connect them directly to LLMs. It’s simple — and surprisingly powerful.
We wrote a blog post where we show how to run Tiny Agents, and dive deeper into how they work and how to build your own.
👉 https://huggingface.co/blog/python-tiny-agents
Inspired by Tiny Agents in JS from @julien-c , we ported the idea to Python and integrated it directly into
huggingface_hub — with a built-in MCP Client and a Tiny Agents CLI.TL;DR: With MCP (Model Context Protocol), you can expose tools like web search or image generation and connect them directly to LLMs. It’s simple — and surprisingly powerful.
pip install "huggingface_hub[mcp]>=0.32.0"We wrote a blog post where we show how to run Tiny Agents, and dive deeper into how they work and how to build your own.
👉 https://huggingface.co/blog/python-tiny-agents
- 1 reply
reacted to wolfram's post with 🔥🤗 10 months ago
Post
7908
Finally finished my extensive **Qwen 3 evaluations** across a range of formats and quantisations, focusing on **MMLU-Pro** (Computer Science).
A few take-aways stood out - especially for those interested in local deployment and performance trade-offs:
1️⃣ **Qwen3-235B-A22B** (via Fireworks API) tops the table at **83.66%** with ~55 tok/s.
2️⃣ But the **30B-A3B Unsloth** quant delivered **82.20%** while running locally at ~45 tok/s and with zero API spend.
3️⃣ The same Unsloth build is ~5x faster than Qwen's **Qwen3-32B**, which scores **82.20%** as well yet crawls at <10 tok/s.
4️⃣ On Apple silicon, the **30B MLX** port hits **79.51%** while sustaining ~64 tok/s - arguably today's best speed/quality trade-off for Mac setups.
5️⃣ The **0.6B** micro-model races above 180 tok/s but tops out at **37.56%** - that's why it's not even on the graph (50 % performance cut-off).
All local runs were done with LM Studio on an M4 MacBook Pro, using Qwen's official recommended settings.
**Conclusion:** Quantised 30B models now get you ~98 % of frontier-class accuracy - at a fraction of the latency, cost, and energy. For most local RAG or agent workloads, they're not just good enough - they're the new default.
Well done, Qwen - you really whipped the llama's ass! And to OpenAI: for your upcoming open model, please make it MoE, with toggleable reasoning, and release it in many sizes. *This* is the future!
A few take-aways stood out - especially for those interested in local deployment and performance trade-offs:
1️⃣ **Qwen3-235B-A22B** (via Fireworks API) tops the table at **83.66%** with ~55 tok/s.
2️⃣ But the **30B-A3B Unsloth** quant delivered **82.20%** while running locally at ~45 tok/s and with zero API spend.
3️⃣ The same Unsloth build is ~5x faster than Qwen's **Qwen3-32B**, which scores **82.20%** as well yet crawls at <10 tok/s.
4️⃣ On Apple silicon, the **30B MLX** port hits **79.51%** while sustaining ~64 tok/s - arguably today's best speed/quality trade-off for Mac setups.
5️⃣ The **0.6B** micro-model races above 180 tok/s but tops out at **37.56%** - that's why it's not even on the graph (50 % performance cut-off).
All local runs were done with LM Studio on an M4 MacBook Pro, using Qwen's official recommended settings.
**Conclusion:** Quantised 30B models now get you ~98 % of frontier-class accuracy - at a fraction of the latency, cost, and energy. For most local RAG or agent workloads, they're not just good enough - they're the new default.
Well done, Qwen - you really whipped the llama's ass! And to OpenAI: for your upcoming open model, please make it MoE, with toggleable reasoning, and release it in many sizes. *This* is the future!
- 4 replies
reacted to sharpenb's post with 🚀👍 10 months ago
Post
3200
How to learn about efficient AI? - Happy to announce the Awesome AI Efficiency repo that gathers a curated list of 100+ materials to understand the challenges and solutions in making AI faster, smaller, cheaper, greener.
🚀 It is designed for a **large audience** including beginners, decision-makers, engineers, and researchers.
📚 It contains **diverse materials** with newspaper articles, blogs, tools, tech reports, research papers, books, and lectures.
This is an ongoing project. Do not hesitate to share your feedback/suggestions and star the repo! 🌟
https://github.com/PrunaAI/awesome-ai-efficiency
🚀 It is designed for a **large audience** including beginners, decision-makers, engineers, and researchers.
📚 It contains **diverse materials** with newspaper articles, blogs, tools, tech reports, research papers, books, and lectures.
This is an ongoing project. Do not hesitate to share your feedback/suggestions and star the repo! 🌟
https://github.com/PrunaAI/awesome-ai-efficiency
- 2 replies
Model seems to not be performing correctly
1
#9 opened 10 months ago
by
daniel-ltw
reacted to abidlabs's post with ❤️ 10 months ago
Post
5263
HOW TO ADD MCP SUPPORT TO ANY 🤗 SPACE
Gradio now supports MCP! If you want to convert an existing Space, like this one hexgrad/Kokoro-TTS, so that you can use it with Claude Desktop / Cursor / Cline / TinyAgents / or any LLM that supports MCP, here's all you need to do:
1. Duplicate the Space (in the Settings Tab)
2. Upgrade the Gradio
3. Set
4. (Optionally) add docstrings to the function so that the LLM knows how to use it, like this:
That's it! Now your LLM will be able to talk to you 🤯
Gradio now supports MCP! If you want to convert an existing Space, like this one hexgrad/Kokoro-TTS, so that you can use it with Claude Desktop / Cursor / Cline / TinyAgents / or any LLM that supports MCP, here's all you need to do:
1. Duplicate the Space (in the Settings Tab)
2. Upgrade the Gradio
sdk_version to 5.28 (in the README.md)3. Set
mcp_server=True in launch()4. (Optionally) add docstrings to the function so that the LLM knows how to use it, like this:
def generate(text, speed=1):
"""
Convert text to speech audio.
Parameters:
text (str): The input text to be converted to speech.
speed (float, optional): Playback speed of the generated speech.That's it! Now your LLM will be able to talk to you 🤯