Post
91
Isaacus just shipped a major update to semchunk: AI-powered chunking based on a document’s knowledge graph representation⚡
This isn’t a tweak on existing semantic chunking. It’s an entirely new paradigm, built on hierarchical document segmentation rather than heuristics or standard embedding-based semantic approaches.
We benchmarked our AI chunking mode across a full RAG pipeline against popular alternatives like LangChain, Chonkie, and our own non-AI semantic chunker. The results were clear: semchunk’s AI mode delivered a 15% relative improvement in RAG correctness over Chonkie. It also produced more aesthetically coherent and readable when judged by a human evaluator while also being faster than all other chunking methods when run on a consumer PC.
These gains are powered by Isaacus' Kanon 2 Enricher model, which performs hierarchical document segmentation and directly powers our AI chunking mode.
As far as we know, semchunk is one of the first chunking libraries to offer true AI-powered, hierarchical-segmentation-based chunking, and the results show how much better RAG can get when chunking improves.
https://huggingface.co/blog/isaacus/introducing-ai-chunking-to-semchunk
This isn’t a tweak on existing semantic chunking. It’s an entirely new paradigm, built on hierarchical document segmentation rather than heuristics or standard embedding-based semantic approaches.
We benchmarked our AI chunking mode across a full RAG pipeline against popular alternatives like LangChain, Chonkie, and our own non-AI semantic chunker. The results were clear: semchunk’s AI mode delivered a 15% relative improvement in RAG correctness over Chonkie. It also produced more aesthetically coherent and readable when judged by a human evaluator while also being faster than all other chunking methods when run on a consumer PC.
These gains are powered by Isaacus' Kanon 2 Enricher model, which performs hierarchical document segmentation and directly powers our AI chunking mode.
As far as we know, semchunk is one of the first chunking libraries to offer true AI-powered, hierarchical-segmentation-based chunking, and the results show how much better RAG can get when chunking improves.
https://huggingface.co/blog/isaacus/introducing-ai-chunking-to-semchunk
