dim of multi-vector in transformers
#81
by Jiajie123222 - opened
I'm using a transformer to load jina-embedding-v4. After enabling return_multivector, each vector I get has 128 dimensions. I want to make it 1024 dimensions. How can I do that?
Hi, can you explain a bit more about your goal behind this? The model includes a projection layer for multi-vector retrieval that transforms the 1024 dimensional vectors to 128 dimensions. This makes the multi-vector search more feasible. The model is trained to perform multi-vector search with those 128 dimensional vectors, so removing the projection layer is possible but it would probably lead much worse retrieval accuracy.