Add prebuild ONNX models for jinaai/jina-embeddings-v5-text-small
Would be great to have a pre-built ONNX of your task specific models like you currently have for gguf and mlx on your jinaai/jina-embeddings-v5-text-small cpllections
Ok, we will add those soon. For nano @Xenova already contributed onnx exports, e.g., here: https://huggingface.co/jinaai/jina-embeddings-v5-text-nano-retrieval
Ok now, there are pre-built onnx model with usage instructions also for the small models
Hello Michael,
Thank you for the pre-built onnx for the small models !
For the onnx export from jinaai/jina-embeddings-v5-text-nano-retrieval , the model outputs "last_hidden_state" and "sentence_embedding" whereas the onnx export from jinaai/jina-embeddings-v5-text-small-retrieval , the model output is just "last_hidden_state".
Is it possible to get the same onnx export here as in jinaai/jina-embeddings-v5-text-nano-retrieval ?
For example, it is more convenient to get "sentence-embedding" output if I want to serve the onnx model to a triton-inference-server to avoid extra-cost network trafic.
Thank you !
Many thanks Michael!
A quick question if you do not mind. Do I understand correctly that unlike qwen base model your models prompt are:
for retrieval: "Document: " and "Query: " respectively
for text matching: "Document: " and "Document: " (this part is a bit confusing - config says "" for query but default prompt is document so i assume query prompt == document prompt?):
"prompts":{
"query": "",
"document": "Document: "
},
"default_prompt_name": "document",
@roytmana yes it is exactly how you described it.
or text matching: "Document: " and "Document: " (this part is a bit confusing - config says "" for query but default prompt is document so i assume query prompt == document prompt?):
Basically, sentence transformers and some other libraries use a prompt_name property that you pass and this maps to the the actual prompt that is added to the inputs as prefix. For STS / Text-Matching, you have symmetric inputs. Therefore, we don't want the model to distinguish between queries and documents and accordingly use the same prompt. There is not really a specific reason why we use "Document: " instead of "Query: " for both, but we decided for "Document: ". The "default_prompt_name": "document" configuration makes sure that "Document: " is added automatically if no prompt type is passed. This means, when using the retrieval task, you need to pass the prompt type, while you don't need to pass it for text-matching.
@aarpin I will look into this and see if I get some info on how the onnx files are generated for the nano model to make them more similar.
@aarpin ok now it should also produce sentence_embedding outputs but the ORTModelForFeatureExtraction still only returns the last_hidden_states output (should be the same for nano)
thank you !