Noblhyon commited on
Commit
6215020
Β·
verified Β·
1 Parent(s): 3e66230

Upload Flask API README.md

Browse files
Files changed (1) hide show
  1. README.md +157 -36
README.md CHANGED
@@ -3,9 +3,8 @@ title: BAAI Vector Api
3
  emoji: πŸš€
4
  colorFrom: blue
5
  colorTo: purple
6
- sdk: gradio
7
- sdk_version: 4.44.0
8
- app_file: app.py
9
  pinned: false
10
  license: mit
11
  models:
@@ -16,11 +15,13 @@ tags:
16
  - multilingual
17
  - retrieval
18
  - bge-m3
 
 
19
  ---
20
 
21
- # BGE-M3 Vector API Demo πŸš€
22
 
23
- This Hugging Face Space demonstrates the powerful capabilities of the **BGE-M3** embedding model, featuring multi-functionality, multi-linguality, and multi-granularity text processing.
24
 
25
  ## 🌟 Features
26
 
@@ -39,13 +40,110 @@ This Hugging Face Space demonstrates the powerful capabilities of the **BGE-M3**
39
  - Handle up to **8192 tokens** in a single input
40
  - Consistent performance across different text lengths
41
 
42
- ## 🎯 Use Cases
43
 
44
- 1. **Semantic Search**: Find relevant documents using natural language queries
45
- 2. **Text Similarity**: Compare semantic similarity between texts
46
- 3. **Multilingual Retrieval**: Search across different languages
47
- 4. **Document Clustering**: Group similar documents together
48
- 5. **Question Answering**: Retrieve relevant passages for questions
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
49
 
50
  ## πŸ”§ Model Details
51
 
@@ -55,6 +153,53 @@ This Hugging Face Space demonstrates the powerful capabilities of the **BGE-M3**
55
  - **Max Sequence Length**: 8192 tokens
56
  - **Languages**: 100+ supported
57
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
58
  ## οΏ½οΏ½ Performance
59
 
60
  BGE-M3 achieves state-of-the-art performance on various benchmarks:
@@ -63,30 +208,6 @@ BGE-M3 achieves state-of-the-art performance on various benchmarks:
63
  - **MLDR**: Long document retrieval
64
  - **NarritiveQA**: Long text understanding
65
 
66
- ## πŸš€ Quick Start
67
-
68
- Try the different tabs in this Space:
69
-
70
- 1. **Text Embeddings**: Generate dense, sparse, or multi-vector embeddings
71
- 2. **Similarity Comparison**: Compare semantic similarity between texts
72
- 3. **Document Search**: Search through your documents using natural language
73
- 4. **Model Info**: Learn more about BGE-M3 capabilities
74
-
75
- ## πŸ’» Code Usage
76
-
77
- ```python
78
- from FlagEmbedding import BGEM3FlagModel
79
-
80
- # Load the model
81
- model = BGEM3FlagModel('Noblhyon/BAAI_Vector_Api', use_fp16=True)
82
-
83
- # Generate embeddings
84
- embeddings = model.encode(["Your text here"], max_length=8192)['dense_vecs']
85
-
86
- # Compute similarity
87
- scores = model.compute_score([["text1", "text2"]])
88
- ```
89
-
90
  ## πŸ“š Citation
91
 
92
  ```bibtex
@@ -108,4 +229,4 @@ scores = model.compute_score([["text1", "text2"]])
108
 
109
  ---
110
 
111
- *Built with ❀️ using Gradio and Hugging Face Spaces*
 
3
  emoji: πŸš€
4
  colorFrom: blue
5
  colorTo: purple
6
+ sdk: docker
7
+ app_port: 7860
 
8
  pinned: false
9
  license: mit
10
  models:
 
15
  - multilingual
16
  - retrieval
17
  - bge-m3
18
+ - flask
19
+ - api
20
  ---
21
 
22
+ # BGE-M3 Vector API πŸš€
23
 
24
+ A Flask-based REST API for the **BGE-M3** embedding model, featuring multi-functionality, multi-linguality, and multi-granularity text processing.
25
 
26
  ## 🌟 Features
27
 
 
40
  - Handle up to **8192 tokens** in a single input
41
  - Consistent performance across different text lengths
42
 
43
+ ## πŸ”§ API Endpoints
44
 
45
+ ### Base Information
46
+ - `GET /` - API information and available endpoints
47
+ - `GET /health` - Health check endpoint
48
+
49
+ ### Core Functionality
50
+ - `POST /embed` - Generate embeddings for text(s)
51
+ - `POST /similarity` - Compute similarity between text pairs
52
+ - `POST /search` - Search through documents using semantic similarity
53
+
54
+ ## πŸ“š API Usage Examples
55
+
56
+ ### 1. Generate Embeddings
57
+
58
+ ```bash
59
+ curl -X POST https://huggingface.co/spaces/Noblhyon/BAAI_Vector_Api/embed \
60
+ -H "Content-Type: application/json" \
61
+ -d '{
62
+ "texts": ["Hello world", "How are you?"],
63
+ "return_dense": true,
64
+ "return_sparse": false,
65
+ "max_length": 512
66
+ }'
67
+ ```
68
+
69
+ **Response:**
70
+ ```json
71
+ {
72
+ "success": true,
73
+ "num_texts": 2,
74
+ "processing_time": 0.123,
75
+ "dense_embeddings": [[0.1, 0.2, ...], [0.3, 0.4, ...]],
76
+ "dense_shape": [2, 1024]
77
+ }
78
+ ```
79
+
80
+ ### 2. Compute Similarity
81
+
82
+ ```bash
83
+ curl -X POST https://huggingface.co/spaces/Noblhyon/BAAI_Vector_Api/similarity \
84
+ -H "Content-Type: application/json" \
85
+ -d '{
86
+ "pairs": [["Hello world", "Hi there"], ["Cat", "Dog"]],
87
+ "method": "all"
88
+ }'
89
+ ```
90
+
91
+ **Response:**
92
+ ```json
93
+ {
94
+ "success": true,
95
+ "method": "all",
96
+ "num_pairs": 2,
97
+ "processing_time": 0.234,
98
+ "scores": {
99
+ "dense": [0.8234, 0.4567],
100
+ "sparse": [0.1234, 0.0567],
101
+ "colbert": [0.7890, 0.5432],
102
+ "combined": [0.7456, 0.4123]
103
+ }
104
+ }
105
+ ```
106
+
107
+ ### 3. Document Search
108
+
109
+ ```bash
110
+ curl -X POST https://huggingface.co/spaces/Noblhyon/BAAI_Vector_Api/search \
111
+ -H "Content-Type: application/json" \
112
+ -d '{
113
+ "query": "machine learning",
114
+ "documents": [
115
+ "Deep learning is a subset of machine learning",
116
+ "Cats are cute animals",
117
+ "Neural networks are used in AI"
118
+ ],
119
+ "top_k": 2
120
+ }'
121
+ ```
122
+
123
+ **Response:**
124
+ ```json
125
+ {
126
+ "success": true,
127
+ "query": "machine learning",
128
+ "num_documents": 3,
129
+ "top_k": 2,
130
+ "processing_time": 0.345,
131
+ "results": [
132
+ {
133
+ "rank": 1,
134
+ "document_index": 0,
135
+ "document": "Deep learning is a subset of machine learning",
136
+ "similarity_score": 0.8765
137
+ },
138
+ {
139
+ "rank": 2,
140
+ "document_index": 2,
141
+ "document": "Neural networks are used in AI",
142
+ "similarity_score": 0.6543
143
+ }
144
+ ]
145
+ }
146
+ ```
147
 
148
  ## πŸ”§ Model Details
149
 
 
153
  - **Max Sequence Length**: 8192 tokens
154
  - **Languages**: 100+ supported
155
 
156
+ ## πŸš€ Python Client Example
157
+
158
+ ```python
159
+ import requests
160
+ import json
161
+
162
+ # API base URL
163
+ BASE_URL = "https://huggingface.co/spaces/Noblhyon/BAAI_Vector_Api"
164
+
165
+ def get_embeddings(texts):
166
+ response = requests.post(
167
+ f"{BASE_URL}/embed",
168
+ json={
169
+ "texts": texts,
170
+ "return_dense": True,
171
+ "max_length": 512
172
+ }
173
+ )
174
+ return response.json()
175
+
176
+ def compute_similarity(text1, text2):
177
+ response = requests.post(
178
+ f"{BASE_URL}/similarity",
179
+ json={
180
+ "pairs": [[text1, text2]],
181
+ "method": "all"
182
+ }
183
+ )
184
+ return response.json()
185
+
186
+ def search_documents(query, documents, top_k=5):
187
+ response = requests.post(
188
+ f"{BASE_URL}/search",
189
+ json={
190
+ "query": query,
191
+ "documents": documents,
192
+ "top_k": top_k
193
+ }
194
+ )
195
+ return response.json()
196
+
197
+ # Example usage
198
+ embeddings = get_embeddings(["Hello world", "How are you?"])
199
+ similarity = compute_similarity("Hello", "Hi")
200
+ search_results = search_documents("AI", ["Machine learning", "Cooking", "Neural networks"])
201
+ ```
202
+
203
  ## οΏ½οΏ½ Performance
204
 
205
  BGE-M3 achieves state-of-the-art performance on various benchmarks:
 
208
  - **MLDR**: Long document retrieval
209
  - **NarritiveQA**: Long text understanding
210
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
211
  ## πŸ“š Citation
212
 
213
  ```bibtex
 
229
 
230
  ---
231
 
232
+ *Built with ❀️ using Flask and Docker*