jiazhizhong/higress
1
0
Fork 0
mirror of https://github.com/alibaba/higress.git synced 2026-05-08 04:17:27 +08:00
Code Issues Packages Projects Releases Wiki Activity

Replace model-router and model-mapper with Go implementation (#3317)

Browse Source
This commit is contained in:
rinfx
2026-01-13 20:14:29 +08:00
committed by jingze
parent 032a69556f
commit e23ab3ca7c
14 changed files with 1418 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

97
plugins/wasm-go/extensions/model-router/README_EN.md Normal file
View File

@@ -0,0 +1,97 @@
## Feature Description
The `model-router` plugin implements routing functionality based on the model parameter in LLM protocols.
## Configuration Fields
| Name | Data Type | Requirement | Default Value | Description |
| ----------- | --------------- | ----------------------- | ------ | ------------------------------------------- |
| `modelKey` | string | Optional | model | Location of the model parameter in the request body |
| `addProviderHeader` | string | Optional | - | Which request header to add the provider name parsed from the model parameter |
| `modelToHeader` | string | Optional | - | Which request header to directly add the model parameter to |
| `enableOnPathSuffix` | array of string | Optional | ["/completions","/embeddings","/images/generations","/audio/speech","/fine_tuning/jobs","/moderations","/image-synthesis","/video-synthesis","/rerank","/messages"] | Only effective for requests with these specific path suffixes, can be configured as "*" to match all paths |
## Runtime Properties
Plugin execution phase: Authentication phase
Plugin execution priority: 900
## Effect Description
### Routing Based on Model Parameter
The following configuration is needed:
```yaml
modelToHeader: x-higress-llm-model
```
The plugin extracts the model parameter from the request and sets it to the x-higress-llm-model request header for subsequent routing. For example, the original LLM request body is:
```json
{
"model": "qwen-long",
"frequency_penalty": 0,
"max_tokens": 800,
"stream": false,
"messages": [{
"role": "user",
"content": "What is the GitHub address of the Higress project's main repository?"
}],
"presence_penalty": 0,
"temperature": 0.7,
"top_p": 0.95
}
```
After processing by this plugin, the following request header will be added (can be used for route matching):
x-higress-llm-model: qwen-long
### Extracting Provider Field from Model Parameter for Routing
> Note that this mode requires the client to specify the provider in the model parameter using the `/` delimiter
The following configuration is needed:
```yaml
addProviderHeader: x-higress-llm-provider
```
The plugin extracts the provider part (if any) from the model parameter in the request, sets it to the x-higress-llm-provider request header for subsequent routing, and rewrites the model parameter to only contain the model name part. For example, the original LLM request body is:
```json
{
"model": "dashscope/qwen-long",
"frequency_penalty": 0,
"max_tokens": 800,
"stream": false,
"messages": [{
"role": "user",
"content": "What is the GitHub address of the Higress project's main repository?"
}],
"presence_penalty": 0,
"temperature": 0.7,
"top_p": 0.95
}
```
After processing by this plugin, the following request header will be added (can be used for route matching):
x-higress-llm-provider: dashscope
The original LLM request body will be changed to:
```json
{
"model": "qwen-long",
"frequency_penalty": 0,
"max_tokens": 800,
"stream": false,
"messages": [{
"role": "user",
"content": "What is the GitHub address of the Higress project's main repository?"
}],
"presence_penalty": 0,
"temperature": 0.7,
"top_p": 0.95
}