jiazhizhong/higress
1
0
Fork 0
mirror of https://github.com/alibaba/higress.git synced 2026-05-26 05:37:25 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat(ai-proxy): add auto protocol compatibility for OpenAI and Claude APIs (#2810)

Browse Source
This commit is contained in:
澄潭
2025-08-25 14:13:51 +08:00
committed by GitHub
parent 72b98ab6cf
commit 16a18c6609
5 changed files with 494 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

77
plugins/wasm-go/extensions/ai-proxy/README_EN.md
View File

@@ -8,10 +8,21 @@ description: Reference for configuring the AI Proxy plugin
The `AI Proxy` plugin implements AI proxy functionality based on the OpenAI API contract. It currently supports AI service providers such as OpenAI, Azure OpenAI, Moonshot, and Qwen.
> **Note:**
**🚀 Auto Protocol Compatibility**
The plugin now supports **automatic protocol detection**, allowing seamless compatibility with both OpenAI and Claude protocol formats without configuration:
- **OpenAI Protocol**: Request path `/v1/chat/completions`, using standard OpenAI Messages API format
- **Claude Protocol**: Request path `/v1/messages`, using Anthropic Claude Messages API format
- **Intelligent Conversion**: Automatically detects request protocol and performs conversion if the target provider doesn't natively support it
- **Zero Configuration**: No need to set `protocol` field, the plugin handles everything automatically
> **Protocol Support:**
> When the request path suffix matches `/v1/chat/completions`, it corresponds to text-to-text scenarios. The request body will be parsed using OpenAI's text-to-text protocol and then converted to the corresponding LLM vendor's text-to-text protocol.
> When the request path suffix matches `/v1/messages`, it corresponds to Claude text-to-text scenarios. The plugin automatically detects provider capabilities: if native Claude protocol is supported, requests are forwarded directly; otherwise, they are converted to OpenAI protocol first.
> When the request path suffix matches `/v1/embeddings`, it corresponds to text vector scenarios. The request body will be parsed using OpenAI's text vector protocol and then converted to the corresponding LLM vendor's text vector protocol.
## Execution Properties
@@ -35,7 +46,7 @@ Plugin execution priority: `100`
| `apiTokens` | array of string | Optional | - | Tokens used for authentication when accessing AI services. If multiple tokens are configured, the plugin randomly selects one for each request. Some service providers only support configuring a single token. |
| `timeout` | number | Optional | - | Timeout for accessing AI services, in milliseconds. The default value is 120000, which equals 2 minutes. Only used when retrieving context data. Won't affect the request forwarded to the LLM upstream. |
| `modelMapping` | map of string | Optional | - | Mapping table for AI models, used to map model names in requests to names supported by the service provider.<br/>1. Supports prefix matching. For example, "gpt-3-\*" matches all model names starting with “gpt-3-”;<br/>2. Supports using "\*" as a key for a general fallback mapping;<br/>3. If the mapped target name is an empty string "", the original model name is preserved. |
| `protocol` | string | Optional | - | API contract provided by the plugin. Currently supports the following values: openai (default, uses OpenAI's interface contract), original (uses the raw interface contract of the target service provider) |
| `protocol` | string | Optional | - | API contract provided by the plugin. Currently supports the following values: openai (default, uses OpenAI's interface contract), original (uses the raw interface contract of the target service provider). **Note: Auto protocol detection is now supported, no need to configure this field to support both OpenAI and Claude protocols** |
| `context` | object | Optional | - | Configuration for AI conversation context information |
| `customSettings` | array of customSetting | Optional | - | Specifies overrides or fills parameters for AI requests |
| `subPath` | string | Optional | - | If subPath is configured, the prefix will be removed from the request path before further processing. |
@@ -883,19 +894,40 @@ provider:
}
```
### Using OpenAI Protocol Proxy for Claude Service
### Using Auto Protocol Compatibility
The plugin now supports automatic protocol detection, capable of handling both OpenAI and Claude protocol format requests simultaneously.
**Configuration Information**
```yaml
provider:
type: claude
type: claude # Provider with native Claude protocol support
apiTokens:
- "YOUR_CLAUDE_API_TOKEN"
version: "2023-06-01"
```
**Example Request**
**OpenAI Protocol Request Example**
URL: `http://your-domain/v1/chat/completions`
```json
{
"model": "claude-3-opus-20240229",
"max_tokens": 1024,
"messages": [
{
"role": "user",
"content": "Hello, who are you?"
}
]
}
```
**Claude Protocol Request Example**
URL: `http://your-domain/v1/messages`
```json
{
@@ -912,6 +944,8 @@ provider:
**Example Response**
Both protocol formats will return responses in their respective formats:
```json
{
"id": "msg_01Jt3GzyjuzymnxmZERJguLK",
@@ -936,6 +970,39 @@ provider:
}
```
### Using Intelligent Protocol Conversion
When the target provider doesn't natively support Claude protocol, the plugin automatically performs protocol conversion:
**Configuration Information**
```yaml
provider:
type: qwen # Doesn't natively support Claude protocol, auto-conversion applied
apiTokens:
- "YOUR_QWEN_API_TOKEN"
modelMapping:
'claude-3-opus-20240229': 'qwen-max'
'*': 'qwen-turbo'
```
**Claude Protocol Request**
URL: `http://your-domain/v1/messages` (automatically converted to OpenAI protocol for provider)
```json
{
"model": "claude-3-opus-20240229",
"max_tokens": 1024,
"messages": [
{
"role": "user",
"content": "Hello, who are you?"
}
]
}
```
### Using OpenAI Protocol Proxy for Hunyuan Service
**Configuration Information**