mirror of
https://github.com/alibaba/higress.git
synced 2026-03-10 11:40:49 +08:00
feat(ai-proxy): add auto protocol compatibility for OpenAI and Claude APIs (#2810)
This commit is contained in:
@@ -9,10 +9,21 @@ description: AI 代理插件配置参考
|
||||
`AI 代理`插件实现了基于 OpenAI API 契约的 AI 代理功能。目前支持 OpenAI、Azure OpenAI、月之暗面(Moonshot)和通义千问等 AI
|
||||
服务提供商。
|
||||
|
||||
> **注意:**
|
||||
**🚀 自动协议兼容 (Auto Protocol Compatibility)**
|
||||
|
||||
插件现在支持**自动协议检测**,无需配置即可同时兼容 OpenAI 和 Claude 两种协议格式:
|
||||
|
||||
- **OpenAI 协议**: 请求路径 `/v1/chat/completions`,使用标准的 OpenAI Messages API 格式
|
||||
- **Claude 协议**: 请求路径 `/v1/messages`,使用 Anthropic Claude Messages API 格式
|
||||
- **智能转换**: 自动检测请求协议,如果目标供应商不原生支持该协议,则自动进行协议转换
|
||||
- **零配置**: 用户无需设置 `protocol` 字段,插件自动处理
|
||||
|
||||
> **协议支持说明:**
|
||||
|
||||
> 请求路径后缀匹配 `/v1/chat/completions` 时,对应文生文场景,会用 OpenAI 的文生文协议解析请求 Body,再转换为对应 LLM 厂商的文生文协议
|
||||
|
||||
> 请求路径后缀匹配 `/v1/messages` 时,对应 Claude 文生文场景,会自动检测供应商能力:如果支持原生 Claude 协议则直接转发,否则先转换为 OpenAI 协议再转发给供应商
|
||||
|
||||
> 请求路径后缀匹配 `/v1/embeddings` 时,对应文本向量场景,会用 OpenAI 的文本向量协议解析请求 Body,再转换为对应 LLM 厂商的文本向量协议
|
||||
|
||||
## 运行属性
|
||||
@@ -937,19 +948,40 @@ provider:
|
||||
}
|
||||
```
|
||||
|
||||
### 使用 OpenAI 协议代理 Claude 服务
|
||||
### 使用自动协议兼容功能
|
||||
|
||||
插件现在支持自动协议检测,可以同时处理 OpenAI 和 Claude 两种协议格式的请求。
|
||||
|
||||
**配置信息**
|
||||
|
||||
```yaml
|
||||
provider:
|
||||
type: claude
|
||||
type: claude # 原生支持 Claude 协议的供应商
|
||||
apiTokens:
|
||||
- 'YOUR_CLAUDE_API_TOKEN'
|
||||
version: '2023-06-01'
|
||||
```
|
||||
|
||||
**请求示例**
|
||||
**OpenAI 协议请求示例**
|
||||
|
||||
URL: `http://your-domain/v1/chat/completions`
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "claude-3-opus-20240229",
|
||||
"max_tokens": 1024,
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "你好,你是谁?"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Claude 协议请求示例**
|
||||
|
||||
URL: `http://your-domain/v1/messages`
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -966,6 +998,8 @@ provider:
|
||||
|
||||
**响应示例**
|
||||
|
||||
两种协议格式的请求都会返回相应格式的响应:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "msg_01Jt3GzyjuzymnxmZERJguLK",
|
||||
@@ -990,6 +1024,39 @@ provider:
|
||||
}
|
||||
```
|
||||
|
||||
### 使用智能协议转换
|
||||
|
||||
当目标供应商不原生支持 Claude 协议时,插件会自动进行协议转换:
|
||||
|
||||
**配置信息**
|
||||
|
||||
```yaml
|
||||
provider:
|
||||
type: qwen # 不原生支持 Claude 协议,会自动转换
|
||||
apiTokens:
|
||||
- 'YOUR_QWEN_API_TOKEN'
|
||||
modelMapping:
|
||||
'claude-3-opus-20240229': 'qwen-max'
|
||||
'*': 'qwen-turbo'
|
||||
```
|
||||
|
||||
**Claude 协议请求**
|
||||
|
||||
URL: `http://your-domain/v1/messages` (自动转换为 OpenAI 协议调用供应商)
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "claude-3-opus-20240229",
|
||||
"max_tokens": 1024,
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "你好,你是谁?"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 使用 OpenAI 协议代理混元服务
|
||||
|
||||
**配置信息**
|
||||
|
||||
@@ -8,10 +8,21 @@ description: Reference for configuring the AI Proxy plugin
|
||||
|
||||
The `AI Proxy` plugin implements AI proxy functionality based on the OpenAI API contract. It currently supports AI service providers such as OpenAI, Azure OpenAI, Moonshot, and Qwen.
|
||||
|
||||
> **Note:**
|
||||
**🚀 Auto Protocol Compatibility**
|
||||
|
||||
The plugin now supports **automatic protocol detection**, allowing seamless compatibility with both OpenAI and Claude protocol formats without configuration:
|
||||
|
||||
- **OpenAI Protocol**: Request path `/v1/chat/completions`, using standard OpenAI Messages API format
|
||||
- **Claude Protocol**: Request path `/v1/messages`, using Anthropic Claude Messages API format
|
||||
- **Intelligent Conversion**: Automatically detects request protocol and performs conversion if the target provider doesn't natively support it
|
||||
- **Zero Configuration**: No need to set `protocol` field, the plugin handles everything automatically
|
||||
|
||||
> **Protocol Support:**
|
||||
|
||||
> When the request path suffix matches `/v1/chat/completions`, it corresponds to text-to-text scenarios. The request body will be parsed using OpenAI's text-to-text protocol and then converted to the corresponding LLM vendor's text-to-text protocol.
|
||||
|
||||
> When the request path suffix matches `/v1/messages`, it corresponds to Claude text-to-text scenarios. The plugin automatically detects provider capabilities: if native Claude protocol is supported, requests are forwarded directly; otherwise, they are converted to OpenAI protocol first.
|
||||
|
||||
> When the request path suffix matches `/v1/embeddings`, it corresponds to text vector scenarios. The request body will be parsed using OpenAI's text vector protocol and then converted to the corresponding LLM vendor's text vector protocol.
|
||||
|
||||
## Execution Properties
|
||||
@@ -35,7 +46,7 @@ Plugin execution priority: `100`
|
||||
| `apiTokens` | array of string | Optional | - | Tokens used for authentication when accessing AI services. If multiple tokens are configured, the plugin randomly selects one for each request. Some service providers only support configuring a single token. |
|
||||
| `timeout` | number | Optional | - | Timeout for accessing AI services, in milliseconds. The default value is 120000, which equals 2 minutes. Only used when retrieving context data. Won't affect the request forwarded to the LLM upstream. |
|
||||
| `modelMapping` | map of string | Optional | - | Mapping table for AI models, used to map model names in requests to names supported by the service provider.<br/>1. Supports prefix matching. For example, "gpt-3-\*" matches all model names starting with “gpt-3-”;<br/>2. Supports using "\*" as a key for a general fallback mapping;<br/>3. If the mapped target name is an empty string "", the original model name is preserved. |
|
||||
| `protocol` | string | Optional | - | API contract provided by the plugin. Currently supports the following values: openai (default, uses OpenAI's interface contract), original (uses the raw interface contract of the target service provider) |
|
||||
| `protocol` | string | Optional | - | API contract provided by the plugin. Currently supports the following values: openai (default, uses OpenAI's interface contract), original (uses the raw interface contract of the target service provider). **Note: Auto protocol detection is now supported, no need to configure this field to support both OpenAI and Claude protocols** |
|
||||
| `context` | object | Optional | - | Configuration for AI conversation context information |
|
||||
| `customSettings` | array of customSetting | Optional | - | Specifies overrides or fills parameters for AI requests |
|
||||
| `subPath` | string | Optional | - | If subPath is configured, the prefix will be removed from the request path before further processing. |
|
||||
@@ -883,19 +894,40 @@ provider:
|
||||
}
|
||||
```
|
||||
|
||||
### Using OpenAI Protocol Proxy for Claude Service
|
||||
### Using Auto Protocol Compatibility
|
||||
|
||||
The plugin now supports automatic protocol detection, capable of handling both OpenAI and Claude protocol format requests simultaneously.
|
||||
|
||||
**Configuration Information**
|
||||
|
||||
```yaml
|
||||
provider:
|
||||
type: claude
|
||||
type: claude # Provider with native Claude protocol support
|
||||
apiTokens:
|
||||
- "YOUR_CLAUDE_API_TOKEN"
|
||||
version: "2023-06-01"
|
||||
```
|
||||
|
||||
**Example Request**
|
||||
**OpenAI Protocol Request Example**
|
||||
|
||||
URL: `http://your-domain/v1/chat/completions`
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "claude-3-opus-20240229",
|
||||
"max_tokens": 1024,
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Hello, who are you?"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Claude Protocol Request Example**
|
||||
|
||||
URL: `http://your-domain/v1/messages`
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -912,6 +944,8 @@ provider:
|
||||
|
||||
**Example Response**
|
||||
|
||||
Both protocol formats will return responses in their respective formats:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "msg_01Jt3GzyjuzymnxmZERJguLK",
|
||||
@@ -936,6 +970,39 @@ provider:
|
||||
}
|
||||
```
|
||||
|
||||
### Using Intelligent Protocol Conversion
|
||||
|
||||
When the target provider doesn't natively support Claude protocol, the plugin automatically performs protocol conversion:
|
||||
|
||||
**Configuration Information**
|
||||
|
||||
```yaml
|
||||
provider:
|
||||
type: qwen # Doesn't natively support Claude protocol, auto-conversion applied
|
||||
apiTokens:
|
||||
- "YOUR_QWEN_API_TOKEN"
|
||||
modelMapping:
|
||||
'claude-3-opus-20240229': 'qwen-max'
|
||||
'*': 'qwen-turbo'
|
||||
```
|
||||
|
||||
**Claude Protocol Request**
|
||||
|
||||
URL: `http://your-domain/v1/messages` (automatically converted to OpenAI protocol for provider)
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "claude-3-opus-20240229",
|
||||
"max_tokens": 1024,
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Hello, who are you?"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Using OpenAI Protocol Proxy for Hunyuan Service
|
||||
|
||||
**Configuration Information**
|
||||
|
||||
@@ -194,6 +194,23 @@ func onHttpRequestHeader(ctx wrapper.HttpContext, pluginConfig config.PluginConf
|
||||
}
|
||||
}
|
||||
|
||||
// Auto-detect protocol based on request path and handle conversion if needed
|
||||
// If request is Claude format (/v1/messages) but provider doesn't support it natively,
|
||||
// convert to OpenAI format (/v1/chat/completions)
|
||||
if apiName == provider.ApiNameAnthropicMessages && !providerConfig.IsSupportedAPI(provider.ApiNameAnthropicMessages) {
|
||||
// Provider doesn't support Claude protocol natively, convert to OpenAI format
|
||||
newPath := strings.Replace(path.Path, provider.PathAnthropicMessages, provider.PathOpenAIChatCompletions, 1)
|
||||
_ = proxywasm.ReplaceHttpRequestHeader(":path", newPath)
|
||||
// Update apiName to match the new path
|
||||
apiName = provider.ApiNameChatCompletion
|
||||
// Mark that we need to convert response back to Claude format
|
||||
ctx.SetContext("needClaudeResponseConversion", true)
|
||||
log.Debugf("[Auto Protocol] Claude request detected, provider doesn't support natively, converted path from %s to %s, apiName: %s", path.Path, newPath, apiName)
|
||||
} else if apiName == provider.ApiNameAnthropicMessages {
|
||||
// Provider supports Claude protocol natively, no conversion needed
|
||||
log.Debugf("[Auto Protocol] Claude request detected, provider supports natively, keeping original path: %s, apiName: %s", path.Path, apiName)
|
||||
}
|
||||
|
||||
if contentType, _ := proxywasm.GetHttpRequestHeader(util.HeaderContentType); contentType != "" && !strings.Contains(contentType, util.MimeTypeApplicationJson) {
|
||||
ctx.DontReadRequestBody()
|
||||
log.Debugf("[onHttpRequestHeader] unsupported content type: %s, will not process the request body", contentType)
|
||||
@@ -354,6 +371,18 @@ func onStreamingResponseBody(ctx wrapper.HttpContext, pluginConfig config.Plugin
|
||||
apiName, _ := ctx.GetContext(provider.CtxKeyApiName).(provider.ApiName)
|
||||
modifiedChunk, err := handler.OnStreamingResponseBody(ctx, apiName, chunk, isLastChunk)
|
||||
if err == nil && modifiedChunk != nil {
|
||||
// Check if we need to convert OpenAI stream response back to Claude format
|
||||
// Only convert if we did the forward conversion (provider doesn't support Claude natively)
|
||||
needClaudeConversion, _ := ctx.GetContext("needClaudeResponseConversion").(bool)
|
||||
if needClaudeConversion {
|
||||
converter := &provider.ClaudeToOpenAIConverter{}
|
||||
claudeChunk, err := converter.ConvertOpenAIStreamResponseToClaude(ctx, modifiedChunk)
|
||||
if err != nil {
|
||||
log.Errorf("failed to convert streaming response to claude format: %v", err)
|
||||
return modifiedChunk
|
||||
}
|
||||
return claudeChunk
|
||||
}
|
||||
return modifiedChunk
|
||||
}
|
||||
return chunk
|
||||
@@ -388,7 +417,23 @@ func onStreamingResponseBody(ctx wrapper.HttpContext, pluginConfig config.Plugin
|
||||
}
|
||||
}
|
||||
}
|
||||
return []byte(responseBuilder.String())
|
||||
|
||||
result := []byte(responseBuilder.String())
|
||||
|
||||
// Check if we need to convert OpenAI stream response back to Claude format
|
||||
// Only convert if we did the forward conversion (provider doesn't support Claude natively)
|
||||
needClaudeConversion, _ := ctx.GetContext("needClaudeResponseConversion").(bool)
|
||||
if needClaudeConversion {
|
||||
converter := &provider.ClaudeToOpenAIConverter{}
|
||||
claudeChunk, err := converter.ConvertOpenAIStreamResponseToClaude(ctx, result)
|
||||
if err != nil {
|
||||
log.Errorf("failed to convert streaming event response to claude format: %v", err)
|
||||
return result
|
||||
}
|
||||
return claudeChunk
|
||||
}
|
||||
|
||||
return result
|
||||
}
|
||||
return chunk
|
||||
}
|
||||
@@ -410,6 +455,19 @@ func onHttpResponseBody(ctx wrapper.HttpContext, pluginConfig config.PluginConfi
|
||||
_ = util.ErrorHandler("ai-proxy.proc_resp_body_failed", fmt.Errorf("failed to process response body: %v", err))
|
||||
return types.ActionContinue
|
||||
}
|
||||
|
||||
// Check if we need to convert OpenAI response back to Claude format
|
||||
// Only convert if we did the forward conversion (provider doesn't support Claude natively)
|
||||
needClaudeConversion, _ := ctx.GetContext("needClaudeResponseConversion").(bool)
|
||||
if needClaudeConversion {
|
||||
converter := &provider.ClaudeToOpenAIConverter{}
|
||||
body, err = converter.ConvertOpenAIResponseToClaude(ctx, body)
|
||||
if err != nil {
|
||||
_ = util.ErrorHandler("ai-proxy.convert_resp_to_claude_failed", fmt.Errorf("failed to convert response to claude format: %v", err))
|
||||
return types.ActionContinue
|
||||
}
|
||||
}
|
||||
|
||||
if err = provider.ReplaceResponseBody(body); err != nil {
|
||||
_ = util.ErrorHandler("ai-proxy.replace_resp_body_failed", fmt.Errorf("failed to replace response body: %v", err))
|
||||
}
|
||||
|
||||
271
plugins/wasm-go/extensions/ai-proxy/provider/claude_to_openai.go
Normal file
271
plugins/wasm-go/extensions/ai-proxy/provider/claude_to_openai.go
Normal file
@@ -0,0 +1,271 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/higress-group/wasm-go/pkg/log"
|
||||
"github.com/higress-group/wasm-go/pkg/wrapper"
|
||||
)
|
||||
|
||||
// ClaudeToOpenAIConverter converts Claude protocol requests to OpenAI protocol
|
||||
type ClaudeToOpenAIConverter struct{}
|
||||
|
||||
// ConvertClaudeRequestToOpenAI converts a Claude chat completion request to OpenAI format
|
||||
func (c *ClaudeToOpenAIConverter) ConvertClaudeRequestToOpenAI(body []byte) ([]byte, error) {
|
||||
var claudeRequest claudeTextGenRequest
|
||||
if err := json.Unmarshal(body, &claudeRequest); err != nil {
|
||||
return nil, fmt.Errorf("unable to unmarshal claude request: %v", err)
|
||||
}
|
||||
|
||||
// Convert Claude request to OpenAI format
|
||||
openaiRequest := chatCompletionRequest{
|
||||
Model: claudeRequest.Model,
|
||||
Stream: claudeRequest.Stream,
|
||||
Temperature: claudeRequest.Temperature,
|
||||
TopP: claudeRequest.TopP,
|
||||
MaxTokens: claudeRequest.MaxTokens,
|
||||
Stop: claudeRequest.StopSequences,
|
||||
}
|
||||
|
||||
// Convert messages from Claude format to OpenAI format
|
||||
for _, claudeMsg := range claudeRequest.Messages {
|
||||
openaiMsg := chatMessage{
|
||||
Role: claudeMsg.Role,
|
||||
}
|
||||
|
||||
// Handle different content types
|
||||
switch content := claudeMsg.Content.(type) {
|
||||
case string:
|
||||
// Simple text content
|
||||
openaiMsg.Content = content
|
||||
case []claudeChatMessageContent:
|
||||
// Multi-modal content
|
||||
var openaiContents []chatMessageContent
|
||||
for _, claudeContent := range content {
|
||||
switch claudeContent.Type {
|
||||
case "text":
|
||||
openaiContents = append(openaiContents, chatMessageContent{
|
||||
Type: contentTypeText,
|
||||
Text: claudeContent.Text,
|
||||
})
|
||||
case "image":
|
||||
if claudeContent.Source != nil {
|
||||
if claudeContent.Source.Type == "base64" {
|
||||
// Convert base64 image to OpenAI format
|
||||
dataUrl := fmt.Sprintf("data:%s;base64,%s", claudeContent.Source.MediaType, claudeContent.Source.Data)
|
||||
openaiContents = append(openaiContents, chatMessageContent{
|
||||
Type: contentTypeImageUrl,
|
||||
ImageUrl: &chatMessageContentImageUrl{
|
||||
Url: dataUrl,
|
||||
},
|
||||
})
|
||||
} else if claudeContent.Source.Type == "url" {
|
||||
openaiContents = append(openaiContents, chatMessageContent{
|
||||
Type: contentTypeImageUrl,
|
||||
ImageUrl: &chatMessageContentImageUrl{
|
||||
Url: claudeContent.Source.Url,
|
||||
},
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
openaiMsg.Content = openaiContents
|
||||
}
|
||||
|
||||
openaiRequest.Messages = append(openaiRequest.Messages, openaiMsg)
|
||||
}
|
||||
|
||||
// Handle system message - Claude has separate system field
|
||||
if claudeRequest.System != "" {
|
||||
systemMsg := chatMessage{
|
||||
Role: roleSystem,
|
||||
Content: claudeRequest.System,
|
||||
}
|
||||
// Insert system message at the beginning
|
||||
openaiRequest.Messages = append([]chatMessage{systemMsg}, openaiRequest.Messages...)
|
||||
}
|
||||
|
||||
// Convert tools if present
|
||||
for _, claudeTool := range claudeRequest.Tools {
|
||||
openaiTool := tool{
|
||||
Type: "function",
|
||||
Function: function{
|
||||
Name: claudeTool.Name,
|
||||
Description: claudeTool.Description,
|
||||
Parameters: claudeTool.InputSchema,
|
||||
},
|
||||
}
|
||||
openaiRequest.Tools = append(openaiRequest.Tools, openaiTool)
|
||||
}
|
||||
|
||||
// Convert tool choice if present
|
||||
if claudeRequest.ToolChoice != nil {
|
||||
if claudeRequest.ToolChoice.Type == "tool" && claudeRequest.ToolChoice.Name != "" {
|
||||
openaiRequest.ToolChoice = &toolChoice{
|
||||
Type: "function",
|
||||
Function: function{
|
||||
Name: claudeRequest.ToolChoice.Name,
|
||||
},
|
||||
}
|
||||
} else {
|
||||
// For other types like "auto", "none", etc.
|
||||
openaiRequest.ToolChoice = claudeRequest.ToolChoice.Type
|
||||
}
|
||||
|
||||
// Handle parallel tool calls
|
||||
openaiRequest.ParallelToolCalls = !claudeRequest.ToolChoice.DisableParallelToolUse
|
||||
}
|
||||
|
||||
return json.Marshal(openaiRequest)
|
||||
}
|
||||
|
||||
// ConvertOpenAIResponseToClaude converts an OpenAI response back to Claude format
|
||||
func (c *ClaudeToOpenAIConverter) ConvertOpenAIResponseToClaude(ctx wrapper.HttpContext, body []byte) ([]byte, error) {
|
||||
var openaiResponse chatCompletionResponse
|
||||
if err := json.Unmarshal(body, &openaiResponse); err != nil {
|
||||
return nil, fmt.Errorf("unable to unmarshal openai response: %v", err)
|
||||
}
|
||||
|
||||
// Convert OpenAI response to Claude format
|
||||
claudeResponse := claudeTextGenResponse{
|
||||
Id: openaiResponse.Id,
|
||||
Type: "message",
|
||||
Role: "assistant",
|
||||
Model: openaiResponse.Model,
|
||||
Usage: claudeTextGenUsage{
|
||||
InputTokens: openaiResponse.Usage.PromptTokens,
|
||||
OutputTokens: openaiResponse.Usage.CompletionTokens,
|
||||
},
|
||||
}
|
||||
|
||||
// Convert the first choice content
|
||||
if len(openaiResponse.Choices) > 0 {
|
||||
choice := openaiResponse.Choices[0]
|
||||
if choice.Message != nil {
|
||||
content := claudeTextGenContent{
|
||||
Type: "text",
|
||||
Text: choice.Message.StringContent(),
|
||||
}
|
||||
claudeResponse.Content = []claudeTextGenContent{content}
|
||||
}
|
||||
|
||||
// Convert finish reason
|
||||
if choice.FinishReason != nil {
|
||||
claudeFinishReason := openAIFinishReasonToClaude(*choice.FinishReason)
|
||||
claudeResponse.StopReason = &claudeFinishReason
|
||||
}
|
||||
}
|
||||
|
||||
return json.Marshal(claudeResponse)
|
||||
}
|
||||
|
||||
// ConvertOpenAIStreamResponseToClaude converts OpenAI streaming response to Claude format
|
||||
func (c *ClaudeToOpenAIConverter) ConvertOpenAIStreamResponseToClaude(ctx wrapper.HttpContext, chunk []byte) ([]byte, error) {
|
||||
// For streaming responses, we need to handle the Server-Sent Events format
|
||||
lines := strings.Split(string(chunk), "\n")
|
||||
var result strings.Builder
|
||||
|
||||
for _, line := range lines {
|
||||
if strings.HasPrefix(line, "data: ") {
|
||||
data := strings.TrimPrefix(line, "data: ")
|
||||
|
||||
// Skip [DONE] messages
|
||||
if data == "[DONE]" {
|
||||
continue
|
||||
}
|
||||
|
||||
var openaiStreamResponse chatCompletionResponse
|
||||
if err := json.Unmarshal([]byte(data), &openaiStreamResponse); err != nil {
|
||||
log.Errorf("unable to unmarshal openai stream response: %v", err)
|
||||
continue
|
||||
}
|
||||
|
||||
// Convert to Claude streaming format
|
||||
claudeStreamResponse := c.buildClaudeStreamResponse(ctx, &openaiStreamResponse)
|
||||
if claudeStreamResponse != nil {
|
||||
responseData, err := json.Marshal(claudeStreamResponse)
|
||||
if err != nil {
|
||||
log.Errorf("unable to marshal claude stream response: %v", err)
|
||||
continue
|
||||
}
|
||||
result.WriteString(fmt.Sprintf("data: %s\n\n", responseData))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return []byte(result.String()), nil
|
||||
}
|
||||
|
||||
// buildClaudeStreamResponse builds a Claude streaming response from OpenAI streaming response
|
||||
func (c *ClaudeToOpenAIConverter) buildClaudeStreamResponse(ctx wrapper.HttpContext, openaiResponse *chatCompletionResponse) *claudeTextGenStreamResponse {
|
||||
if len(openaiResponse.Choices) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
choice := openaiResponse.Choices[0]
|
||||
|
||||
// Determine the response type based on the content
|
||||
if choice.Delta != nil && choice.Delta.Content != "" {
|
||||
// Content delta
|
||||
if deltaContent, ok := choice.Delta.Content.(string); ok {
|
||||
return &claudeTextGenStreamResponse{
|
||||
Type: "content_block_delta",
|
||||
Index: choice.Index,
|
||||
Delta: &claudeTextGenDelta{
|
||||
Type: "text_delta",
|
||||
Text: deltaContent,
|
||||
},
|
||||
}
|
||||
}
|
||||
} else if choice.FinishReason != nil {
|
||||
// Message completed
|
||||
claudeFinishReason := openAIFinishReasonToClaude(*choice.FinishReason)
|
||||
return &claudeTextGenStreamResponse{
|
||||
Type: "message_delta",
|
||||
Index: choice.Index,
|
||||
Delta: &claudeTextGenDelta{
|
||||
Type: "message_delta",
|
||||
StopReason: &claudeFinishReason,
|
||||
},
|
||||
Usage: &claudeTextGenUsage{
|
||||
InputTokens: openaiResponse.Usage.PromptTokens,
|
||||
OutputTokens: openaiResponse.Usage.CompletionTokens,
|
||||
},
|
||||
}
|
||||
} else if choice.Delta != nil && choice.Delta.Role != "" {
|
||||
// Message start
|
||||
return &claudeTextGenStreamResponse{
|
||||
Type: "message_start",
|
||||
Index: choice.Index,
|
||||
Message: &claudeTextGenResponse{
|
||||
Id: openaiResponse.Id,
|
||||
Type: "message",
|
||||
Role: "assistant",
|
||||
Model: openaiResponse.Model,
|
||||
Usage: claudeTextGenUsage{
|
||||
InputTokens: openaiResponse.Usage.PromptTokens,
|
||||
OutputTokens: 0,
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// openAIFinishReasonToClaude converts OpenAI finish reason to Claude format
|
||||
func openAIFinishReasonToClaude(reason string) string {
|
||||
switch reason {
|
||||
case finishReasonStop:
|
||||
return "end_turn"
|
||||
case finishReasonLength:
|
||||
return "max_tokens"
|
||||
case finishReasonToolCall:
|
||||
return "tool_use"
|
||||
default:
|
||||
return reason
|
||||
}
|
||||
}
|
||||
@@ -3,6 +3,7 @@ package provider
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"fmt"
|
||||
"math/rand"
|
||||
"net/http"
|
||||
"path"
|
||||
@@ -522,10 +523,9 @@ func (c *ProviderConfig) FromJson(json gjson.Result) {
|
||||
c.reasoningContentMode = strings.ToLower(c.reasoningContentMode)
|
||||
switch c.reasoningContentMode {
|
||||
case reasoningBehaviorPassThrough, reasoningBehaviorIgnore, reasoningBehaviorConcat:
|
||||
break
|
||||
// valid values, no action needed
|
||||
default:
|
||||
c.reasoningContentMode = reasoningBehaviorPassThrough
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
@@ -832,6 +832,10 @@ func (c *ProviderConfig) isSupportedAPI(apiName ApiName) bool {
|
||||
return exist
|
||||
}
|
||||
|
||||
func (c *ProviderConfig) IsSupportedAPI(apiName ApiName) bool {
|
||||
return c.isSupportedAPI(apiName)
|
||||
}
|
||||
|
||||
func (c *ProviderConfig) setDefaultCapabilities(capabilities map[string]string) {
|
||||
for capability, path := range capabilities {
|
||||
c.capabilities[capability] = path
|
||||
@@ -855,8 +859,22 @@ func (c *ProviderConfig) handleRequestBody(
|
||||
return types.ActionContinue, nil
|
||||
}
|
||||
|
||||
// use openai protocol
|
||||
var err error
|
||||
|
||||
// handle claude protocol input - auto-detect based on conversion marker
|
||||
// If main.go detected a Claude request that needs conversion, convert the body
|
||||
needClaudeConversion, _ := ctx.GetContext("needClaudeResponseConversion").(bool)
|
||||
if needClaudeConversion {
|
||||
// Convert Claude protocol to OpenAI protocol
|
||||
converter := &ClaudeToOpenAIConverter{}
|
||||
body, err = converter.ConvertClaudeRequestToOpenAI(body)
|
||||
if err != nil {
|
||||
return types.ActionContinue, fmt.Errorf("failed to convert claude request to openai: %v", err)
|
||||
}
|
||||
log.Debugf("[Auto Protocol] converted Claude request body to OpenAI format")
|
||||
}
|
||||
|
||||
// use openai protocol (either original openai or converted from claude)
|
||||
if handler, ok := provider.(TransformRequestBodyHandler); ok {
|
||||
body, err = handler.TransformRequestBody(ctx, apiName, body)
|
||||
} else if handler, ok := provider.(TransformRequestBodyHeadersHandler); ok {
|
||||
|
||||
Reference in New Issue
Block a user