mirror of
https://github.com/alibaba/higress.git
synced 2026-03-03 08:00:49 +08:00
963 lines
24 KiB
Markdown
963 lines
24 KiB
Markdown
---
|
||
title: AI 代理
|
||
keywords: [ higress,ai,proxy,rag ]
|
||
description: AI 代理插件配置参考
|
||
---
|
||
|
||
## 功能说明
|
||
|
||
`AI 代理`插件实现了基于 OpenAI API 契约的 AI 代理功能。目前支持 OpenAI、Azure OpenAI、月之暗面（Moonshot）和通义千问等 AI
|
||
服务提供商。
|
||
|
||
## 配置字段
|
||
|
||
### 基本配置
|
||
|
||
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|
||
|------------|--------|------|-----|------------------|
|
||
| `provider` | object | 必填 | - | 配置目标 AI 服务提供商的信息 |
|
||
|
||
`provider`的配置字段说明如下：
|
||
|
||
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|
||
| -------------- | --------------- | -------- | ------ | ------------------------------------------------------------ |
|
||
| `type` | string | 必填 | - | AI 服务提供商名称 |
|
||
| `apiTokens` | array of string | 必填 | - | 用于在访问 AI 服务时进行认证的令牌。如果配置了多个 token，插件会在请求时随机进行选择。部分服务提供商只支持配置一个 token。 |
|
||
| `timeout` | number | 非必填 | - | 访问 AI 服务的超时时间。单位为毫秒。默认值为 120000，即 2 分钟 |
|
||
| `modelMapping` | map of string | 非必填 | - | AI 模型映射表，用于将请求中的模型名称映射为服务提供商支持模型名称。<br/>可以使用 "*" 为键来配置通用兜底映射关系 |
|
||
| `protocol` | string | 非必填 | - | 插件对外提供的 API 接口契约。目前支持以下取值：openai（默认值，使用 OpenAI 的接口契约）、original（使用目标服务提供商的原始接口契约） |
|
||
| `context` | object | 非必填 | - | 配置 AI 对话上下文信息 |
|
||
|
||
`context`的配置字段说明如下：
|
||
|
||
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|
||
|---------------|--------|------|-----|----------------------------------|
|
||
| `fileUrl` | string | 必填 | - | 保存 AI 对话上下文的文件 URL。仅支持纯文本类型的文件内容 |
|
||
| `serviceName` | string | 必填 | - | URL 所对应的 Higress 后端服务完整名称 |
|
||
| `servicePort` | number | 必填 | - | URL 所对应的 Higress 后端服务访问端口 |
|
||
|
||
### 提供商特有配置
|
||
|
||
#### OpenAI
|
||
|
||
OpenAI 所对应的 `type` 为 `openai`。它并无特有的配置字段。
|
||
|
||
#### Azure OpenAI
|
||
|
||
Azure OpenAI 所对应的 `type` 为 `azure`。它特有的配置字段如下：
|
||
|
||
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|
||
|-------------------|--------|------|-----|----------------------------------------------|
|
||
| `azureServiceUrl` | string | 必填 | - | Azure OpenAI 服务的 URL，须包含 `api-version` 查询参数。 |
|
||
|
||
**注意：** Azure OpenAI 只支持配置一个 API Token。
|
||
|
||
#### 月之暗面（Moonshot）
|
||
|
||
月之暗面所对应的 `type` 为 `moonshot`。它特有的配置字段如下：
|
||
|
||
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|
||
|------------------|--------|------|-----|-------------------------------------------------------------|
|
||
| `moonshotFileId` | string | 非必填 | - | 通过文件接口上传至月之暗面的文件 ID，其内容将被用做 AI 对话的上下文。不可与 `context` 字段同时配置。 |
|
||
|
||
#### 通义千问（Qwen）
|
||
|
||
通义千问所对应的 `type` 为 `qwen`。它特有的配置字段如下：
|
||
|
||
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|
||
|--------------------|-----------------|------|-----|------------------------------------------------------------------|
|
||
| `qwenEnableSearch` | boolean | 非必填 | - | 是否启用通义千问内置的互联网搜索功能。 |
|
||
| `qwenFileIds` | array of string | 非必填 | - | 通过文件接口上传至Dashscope的文件 ID，其内容将被用做 AI 对话的上下文。不可与 `context` 字段同时配置。 |
|
||
|
||
#### 百川智能 (Baichuan AI)
|
||
|
||
百川智能所对应的 `type` 为 `baichuan` 。它并无特有的配置字段。
|
||
|
||
#### 零一万物（Yi）
|
||
|
||
零一万物所对应的 `type` 为 `yi`。它并无特有的配置字段。
|
||
|
||
#### 智谱AI（Zhipu AI）
|
||
|
||
智谱AI所对应的 `type` 为 `zhipuai`。它并无特有的配置字段。
|
||
|
||
#### DeepSeek（DeepSeek）
|
||
|
||
DeepSeek所对应的 `type` 为 `deepseek`。它并无特有的配置字段。
|
||
|
||
#### Groq
|
||
|
||
Groq 所对应的 `type` 为 `groq`。它并无特有的配置字段。
|
||
|
||
#### 文心一言（Baidu）
|
||
|
||
文心一言所对应的 `type` 为 `baidu`。它并无特有的配置字段。
|
||
|
||
#### MiniMax
|
||
|
||
MiniMax所对应的 `type` 为 `minimax`。它特有的配置字段如下：
|
||
|
||
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|
||
| ---------------- | -------- | ------------------------------------------------------------ | ------ | ------------------------------------------------------------ |
|
||
| `minimaxGroupId` | string | 当使用`abab6.5-chat`, `abab6.5s-chat`, `abab5.5s-chat`, `abab5.5-chat`四种模型时必填 | - | 当使用`abab6.5-chat`, `abab6.5s-chat`, `abab5.5s-chat`, `abab5.5-chat`四种模型时会使用ChatCompletion Pro，需要设置groupID |
|
||
|
||
#### Anthropic Claude
|
||
|
||
Anthropic Claude 所对应的 `type` 为 `claude`。它特有的配置字段如下：
|
||
|
||
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|
||
|-----------|--------|------|-----|----------------------------------|
|
||
| `claudeVersion` | string | 可选 | - | Claude 服务的 API 版本，默认为 2023-06-01 |
|
||
|
||
#### Ollama
|
||
|
||
Ollama 所对应的 `type` 为 `ollama`。它特有的配置字段如下：
|
||
|
||
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|
||
|-------------------|--------|------|-----|----------------------------------------------|
|
||
| `ollamaServerHost` | string | 必填 | - | Ollama 服务器的主机地址 |
|
||
| `ollamaServerPort` | number | 必填 | - | Ollama 服务器的端口号，默认为11434 |
|
||
|
||
#### 混元
|
||
|
||
混元所对应的 `type` 为 `hunyuan`。它特有的配置字段如下：
|
||
|
||
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|
||
|-------------------|--------|------|-----|----------------------------------------------|
|
||
| `hunyuanAuthId` | string | 必填 | - | 混元用于v3版本认证的id |
|
||
| `hunyuanAuthKey` | string | 必填 | - | 混元用于v3版本认证的key |
|
||
|
||
#### 阶跃星辰 (Stepfun)
|
||
|
||
阶跃星辰所对应的 `type` 为 `stepfun`。它并无特有的配置字段。
|
||
|
||
## 用法示例
|
||
|
||
### 使用 OpenAI 协议代理 Azure OpenAI 服务
|
||
|
||
使用最基本的 Azure OpenAI 服务，不配置任何上下文。
|
||
|
||
**配置信息**
|
||
|
||
```yaml
|
||
provider:
|
||
type: azure
|
||
apiTokens:
|
||
- "YOUR_AZURE_OPENAI_API_TOKEN"
|
||
azureServiceUrl: "https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT_NAME/chat/completions?api-version=2024-02-15-preview",
|
||
```
|
||
|
||
**请求示例**
|
||
|
||
```json
|
||
{
|
||
"model": "gpt-3",
|
||
"messages": [
|
||
{
|
||
"role": "user",
|
||
"content": "你好，你是谁？"
|
||
}
|
||
],
|
||
"temperature": 0.3
|
||
}
|
||
```
|
||
|
||
**响应示例**
|
||
|
||
```json
|
||
{
|
||
"choices": [
|
||
{
|
||
"content_filter_results": {
|
||
"hate": {
|
||
"filtered": false,
|
||
"severity": "safe"
|
||
},
|
||
"self_harm": {
|
||
"filtered": false,
|
||
"severity": "safe"
|
||
},
|
||
"sexual": {
|
||
"filtered": false,
|
||
"severity": "safe"
|
||
},
|
||
"violence": {
|
||
"filtered": false,
|
||
"severity": "safe"
|
||
}
|
||
},
|
||
"finish_reason": "stop",
|
||
"index": 0,
|
||
"logprobs": null,
|
||
"message": {
|
||
"content": "你好！我是一个AI助手，可以回答你的问题和提供帮助。有什么我可以帮到你的吗？",
|
||
"role": "assistant"
|
||
}
|
||
}
|
||
],
|
||
"created": 1714807624,
|
||
"id": "chatcmpl-abcdefg1234567890",
|
||
"model": "gpt-35-turbo-16k",
|
||
"object": "chat.completion",
|
||
"prompt_filter_results": [
|
||
{
|
||
"prompt_index": 0,
|
||
"content_filter_results": {
|
||
"hate": {
|
||
"filtered": false,
|
||
"severity": "safe"
|
||
},
|
||
"self_harm": {
|
||
"filtered": false,
|
||
"severity": "safe"
|
||
},
|
||
"sexual": {
|
||
"filtered": false,
|
||
"severity": "safe"
|
||
},
|
||
"violence": {
|
||
"filtered": false,
|
||
"severity": "safe"
|
||
}
|
||
}
|
||
}
|
||
],
|
||
"system_fingerprint": null,
|
||
"usage": {
|
||
"completion_tokens": 40,
|
||
"prompt_tokens": 15,
|
||
"total_tokens": 55
|
||
}
|
||
}
|
||
```
|
||
|
||
### 使用 OpenAI 协议代理通义千问服务
|
||
|
||
使用通义千问服务，并配置从 OpenAI 大模型到通义千问的模型映射关系。
|
||
|
||
**配置信息**
|
||
|
||
```yaml
|
||
provider:
|
||
type: qwen
|
||
apiTokens:
|
||
- "YOUR_QWEN_API_TOKEN"
|
||
modelMapping:
|
||
'gpt-3': "qwen-turbo"
|
||
'gpt-35-turbo': "qwen-plus"
|
||
'gpt-4-turbo': "qwen-max"
|
||
'*': "qwen-turbo"
|
||
```
|
||
|
||
**请求示例**
|
||
|
||
```json
|
||
{
|
||
"model": "gpt-3",
|
||
"messages": [
|
||
{
|
||
"role": "user",
|
||
"content": "你好，你是谁？"
|
||
}
|
||
],
|
||
"temperature": 0.3
|
||
}
|
||
```
|
||
|
||
**响应示例**
|
||
|
||
```json
|
||
{
|
||
"id": "c2518bd3-0f46-97d1-be34-bb5777cb3108",
|
||
"choices": [
|
||
{
|
||
"index": 0,
|
||
"message": {
|
||
"role": "assistant",
|
||
"content": "我是通义千问，由阿里云开发的AI助手。我可以回答各种问题、提供信息和与用户进行对话。有什么我可以帮助你的吗？"
|
||
},
|
||
"finish_reason": "stop"
|
||
}
|
||
],
|
||
"created": 1715175072,
|
||
"model": "qwen-turbo",
|
||
"object": "chat.completion",
|
||
"usage": {
|
||
"prompt_tokens": 24,
|
||
"completion_tokens": 33,
|
||
"total_tokens": 57
|
||
}
|
||
}
|
||
```
|
||
|
||
### 使用通义千问配合纯文本上下文信息
|
||
|
||
使用通义千问服务，同时配置纯文本上下文信息。
|
||
|
||
**配置信息**
|
||
|
||
```yaml
|
||
provider:
|
||
type: qwen
|
||
apiTokens:
|
||
- "YOUR_QWEN_API_TOKEN"
|
||
modelMapping:
|
||
"*": "qwen-turbo"
|
||
context:
|
||
- fileUrl: "http://file.default.svc.cluster.local/ai/context.txt",
|
||
serviceName: "file.dns",
|
||
servicePort: 80
|
||
```
|
||
|
||
**请求示例**
|
||
|
||
```json
|
||
{
|
||
"model": "gpt-3",
|
||
"messages": [
|
||
{
|
||
"role": "user",
|
||
"content": "请概述文案内容"
|
||
}
|
||
],
|
||
"temperature": 0.3
|
||
}
|
||
```
|
||
|
||
**响应示例**
|
||
|
||
```json
|
||
{
|
||
"id": "cmpl-77861a17681f4987ab8270dbf8001936",
|
||
"object": "chat.completion",
|
||
"created": 9756990,
|
||
"model": "moonshot-v1-128k",
|
||
"choices": [
|
||
{
|
||
"index": 0,
|
||
"message": {
|
||
"role": "assistant",
|
||
"content": "这份文案是一份关于..."
|
||
},
|
||
"finish_reason": "stop"
|
||
}
|
||
],
|
||
"usage": {
|
||
"prompt_tokens": 20181,
|
||
"completion_tokens": 439,
|
||
"total_tokens": 20620
|
||
}
|
||
}
|
||
```
|
||
|
||
### 使用通义千问配合其原生的文件上下文
|
||
|
||
提前上传文件至通义千问，以文件内容作为上下文使用其 AI 服务。
|
||
|
||
**配置信息**
|
||
|
||
```yaml
|
||
provider:
|
||
type: qwen
|
||
apiTokens:
|
||
- "YOUR_QWEN_API_TOKEN"
|
||
modelMapping:
|
||
"*": "qwen-long" # 通义千问的文件上下文只能在 qwen-long 模型下使用
|
||
qwenFileIds:
|
||
- "file-fe-xxx"
|
||
- "file-fe-yyy"
|
||
```
|
||
|
||
**请求示例**
|
||
|
||
```json
|
||
{
|
||
"model": "gpt-4-turbo",
|
||
"messages": [
|
||
{
|
||
"role": "user",
|
||
"content": "请概述文案内容"
|
||
}
|
||
],
|
||
"temperature": 0.3
|
||
}
|
||
```
|
||
|
||
**响应示例**
|
||
|
||
```json
|
||
{
|
||
"output": {
|
||
"choices": [
|
||
{
|
||
"finish_reason": "stop",
|
||
"message": {
|
||
"role": "assistant",
|
||
"content": "您上传了两个文件，`context.txt` 和 `context_2.txt`，它们似乎都包含了关于xxxx"
|
||
}
|
||
}
|
||
]
|
||
},
|
||
"usage": {
|
||
"total_tokens": 2023,
|
||
"output_tokens": 530,
|
||
"input_tokens": 1493
|
||
},
|
||
"request_id": "187e99ba-5b64-9ffe-8f69-01dafbaf6ed7"
|
||
}
|
||
```
|
||
|
||
### 使用月之暗面配合其原生的文件上下文
|
||
|
||
提前上传文件至月之暗面，以文件内容作为上下文使用其 AI 服务。
|
||
|
||
**配置信息**
|
||
|
||
```yaml
|
||
provider:
|
||
type: moonshot
|
||
apiTokens:
|
||
- "YOUR_MOONSHOT_API_TOKEN"
|
||
moonshotFileId: "YOUR_MOONSHOT_FILE_ID",
|
||
modelMapping:
|
||
'*': "moonshot-v1-32k"
|
||
```
|
||
|
||
**请求示例**
|
||
|
||
```json
|
||
{
|
||
"model": "gpt-4-turbo",
|
||
"messages": [
|
||
{
|
||
"role": "user",
|
||
"content": "请概述文案内容"
|
||
}
|
||
],
|
||
"temperature": 0.3
|
||
}
|
||
```
|
||
|
||
**响应示例**
|
||
|
||
```json
|
||
{
|
||
"id": "cmpl-e5ca873642ca4f5d8b178c1742f9a8e8",
|
||
"object": "chat.completion",
|
||
"created": 1872961,
|
||
"model": "moonshot-v1-128k",
|
||
"choices": [
|
||
{
|
||
"index": 0,
|
||
"message": {
|
||
"role": "assistant",
|
||
"content": "文案内容是关于一个名为“xxxx”的支付平台..."
|
||
},
|
||
"finish_reason": "stop"
|
||
}
|
||
],
|
||
"usage": {
|
||
"prompt_tokens": 11,
|
||
"completion_tokens": 498,
|
||
"total_tokens": 509
|
||
}
|
||
}
|
||
```
|
||
|
||
### 使用 OpenAI 协议代理 Groq 服务
|
||
|
||
**配置信息**
|
||
|
||
```yaml
|
||
provider:
|
||
type: groq
|
||
apiTokens:
|
||
- "YOUR_GROQ_API_TOKEN"
|
||
```
|
||
|
||
**请求示例**
|
||
|
||
```json
|
||
{
|
||
"model": "llama3-8b-8192",
|
||
"messages": [
|
||
{
|
||
"role": "user",
|
||
"content": "你好，你是谁？"
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
**响应示例**
|
||
|
||
```json
|
||
{
|
||
"id": "chatcmpl-26733989-6c52-4056-b7a9-5da791bd7102",
|
||
"object": "chat.completion",
|
||
"created": 1715917967,
|
||
"model": "llama3-8b-8192",
|
||
"choices": [
|
||
{
|
||
"index": 0,
|
||
"message": {
|
||
"role": "assistant",
|
||
"content": "😊 Ni Hao! (That's \"hello\" in Chinese!)\n\nI am LLaMA, an AI assistant developed by Meta AI that can understand and respond to human input in a conversational manner. I'm not a human, but a computer program designed to simulate conversations and answer questions to the best of my ability. I'm happy to chat with you in Chinese or help with any questions or topics you'd like to discuss! 😊"
|
||
},
|
||
"logprobs": null,
|
||
"finish_reason": "stop"
|
||
}
|
||
],
|
||
"usage": {
|
||
"prompt_tokens": 16,
|
||
"prompt_time": 0.005,
|
||
"completion_tokens": 89,
|
||
"completion_time": 0.104,
|
||
"total_tokens": 105,
|
||
"total_time": 0.109
|
||
},
|
||
"system_fingerprint": "fp_dadc9d6142",
|
||
"x_groq": {
|
||
"id": "req_01hy2awmcxfpwbq56qh6svm7qz"
|
||
}
|
||
}
|
||
```
|
||
|
||
### 使用 OpenAI 协议代理 Claude 服务
|
||
|
||
**配置信息**
|
||
|
||
```yaml
|
||
provider:
|
||
type: claude
|
||
apiTokens:
|
||
- "YOUR_CLAUDE_API_TOKEN"
|
||
version: "2023-06-01"
|
||
```
|
||
|
||
**请求示例**
|
||
|
||
```json
|
||
{
|
||
"model": "claude-3-opus-20240229",
|
||
"max_tokens": 1024,
|
||
"messages": [
|
||
{
|
||
"role": "user",
|
||
"content": "你好，你是谁？"
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
**响应示例**
|
||
|
||
```json
|
||
{
|
||
"id": "msg_01Jt3GzyjuzymnxmZERJguLK",
|
||
"choices": [
|
||
{
|
||
"index": 0,
|
||
"message": {
|
||
"role": "assistant",
|
||
"content": "您好,我是一个由人工智能公司Anthropic开发的聊天助手。我的名字叫Claude,是一个聪明友善、知识渊博的对话系统。很高兴认识您!我可以就各种话题与您聊天,回答问题,提供建议和帮助。我会尽最大努力给您有帮助的回复。希望我们能有个愉快的交流!"
|
||
},
|
||
"finish_reason": "stop"
|
||
}
|
||
],
|
||
"created": 1717385918,
|
||
"model": "claude-3-opus-20240229",
|
||
"object": "chat.completion",
|
||
"usage": {
|
||
"prompt_tokens": 16,
|
||
"completion_tokens": 126,
|
||
"total_tokens": 142
|
||
}
|
||
}
|
||
```
|
||
### 使用 OpenAI 协议代理混元服务
|
||
|
||
**配置信息**
|
||
|
||
```yaml
|
||
provider:
|
||
type: "hunyuan"
|
||
hunyuanAuthKey: "<YOUR AUTH KEY>"
|
||
apiTokens:
|
||
- ""
|
||
hunyuanAuthId: "<YOUR AUTH ID>"
|
||
timeout: 1200000
|
||
modelMapping:
|
||
"*": "hunyuan-lite"
|
||
```
|
||
|
||
**请求示例**
|
||
请求脚本：
|
||
```sh
|
||
|
||
curl --location 'http://<your higress domain>/v1/chat/completions' \
|
||
--header 'Content-Type: application/json' \
|
||
--data '{
|
||
"model": "gpt-3",
|
||
"messages": [
|
||
{
|
||
"role": "system",
|
||
"content": "你是一个名专业的开发人员！"
|
||
},
|
||
{
|
||
"role": "user",
|
||
"content": "你好，你是谁？"
|
||
}
|
||
],
|
||
"temperature": 0.3,
|
||
"stream": false
|
||
}'
|
||
```
|
||
|
||
**响应示例**
|
||
|
||
```json
|
||
{
|
||
"id": "fd140c3e-0b69-4b19-849b-d354d32a6162",
|
||
"choices": [
|
||
{
|
||
"index": 0,
|
||
"delta": {
|
||
"role": "assistant",
|
||
"content": "你好！我是一名专业的开发人员。"
|
||
},
|
||
"finish_reason": "stop"
|
||
}
|
||
],
|
||
"created": 1717493117,
|
||
"model": "hunyuan-lite",
|
||
"object": "chat.completion",
|
||
"usage": {
|
||
"prompt_tokens": 15,
|
||
"completion_tokens": 9,
|
||
"total_tokens": 24
|
||
}
|
||
}
|
||
```
|
||
|
||
### 使用 OpenAI 协议代理百度文心一言服务
|
||
|
||
**配置信息**
|
||
|
||
```yaml
|
||
provider:
|
||
type: baidu
|
||
apiTokens:
|
||
- "YOUR_BAIDU_API_TOKEN"
|
||
modelMapping:
|
||
'gpt-3': "ERNIE-4.0"
|
||
'*': "ERNIE-4.0"
|
||
```
|
||
|
||
**请求示例**
|
||
|
||
```json
|
||
{
|
||
"model": "gpt-4-turbo",
|
||
"messages": [
|
||
{
|
||
"role": "user",
|
||
"content": "你好，你是谁？"
|
||
}
|
||
],
|
||
"stream": false
|
||
}
|
||
```
|
||
|
||
**响应示例**
|
||
|
||
```json
|
||
{
|
||
"id": "as-e90yfg1pk1",
|
||
"choices": [
|
||
{
|
||
"index": 0,
|
||
"message": {
|
||
"role": "assistant",
|
||
"content": "你好，我是文心一言，英文名是ERNIE Bot。我能够与人对话互动，回答问题，协助创作，高效便捷地帮助人们获取信息、知识和灵感。"
|
||
},
|
||
"finish_reason": "stop"
|
||
}
|
||
],
|
||
"created": 1717251488,
|
||
"model": "ERNIE-4.0",
|
||
"object": "chat.completion",
|
||
"usage": {
|
||
"prompt_tokens": 4,
|
||
"completion_tokens": 33,
|
||
"total_tokens": 37
|
||
}
|
||
}
|
||
```
|
||
|
||
### 使用 OpenAI 协议代理MiniMax服务
|
||
|
||
**配置信息**
|
||
|
||
```yaml
|
||
provider:
|
||
type: minimax
|
||
apiTokens:
|
||
- "YOUR_MINIMAX_API_TOKEN"
|
||
modelMapping:
|
||
"gpt-3": "abab6.5g-chat"
|
||
"gpt-4": "abab6.5-chat"
|
||
"*": "abab6.5g-chat"
|
||
minimaxGroupId: "YOUR_MINIMAX_GROUP_ID"
|
||
```
|
||
|
||
**请求示例**
|
||
|
||
```json
|
||
{
|
||
"model": "gpt-4-turbo",
|
||
"messages": [
|
||
{
|
||
"role": "user",
|
||
"content": "你好，你是谁？"
|
||
}
|
||
],
|
||
"stream": false
|
||
}
|
||
```
|
||
|
||
**响应示例**
|
||
|
||
```json
|
||
{
|
||
"id": "02b2251f8c6c09d68c1743f07c72afd7",
|
||
"choices": [
|
||
{
|
||
"finish_reason": "stop",
|
||
"index": 0,
|
||
"message": {
|
||
"content": "你好！我是MM智能助理，一款由MiniMax自研的大型语言模型。我可以帮助你解答问题，提供信息，进行对话等。有什么可以帮助你的吗？",
|
||
"role": "assistant"
|
||
}
|
||
}
|
||
],
|
||
"created": 1717760544,
|
||
"model": "abab6.5s-chat",
|
||
"object": "chat.completion",
|
||
"usage": {
|
||
"total_tokens": 106
|
||
},
|
||
"input_sensitive": false,
|
||
"output_sensitive": false,
|
||
"input_sensitive_type": 0,
|
||
"output_sensitive_type": 0,
|
||
"base_resp": {
|
||
"status_code": 0,
|
||
"status_msg": ""
|
||
}
|
||
}
|
||
```
|
||
|
||
## 完整配置示例
|
||
|
||
### Kubernetes 示例
|
||
|
||
以下以使用 OpenAI 协议代理 Groq 服务为例，展示完整的插件配置示例。
|
||
|
||
```yaml
|
||
apiVersion: extensions.higress.io/v1alpha1
|
||
kind: WasmPlugin
|
||
metadata:
|
||
name: ai-proxy-groq
|
||
namespace: higress-system
|
||
spec:
|
||
matchRules:
|
||
- config:
|
||
provider:
|
||
type: groq
|
||
apiTokens:
|
||
- "YOUR_API_TOKEN"
|
||
ingress:
|
||
- groq
|
||
url: oci://higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/ai-proxy:1.0.0
|
||
---
|
||
apiVersion: networking.k8s.io/v1
|
||
kind: Ingress
|
||
metadata:
|
||
annotations:
|
||
higress.io/backend-protocol: HTTPS
|
||
higress.io/destination: groq.dns
|
||
higress.io/proxy-ssl-name: api.groq.com
|
||
higress.io/proxy-ssl-server-name: "on"
|
||
labels:
|
||
higress.io/resource-definer: higress
|
||
name: groq
|
||
namespace: higress-system
|
||
spec:
|
||
ingressClassName: higress
|
||
rules:
|
||
- host: <YOUR-DOMAIN>
|
||
http:
|
||
paths:
|
||
- backend:
|
||
resource:
|
||
apiGroup: networking.higress.io
|
||
kind: McpBridge
|
||
name: default
|
||
path: /
|
||
pathType: Prefix
|
||
---
|
||
apiVersion: networking.higress.io/v1
|
||
kind: McpBridge
|
||
metadata:
|
||
name: default
|
||
namespace: higress-system
|
||
spec:
|
||
registries:
|
||
- domain: api.groq.com
|
||
name: groq
|
||
port: 443
|
||
type: dns
|
||
```
|
||
|
||
访问示例：
|
||
|
||
```bash
|
||
curl "http://<YOUR-DOMAIN>/v1/chat/completions" -H "Content-Type: application/json" -d '{
|
||
"model": "llama3-8b-8192",
|
||
"messages": [
|
||
{
|
||
"role": "user",
|
||
"content": "你好，你是谁？"
|
||
}
|
||
]
|
||
}'
|
||
```
|
||
|
||
### Docker-Compose 示例
|
||
|
||
`docker-compose.yml` 配置文件：
|
||
|
||
```yaml
|
||
version: '3.7'
|
||
services:
|
||
envoy:
|
||
image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/envoy:1.20
|
||
entrypoint: /usr/local/bin/envoy
|
||
# 开启了 debug 级别日志方便调试
|
||
command: -c /etc/envoy/envoy.yaml --component-log-level wasm:debug
|
||
networks:
|
||
- higress-net
|
||
ports:
|
||
- "10000:10000"
|
||
volumes:
|
||
- ./envoy.yaml:/etc/envoy/envoy.yaml
|
||
- ./plugin.wasm:/etc/envoy/plugin.wasm
|
||
networks:
|
||
higress-net: {}
|
||
```
|
||
|
||
`envoy.yaml` 配置文件：
|
||
|
||
```yaml
|
||
admin:
|
||
address:
|
||
socket_address:
|
||
protocol: TCP
|
||
address: 0.0.0.0
|
||
port_value: 9901
|
||
static_resources:
|
||
listeners:
|
||
- name: listener_0
|
||
address:
|
||
socket_address:
|
||
protocol: TCP
|
||
address: 0.0.0.0
|
||
port_value: 10000
|
||
filter_chains:
|
||
- filters:
|
||
- name: envoy.filters.network.http_connection_manager
|
||
typed_config:
|
||
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
|
||
scheme_header_transformation:
|
||
scheme_to_overwrite: https
|
||
stat_prefix: ingress_http
|
||
# Output envoy logs to stdout
|
||
access_log:
|
||
- name: envoy.access_loggers.stdout
|
||
typed_config:
|
||
"@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
|
||
# Modify as required
|
||
route_config:
|
||
name: local_route
|
||
virtual_hosts:
|
||
- name: local_service
|
||
domains: [ "*" ]
|
||
routes:
|
||
- match:
|
||
prefix: "/"
|
||
route:
|
||
cluster: claude
|
||
timeout: 300s
|
||
http_filters:
|
||
- name: claude
|
||
typed_config:
|
||
"@type": type.googleapis.com/udpa.type.v1.TypedStruct
|
||
type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
|
||
value:
|
||
config:
|
||
name: claude
|
||
vm_config:
|
||
runtime: envoy.wasm.runtime.v8
|
||
code:
|
||
local:
|
||
filename: /etc/envoy/plugin.wasm
|
||
configuration:
|
||
"@type": "type.googleapis.com/google.protobuf.StringValue"
|
||
value: | # 插件配置
|
||
{
|
||
"provider": {
|
||
"type": "claude",
|
||
"apiTokens": [
|
||
"YOUR_API_TOKEN"
|
||
]
|
||
}
|
||
}
|
||
- name: envoy.filters.http.router
|
||
clusters:
|
||
- name: claude
|
||
connect_timeout: 30s
|
||
type: LOGICAL_DNS
|
||
dns_lookup_family: V4_ONLY
|
||
lb_policy: ROUND_ROBIN
|
||
load_assignment:
|
||
cluster_name: claude
|
||
endpoints:
|
||
- lb_endpoints:
|
||
- endpoint:
|
||
address:
|
||
socket_address:
|
||
address: api.anthropic.com # API 服务地址
|
||
port_value: 443
|
||
transport_socket:
|
||
name: envoy.transport_sockets.tls
|
||
typed_config:
|
||
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
|
||
"sni": "api.anthropic.com"
|
||
```
|
||
|
||
访问示例：
|
||
|
||
```bash
|
||
curl "http://localhost:10000/v1/chat/completions" -H "Content-Type: application/json" -d '{
|
||
"model": "claude-3-opus-20240229",
|
||
"max_tokens": 1024,
|
||
"messages": [
|
||
{
|
||
"role": "user",
|
||
"content": "你好，你是谁？"
|
||
}
|
||
]
|
||
}'
|
||
```
|