jiazhizhong/higress
1
0
Fork 0
mirror of https://github.com/alibaba/higress.git synced 2026-06-08 12:17:28 +08:00
Code Issues Packages Projects Releases Wiki Activity

update plugins doc (#1305)

Browse Source
This commit is contained in:
澄潭
2024-09-12 21:48:40 +08:00
committed by GitHub
parent 0f9113ed82
commit c7c4ae1da2
80 changed files with 7373 additions and 2368 deletions
Download Patch File Download Diff File Expand all files Collapse all files

21
plugins/wasm-go/extensions/ai-intent/README.md
View File

@@ -1,15 +1,20 @@
## 简介
---
title: AI 意图识别
keywords: [ AI网关, AI意图识别 ]
description: AI 意图识别插件配置参考
---
**Note**
> 需要数据面的proxy wasm版本大于等于0.2.100
> 编译时，需要带上版本的tag，例如：`tinygo build -o main.wasm -scheduler=none -target=wasi -gc=custom -tags="custommalloc nottinygc_finalizer proxy_wasm_version_0_2_100" ./`
## 功能说明
LLM 意图识别插件，能够智能判断用户请求与某个领域或agent的功能契合度，从而提升不同模型的应用效果和用户体验
## 运行属性
插件执行阶段：`默认阶段`
插件执行优先级：`700`
## 配置说明
> 1.该插件的优先级要高于ai-cache、ai-proxy等后续使用意图的插件，后续插件可以通过proxywasm.GetProperty([]string{"intent_category"})方法获取到意图主题，按照意图主题去做不同缓存库或者大模型的选择
> 1.该插件的优先级高于ai-proxy等后续使用意图的插件，后续插件可以通过proxywasm.GetProperty([]string{"intent_category"})方法获取到意图主题，按照意图主题去做不同缓存库或者大模型的选择
> 2.需新建一条higress的大模型路由，供该插件访问大模型,如：路由以 /intent 作为前缀，服务选择大模型服务，为该路由开启ai-proxy插件
@@ -19,7 +24,7 @@ LLM 意图识别插件，能够智能判断用户请求与某个领域或agent
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| -------------- | --------------- | -------- | ------ | ------------------------------------------------------------ |
| `scene.category` | string | 必填 | - | 预设场景类别，以"|"分割，如："金融|电商|法律|Higress"|
| `scene.category` | string | 必填 | - | 预设场景类别，以`|`分割，如：`金融|电商|法律|Higress`|
| `scene.prompt` | string | 非必填 | 你是一个智能类别识别助手，负责根据用户提出的问题和预设的类别，确定问题属于哪个预设的类别，并给出相应的类别。用户提出的问题为:%s,预设的类别为%s，直接返回一种具体类别，如果没有找到就返回'NotFound'。 | llm请求prompt模板 |
| `llm.proxyServiceName` | string | 必填 | - | 新建的higress服务，指向大模型 (取higress中的 FQDN 值)|
| `llm.proxyUrl` | string | 必填 | - | 大模型路由请求地址全路径，可以是网关自身的地址，也可以是其他大模型的地址（openai协议），例如：http://127.0.0.1:80/intent/compatible-mode/v1/chat/completions |

45
plugins/wasm-go/extensions/ai-intent/README_EN.md Normal file
View File

@@ -0,0 +1,45 @@
---
title: AI Intent Recognition
keywords: [ AI Gateway, AI Intent Recognition ]
description: AI Intent Recognition Plugin Configuration Reference
---
## Function Description
LLM Intent Recognition plugin can intelligently determine the alignment between user requests and the functionalities of a certain domain or agent, thereby enhancing the application effectiveness of different models and user experience.
## Execution Attributes
Plugin execution phase: `Default Phase`
Plugin execution priority: `700`
## Configuration Instructions
> 1. This plugin's priority is higher than that of plugins such as ai-proxy which follow up and use intent. Subsequent plugins can retrieve the intent category using the proxywasm.GetProperty([]string{"intent_category"}) method and make selections for different cache libraries or large models based on the intent category.
> 2. A new Higress large model route needs to be created to allow this plugin to access the large model. For example: the route should use `/intent` as a prefix, the service should select the large model service, and the ai-proxy plugin should be enabled for this route.
> 3. A fixed-address service needs to be created (for example, intent-service), which points to 127.0.0.1:80 (i.e., the gateway instance and port). The ai-intent plugin requires this service for calling to access the newly added route. The service name corresponds to llm.proxyServiceName (a DNS type service can also be created to allow the plugin to access other large models).
> 4. If using a fixed-address service to call the gateway itself, 127.0.0.1 must be added to the gateway's access whitelist.
| Name | Data Type | Requirement | Default Value | Description |
| -------------- | --------------- | ----------- | ------------- | --------------------------------------------------------------- |
| `scene.category` | string | Required | - | Preset scene categories, separated by "|", e.g.: "Finance|E-commerce|Law|Higress" |
| `scene.prompt` | string | Optional | You are a smart category recognition assistant responsible for determining which preset category a user’s question belongs to based on the question posed by the user and the preset categories, and returning the corresponding category. The user's question is: %s, the preset categories are %s, directly return a specific category; if not found, return 'NotFound'. | llm request prompt template |
| `llm.proxyServiceName` | string | Required | - | Newly created Higress service pointing to the large model (use the FQDN value from Higress) |
| `llm.proxyUrl` | string | Required | - | The full path to the large model route request address, which can be the gateway’s own address or the address of another large model (OpenAI protocol), for example: http://127.0.0.1:80/intent/compatible-mode/v1/chat/completions |
| `llm.proxyDomain` | string | Optional | Retrieved from proxyUrl | Domain of the large model service |
| `llm.proxyPort` | string | Optional | Retrieved from proxyUrl | Port number of the large model service |
| `llm.proxyApiKey` | string | Optional | - | API_KEY corresponding to the external large model service when using it |
| `llm.proxyModel` | string | Optional | qwen-long | Type of the large model |
| `llm.proxyTimeout` | number | Optional | 10000 | Timeout for calling the large model, unit ms, default: 10000ms |
## Configuration Example
```yaml
scene:
category: "Finance|E-commerce|Law|Higress"
prompt: "You are a smart category recognition assistant responsible for determining which preset category a user's question belongs to based on the question posed by the user and the preset categories, and returning the corresponding category. The user's question is: '%s', the preset categories are '%s', directly return a specific category; if not found, return 'NotFound'."
llm:
proxyServiceName: "intent-service.static"
proxyUrl: "http://127.0.0.1:80/intent/compatible-mode/v1/chat/completions"
proxyDomain: "127.0.0.1"
proxyPort: "80"
proxyModel: "qwen-long"
proxyApiKey: ""
proxyTimeout: "10000"
```