add plugin: ai-statistics (#1011)

This commit is contained in:
rinfx
2024-06-18 17:53:04 +08:00
committed by GitHub
parent 4ca4bec2b5
commit 0785d4aac4
5 changed files with 236 additions and 0 deletions

View File

@@ -0,0 +1,44 @@
# 介绍
提供AI可观测基础能力其后需接ai-proxy插件如果不接ai-proxy插件的话则只支持openai协议。
# 配置说明
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|------------|--------|------|-----|------------------|
| `enable` | bool | 必填 | - | 是否开启ai统计功能 |
开启后 metrics 示例:
```
route_upstream_model_input_token{ai_route="openai",ai_cluster="qwen",ai_model="qwen-max"} 21
route_upstream_model_output_token{ai_route="openai",ai_cluster="qwen",ai_model="qwen-max"} 17
```
日志示例:
```json
{
"model": "qwen-max",
"input_token": "21",
"output_token": "17",
"authority": "dashscope.aliyuncs.com",
"bytes_received": "336",
"bytes_sent": "1675",
"duration": "1590",
"istio_policy_status": "-",
"method": "POST",
"path": "/v1/chat/completions",
"protocol": "HTTP/1.1",
"request_id": "5895f5a9-e4e3-425b-98db-6c6a926195b7",
"requested_server_name": "-",
"response_code": "200",
"response_flags": "-",
"route_name": "openai",
"start_time": "2024-06-18T09:37:14.078Z",
"trace_id": "-",
"upstream_cluster": "qwen",
"upstream_service_time": "496",
"upstream_transport_failure_reason": "-",
"user_agent": "PostmanRuntime/7.37.3",
"x_forwarded_for": "-"
}
```