diff --git a/plugins/wasm-go/extensions/ai-agent/README.md b/plugins/wasm-go/extensions/ai-agent/README.md new file mode 100644 index 000000000..b1fbcac39 --- /dev/null +++ b/plugins/wasm-go/extensions/ai-agent/README.md @@ -0,0 +1,251 @@ +--- +title: AI Agent +keywords: [ AI网关, AI Agent ] +description: AI Agent插件配置参考 +--- + +## 功能说明 +一个可定制化的 API AI Agent，目前第一版本只支持配置 http method 类型为 GET 的 API，且只支持非流式模式。agent流程图如下： +![ai-agent](https://github.com/user-attachments/assets/b0761a0c-1afa-496c-a98e-bb9f38b340f8) + +由于 Agent 是多轮对话场景，需要维护历史对话记录，本版本目前是在内存中维护历史对话记录，因此只支持单机。后续会支持通过 redis 存储回话记录 + +## 配置字段 + +### 基本配置 +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +|------------------|-----------|---------|--------|----------------------------| +| `llm` | object | 必填 | - | 配置 AI 服务提供商的信息 | +| `apis` | object | 必填 | - | 配置外部 API 服务提供商的信息 | +| `promptTemplate` | object | 非必填 | - | 配置 Agent ReAct 模板的信息 | + +`llm`的配置字段说明如下： + +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +|-----------------|-----------|---------|--------|-----------------------------------| +| `apiKey` | string | 必填 | - | 用于在访问大模型服务时进行认证的令牌。| +| `serviceName` | string | 必填 | - | 大模型服务名 | +| `servicePort` | int | 必填 | - | 大模型服务端口 | +| `domain` | string | 必填 | - | 访问大模型服务时域名 | +| `path` | string | 必填 | - | 访问大模型服务时路径 | +| `model` | string | 必填 | - | 访问大模型服务时模型名 | + +`apis`的配置字段说明如下： + +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +|-----------------|-----------|---------|--------|-----------------------------------| +| `apiProvider` | object | 必填 | - | 外部 API 服务信息 | +| `api` | string | 必填 | - | 工具的 OpenAPI 文档 | + +`apiProvider`的配置字段说明如下： + +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +|-----------------|-----------|---------|--------|------------------------------------------| +| `apiKey` | object | 非必填 | - | 用于在访问外部 API 服务时进行认证的令牌。 | +| `serviceName` | string | 必填 | - | 访问外部 API 服务名 | +| `servicePort` | int | 必填 | - | 访问外部 API 服务端口 | +| `domain` | string | 必填 | - | 访访问外部 API 时域名 | + +`apiKey`的配置字段说明如下： + +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +|-------------------|---------|------------|--------|-------------------------------------------------------------------------------| +| `in` | string | 非必填 | header | 在访问外部 API 服务时进行认证的令牌是放在 header 中还是放在 query 中，默认是 header。 +| `name` | string | 非必填 | - | 用于在访问外部 API 服务时进行认证的令牌的名称。 | +| `value` | string | 非必填 | - | 用于在访问外部 API 服务时进行认证的令牌的值。 | + +`promptTemplate`的配置字段说明如下： + +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +|-----------------|-----------|-----------|--------|--------------------------------------------| +| `language` | string | 非必填 | EN | Agent ReAct 模板的语言类型，包括 CH 和 EN 两种| +| `chTemplate` | object | 非必填 | - | Agent ReAct 中文模板 | +| `enTemplate` | object | 非必填 | - | Agent ReAct 英文模板 | + +`chTemplate`和`enTemplate`的配置字段说明如下： + +| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 | +|-----------------|-----------|-----------|--------|---------------------------------------------| +| `question` | string | 非必填 | - | Agent ReAct 模板的 question 部分 | +| `thought1` | string | 非必填 | - | Agent ReAct 模板的 thought1 部分 | +| `actionInput` | string | 非必填 | - | Agent ReAct 模板的 actionInput 部分 | +| `observation` | string | 非必填 | - | Agent ReAct 模板的 observation 部分 | +| `thought2` | string | 非必填 | - | Agent ReAct 模板的 thought2 部分 | +| `finalAnswer` | string | 非必填 | - | Agent ReAct 模板的 finalAnswer 部分 | +| `begin` | string | 非必填 | - | Agent ReAct 模板的 begin 部分 | + +## 用法示例 + +**配置信息** + +```yaml +llm: + apiKey: xxxxxxxxxxxxxxxxxx + domain: dashscope.aliyuncs.com + serviceName: dashscope.dns + servicePort: 443 + path: /compatible-mode/v1/chat/completions + model: qwen-max-0403 +promptTemplate: + language: CH +apis: +- apiProvider: + domain: restapi.amap.com + serviceName: geo.dns + servicePort: 80 + apiKey: + in: query + name: key + value: xxxxxxxxxxxxxxx + api: | + openapi: 3.1.0 + info: + title: 高德地图 + description: 获取 POI 的相关信息 + version: v1.0.0 + servers: + - url: https://restapi.amap.com + paths: + /v5/place/text: + get: + description: 根据POI名称，获得POI的经纬度坐标 + operationId: get_location_coordinate + parameters: + - name: keywords + in: query + description: POI名称，必须是中文 + required: true + schema: + type: string + - name: region + in: query + description: POI所在的区域名，必须是中文 + required: true + schema: + type: string + deprecated: false + /v5/place/around: + get: + description: 搜索给定坐标附近的POI + operationId: search_nearby_pois + parameters: + - name: keywords + in: query + description: 目标POI的关键字 + required: true + schema: + type: string + - name: location + in: query + description: 中心点的经度和纬度，用逗号隔开 + required: true + schema: + type: string + deprecated: false + components: + schemas: {} +- apiProvider: + domain: api.seniverse.com + serviceName: seniverse.dns + servicePort: 80 + apiKey: + in: query + name: key + value: xxxxxxxxxxxxxxx + api: | + openapi: 3.1.0 + info: + title: 心知天气 + description: 获取 天气预办相关信息 + version: v1.0.0 + servers: + - url: https://api.seniverse.com + paths: + /v3/weather/now.json: + get: + description: 获取指定城市的天气实况 + operationId: get_weather_now + parameters: + - name: location + in: query + description: 所查询的城市 + required: true + schema: + type: string + - name: language + in: query + description: 返回天气查询结果所使用的语言 + required: true + schema: + type: string + default: zh-Hans + enum: + - zh-Hans + - en + - ja + - name: unit + in: query + description: 表示温度的的单位，有摄氏度和华氏度两种 + required: true + schema: + type: string + default: c + enum: + - c + - f + deprecated: false + components: + schemas: {} +``` + +本示例配置了两个服务，一个是高德地图，另一个是心知天气，两个服务都需要现在Higress的服务中以DNS域名的方式配置好，并确保健康。 +高德地图提供了两个工具，分别是获取指定地点的坐标，以及搜索坐标附近的感兴趣的地点。文档：https://lbs.amap.com/api/webservice/guide/api-advanced/newpoisearch +心知天气提供了一个工具，用于获取指定城市的实时天气情况，支持中文，英文，日语返回，以及摄氏度和华氏度的表示。文档：https://seniverse.yuque.com/hyper_data/api_v3/nyiu3t + + +以下为测试用例，为了效果的稳定性，建议保持大模型版本的稳定，本例子中使用的qwen-max-0403： + +**请求示例** + +```shell +curl 'http://<这里换成网关公网IP>/api/openai/v1/chat/completions' \ +-H 'Accept: application/json, text/event-stream' \ +-H 'Content-Type: application/json' \ +--data-raw '{"model":"qwen","frequency_penalty":0,"max_tokens":800,"stream":false,"messages":[{"role":"user","content":"我想在济南市鑫盛大厦附近喝咖啡，给我推荐几个"}],"presence_penalty":0,"temperature":0,"top_p":0}' +``` + +**响应示例** + +```json +{"id":"139487e7-96a0-9b13-91b4-290fb79ac992","choices":[{"index":0,"message":{"role":"assistant","content":" 在济南市鑫盛大厦附近，您可以选择以下咖啡店：\n1. luckin coffee 瑞幸咖啡(鑫盛大厦店)，位于新泺大街1299号鑫盛大厦2号楼大堂；\n2. 三庆齐盛广场挪瓦咖啡(三庆·齐盛广场店)，位于新泺大街与颖秀路交叉口西南60米；\n3. luckin coffee 瑞幸咖啡(三庆·齐盛广场店)，位于颖秀路1267号；\n4. 库迪咖啡(齐鲁软件园店)，位于新泺大街三庆齐盛广场4号楼底商；\n5. 库迪咖啡(美莲广场店)，位于高新区新泺大街1166号美莲广场L117号；以及其他一些选项。希望这些建议对您有所帮助！"},"finish_reason":"stop"}],"created":1723172296,"model":"qwen-max-0403","object":"chat.completion","usage":{"prompt_tokens":886,"completion_tokens":50,"total_tokens":936}} +``` + +**请求示例** + +```shell +curl 'http://<这里换成网关公网IP>/api/openai/v1/chat/completions' \ +-H 'Accept: application/json, text/event-stream' \ +-H 'Content-Type: application/json' \ +--data-raw '{"model":"qwen","frequency_penalty":0,"max_tokens":800,"stream":false,"messages":[{"role":"user","content":"济南市现在的天气情况如何？"}],"presence_penalty":0,"temperature":0,"top_p":0}' +``` + +**响应示例** + +```json +{"id":"ebd6ea91-8e38-9e14-9a5b-90178d2edea4","choices":[{"index":0,"message":{"role":"assistant","content":" 济南市现在的天气状况为阴天，温度为31℃。此信息最后更新于2024年8月9日15时12分（北京时间）。"},"finish_reason":"stop"}],"created":1723187991,"model":"qwen-max-0403","object":"chat.completion","usage":{"prompt_tokens":890,"completion_tokens":56,"total_tokens":946}} +``` + +**请求示例** + +```shell +curl 'http://<这里换成网关公网IP>/api/openai/v1/chat/completions' \ +-H 'Accept: application/json, text/event-stream' \ +-H 'Content-Type: application/json' \ +--data-raw '{"model":"qwen","frequency_penalty":0,"max_tokens":800,"stream":false,"messages":[{"role":"user","content":"济南市现在的天气情况如何？用华氏度表示，用日语回答"}],"presence_penalty":0,"temperature":0,"top_p":0}' +``` + +**响应示例** + +```json +{"id":"ebd6ea91-8e38-9e14-9a5b-90178d2edea4","choices":[{"index":0,"message":{"role":"assistant","content":" 济南市の現在の天気は雨曇りで、気温は88°Fです。この情報は2024年8月9日15時12分（東京時間）に更新されました。"},"finish_reason":"stop"}],"created":1723187991,"model":"qwen-max-0403","object":"chat.completion","usage":{"prompt_tokens":890,"completion_tokens":56,"total_tokens":946}} +``` diff --git a/plugins/wasm-go/extensions/ai-agent/config.go b/plugins/wasm-go/extensions/ai-agent/config.go new file mode 100644 index 000000000..9aa47ef1f --- /dev/null +++ b/plugins/wasm-go/extensions/ai-agent/config.go @@ -0,0 +1,361 @@ +package main + +import ( + "encoding/json" + "errors" + + "github.com/alibaba/higress/plugins/wasm-go/pkg/wrapper" + "github.com/tidwall/gjson" + "gopkg.in/yaml.v2" +) + +type Message struct { + Role string `json:"role"` + Content string `json:"content"` +} + +type Request struct { + Model string `json:"model"` + Messages []Message `json:"messages"` + FrequencyPenalty float64 `json:"frequency_penalty"` + PresencePenalty float64 `json:"presence_penalty"` + Stream bool `json:"stream"` + Temperature float64 `json:"temperature"` + Topp int32 `json:"top_p"` +} + +type Choice struct { + Index int `json:"index"` + Message Message `json:"message"` + FinishReason string `json:"finish_reason"` +} + +type Usage struct { + PromptTokens int `json:"prompt_tokens"` + CompletionTokens int `json:"completion_tokens"` + TotalTokens int `json:"total_tokens"` +} + +type Response struct { + ID string `json:"id"` + Choices []Choice `json:"choices"` + Created int64 `json:"created"` + Model string `json:"model"` + Object string `json:"object"` + Usage Usage `json:"usage"` +} + +// 用于存放拆解出来的工具相关信息 +type Tool_Param struct { + ToolName string `yaml:"toolName"` + Path string `yaml:"path"` + Method string `yaml:"method"` + ParamName []string `yaml:"paramName"` + Parameter string `yaml:"parameter"` + Desciption string `yaml:"description"` +} + +// 用于存放拆解出来的api相关信息 +type API_Param struct { + APIKey APIKey `yaml:"apiKey"` + URL string `yaml:"url"` + Tool_Param []Tool_Param `yaml:"tool_Param"` +} + +type Info struct { + Title string `yaml:"title"` + Description string `yaml:"description"` + Version string `yaml:"version"` +} + +type Server struct { + URL string `yaml:"url"` +} + +type Parameter struct { + Name string `yaml:"name"` + In string `yaml:"in"` + Description string `yaml:"description"` + Required bool `yaml:"required"` + Schema struct { + Type string `yaml:"type"` + Default string `yaml:"default"` + Enum []string `yaml:"enum"` + } `yaml:"schema"` +} + +type PathItem struct { + Description string `yaml:"description"` + OperationID string `yaml:"operationId"` + Parameters []Parameter `yaml:"parameters"` + Deprecated bool `yaml:"deprecated"` +} + +type Paths map[string]map[string]PathItem + +type Components struct { + Schemas map[string]interface{} `yaml:"schemas"` +} + +type API struct { + OpenAPI string `yaml:"openapi"` + Info Info `yaml:"info"` + Servers []Server `yaml:"servers"` + Paths Paths `yaml:"paths"` + Components Components `yaml:"components"` +} + +type APIKey struct { + In string `yaml:"in" json:"in"` + Name string `yaml:"name" json:"name"` + Value string `yaml:"value" json:"value"` +} + +type APIProvider struct { + // @Title zh-CN 服务名称 + // @Description zh-CN 带服务类型的完整 FQDN 名称，例如 my-redis.dns、redis.my-ns.svc.cluster.local + ServiceName string `required:"true" yaml:"serviceName" json:"serviceName"` + // @Title zh-CN 服务端口 + // @Description zh-CN 服务端口 + ServicePort int64 `required:"true" yaml:"servicePort" json:"servicePort"` + // @Title zh-CN 服务域名 + // @Description zh-CN 服务域名，例如 restapi.amap.com + Domin string `required:"true" yaml:"domain" json:"domain"` + // @Title zh-CN 通义千问大模型服务的key + // @Description zh-CN 通义千问大模型服务的key + APIKey APIKey `required:"true" yaml:"apiKey" json:"apiKey"` +} + +type APIs struct { + APIProvider APIProvider `required:"true" yaml:"apiProvider" json:"apiProvider"` + API string `required:"true" yaml:"api" json:"api"` +} + +type Template struct { + Question string `yaml:"question" json:"question"` + Thought1 string `yaml:"thought1" json:"thought1"` + ActionInput string `yaml:"actionInput" json:"actionInput"` + Observation string `yaml:"observation" json:"observation"` + Thought2 string `yaml:"thought2" json:"thought2"` + FinalAnswer string `yaml:"finalAnswer" json:"finalAnswer"` + Begin string `yaml:"begin" json:"begin"` +} + +type PromptTemplate struct { + Language string `required:"true" yaml:"language" json:"language"` + CHTemplate Template `yaml:"chTemplate" json:"chTemplate"` + ENTemplate Template `yaml:"enTemplate" json:"enTemplate"` +} + +type LLMInfo struct { + // @Title zh-CN 大模型服务名称 + // @Description zh-CN 带服务类型的完整 FQDN 名称 + ServiceName string `required:"true" yaml:"serviceName" json:"serviceName"` + // @Title zh-CN 大模型服务端口 + // @Description zh-CN 服务端口 + ServicePort int64 `required:"true" yaml:"servicePort" json:"servicePort"` + // @Title zh-CN 大模型服务域名 + // @Description zh-CN 大模型服务域名，例如 dashscope.aliyuncs.com + Domin string `required:"true" yaml:"domin" json:"domin"` + // @Title zh-CN 大模型服务的key + // @Description zh-CN 大模型服务的key + APIKey string `required:"true" yaml:"apiKey" json:"apiKey"` + // @Title zh-CN 大模型服务的请求路径 + // @Description zh-CN 大模型服务的请求路径，如"/compatible-mode/v1/chat/completions" + Path string `required:"true" yaml:"path" json:"path"` + // @Title zh-CN 大模型服务的模型名称 + // @Description zh-CN 大模型服务的模型名称，如"qwen-max-0403" + Model string `required:"true" yaml:"model" json:"model"` +} + +type PluginConfig struct { + // @Title zh-CN 返回 HTTP 响应的模版 + // @Description zh-CN 用 %s 标记需要被 cache value 替换的部分 + ReturnResponseTemplate string `required:"true" yaml:"returnResponseTemplate" json:"returnResponseTemplate"` + // @Title zh-CN 工具服务商以及工具信息 + // @Description zh-CN 用于存储工具服务商以及工具信息 + APIs []APIs `required:"true" yaml:"apis" json:"apis"` + APIClient []wrapper.HttpClient `yaml:"-" json:"-"` + // @Title zh-CN llm信息 + // @Description zh-CN 用于存储llm使用信息 + LLMInfo LLMInfo `required:"true" yaml:"llm" json:"llm"` + LLMClient wrapper.HttpClient `yaml:"-" json:"-"` + API_Param []API_Param `yaml:"-" json:"-"` + PromptTemplate PromptTemplate `yaml:"promptTemplate" json:"promptTemplate"` +} + +func initResponsePromptTpl(gjson gjson.Result, c *PluginConfig) { + //设置回复模板 + c.ReturnResponseTemplate = gjson.Get("returnResponseTemplate").String() + if c.ReturnResponseTemplate == "" { + c.ReturnResponseTemplate = `{"id":"from-cache","choices":[{"index":0,"message":{"role":"assistant","content":"%s"},"finish_reason":"stop"}],"model":"gpt-4o","object":"chat.completion","usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}` + } +} + +func initAPIs(gjson gjson.Result, c *PluginConfig) error { + //从插件配置中获取apis信息 + apis := gjson.Get("apis") + if !apis.Exists() { + return errors.New("apis is required") + } + if len(apis.Array()) == 0 { + return errors.New("apis cannot be empty") + } + + for _, item := range apis.Array() { + serviceName := item.Get("apiProvider.serviceName") + if !serviceName.Exists() || serviceName.String() == "" { + return errors.New("apiProvider serviceName is required") + } + + servicePort := item.Get("apiProvider.servicePort") + if !servicePort.Exists() || servicePort.Int() == 0 { + return errors.New("apiProvider servicePort is required") + } + + domain := item.Get("apiProvider.domain") + if !domain.Exists() || domain.String() == "" { + return errors.New("apiProvider domain is required") + } + + apiKeyIn := item.Get("apiProvider.apiKey.in").String() + if apiKeyIn != "query" { + apiKeyIn = "header" + } + + apiKeyName := item.Get("apiProvider.apiKey.name") + + apiKeyValue := item.Get("apiProvider.apiKey.value") + + //根据多个toolsClientInfo的信息，分别初始化toolsClient + apiClient := wrapper.NewClusterClient(wrapper.FQDNCluster{ + FQDN: serviceName.String(), + Port: servicePort.Int(), + Host: domain.String(), + }) + + c.APIClient = append(c.APIClient, apiClient) + + api := item.Get("api") + if !api.Exists() || api.String() == "" { + return errors.New("api is required") + } + + var apiStrcut API + err := yaml.Unmarshal([]byte(api.String()), &apiStrcut) + if err != nil { + return err + } + + var allTool_param []Tool_Param + //拆除服务下面的每个api的path + for path, pathmap := range apiStrcut.Paths { + //拆解出每个api对应的参数 + for method, submap := range pathmap { + //把参数列表存起来 + var param Tool_Param + param.Path = path + param.Method = method + param.ToolName = submap.OperationID + paramName := make([]string, 0) + for _, parammeter := range submap.Parameters { + paramName = append(paramName, parammeter.Name) + } + param.ParamName = paramName + out, _ := json.Marshal(submap.Parameters) + param.Parameter = string(out) + param.Desciption = submap.Description + allTool_param = append(allTool_param, param) + } + } + api_param := API_Param{ + APIKey: APIKey{In: apiKeyIn, Name: apiKeyName.String(), Value: apiKeyValue.String()}, + URL: apiStrcut.Servers[0].URL, + Tool_Param: allTool_param, + } + + c.API_Param = append(c.API_Param, api_param) + } + return nil +} + +func initReActPromptTpl(gjson gjson.Result, c *PluginConfig) { + c.PromptTemplate.Language = gjson.Get("promptTemplate.language").String() + if c.PromptTemplate.Language != "EN" && c.PromptTemplate.Language != "CH" { + c.PromptTemplate.Language = "EN" + } + if c.PromptTemplate.Language == "EN" { + c.PromptTemplate.ENTemplate.Question = gjson.Get("promptTemplate.enTemplate.question").String() + if c.PromptTemplate.ENTemplate.Question == "" { + c.PromptTemplate.ENTemplate.Question = "the input question you must answer" + } + c.PromptTemplate.ENTemplate.Thought1 = gjson.Get("promptTemplate.enTemplate.thought1").String() + if c.PromptTemplate.ENTemplate.Thought1 == "" { + c.PromptTemplate.ENTemplate.Thought1 = "you should always think about what to do" + } + c.PromptTemplate.ENTemplate.ActionInput = gjson.Get("promptTemplate.enTemplate.actionInput").String() + if c.PromptTemplate.ENTemplate.ActionInput == "" { + c.PromptTemplate.ENTemplate.ActionInput = "the input to the action" + } + c.PromptTemplate.ENTemplate.Observation = gjson.Get("promptTemplate.enTemplate.observation").String() + if c.PromptTemplate.ENTemplate.Observation == "" { + c.PromptTemplate.ENTemplate.Observation = "the result of the action" + } + c.PromptTemplate.ENTemplate.Thought1 = gjson.Get("promptTemplate.enTemplate.thought2").String() + if c.PromptTemplate.ENTemplate.Thought1 == "" { + c.PromptTemplate.ENTemplate.Thought1 = "I now know the final answer" + } + c.PromptTemplate.ENTemplate.FinalAnswer = gjson.Get("promptTemplate.enTemplate.finalAnswer").String() + if c.PromptTemplate.ENTemplate.FinalAnswer == "" { + c.PromptTemplate.ENTemplate.FinalAnswer = "the final answer to the original input question, please give the most direct answer directly in Chinese, not English, and do not add extra content." + } + c.PromptTemplate.ENTemplate.Begin = gjson.Get("promptTemplate.enTemplate.begin").String() + if c.PromptTemplate.ENTemplate.Begin == "" { + c.PromptTemplate.ENTemplate.Begin = "Begin! Remember to speak as a pirate when giving your final answer. Use lots of \"Arg\"s" + } + } else if c.PromptTemplate.Language == "CH" { + c.PromptTemplate.CHTemplate.Question = gjson.Get("promptTemplate.chTemplate.question").String() + if c.PromptTemplate.CHTemplate.Question == "" { + c.PromptTemplate.CHTemplate.Question = "你需要回答的输入问题" + } + c.PromptTemplate.CHTemplate.Thought1 = gjson.Get("promptTemplate.chTemplate.thought1").String() + if c.PromptTemplate.CHTemplate.Thought1 == "" { + c.PromptTemplate.CHTemplate.Thought1 = "你应该总是思考该做什么" + } + c.PromptTemplate.CHTemplate.ActionInput = gjson.Get("promptTemplate.chTemplate.actionInput").String() + if c.PromptTemplate.CHTemplate.ActionInput == "" { + c.PromptTemplate.CHTemplate.ActionInput = "行动的输入，必须出现在Action后" + } + c.PromptTemplate.CHTemplate.Observation = gjson.Get("promptTemplate.chTemplate.observation").String() + if c.PromptTemplate.CHTemplate.Observation == "" { + c.PromptTemplate.CHTemplate.Observation = "行动的结果" + } + c.PromptTemplate.CHTemplate.Thought1 = gjson.Get("promptTemplate.chTemplate.thought2").String() + if c.PromptTemplate.CHTemplate.Thought1 == "" { + c.PromptTemplate.CHTemplate.Thought1 = "我现在知道最终答案" + } + c.PromptTemplate.CHTemplate.FinalAnswer = gjson.Get("promptTemplate.chTemplate.finalAnswer").String() + if c.PromptTemplate.CHTemplate.FinalAnswer == "" { + c.PromptTemplate.CHTemplate.FinalAnswer = "对原始输入问题的最终答案" + } + c.PromptTemplate.CHTemplate.Begin = gjson.Get("promptTemplate.chTemplate.begin").String() + if c.PromptTemplate.CHTemplate.Begin == "" { + c.PromptTemplate.CHTemplate.Begin = "再次重申，不要修改以上模板的字段名称，开始吧！" + } + } +} + +func initLLMClient(gjson gjson.Result, c *PluginConfig) { + c.LLMInfo.APIKey = gjson.Get("llm.apiKey").String() + c.LLMInfo.ServiceName = gjson.Get("llm.serviceName").String() + c.LLMInfo.ServicePort = gjson.Get("llm.servicePort").Int() + c.LLMInfo.Domin = gjson.Get("llm.domain").String() + c.LLMInfo.Path = gjson.Get("llm.path").String() + c.LLMInfo.Model = gjson.Get("llm.model").String() + + c.LLMClient = wrapper.NewClusterClient(wrapper.FQDNCluster{ + FQDN: c.LLMInfo.ServiceName, + Port: c.LLMInfo.ServicePort, + Host: c.LLMInfo.Domin, + }) +} diff --git a/plugins/wasm-go/extensions/ai-agent/dashscope/message.go b/plugins/wasm-go/extensions/ai-agent/dashscope/message.go new file mode 100644 index 000000000..c0f38606e --- /dev/null +++ b/plugins/wasm-go/extensions/ai-agent/dashscope/message.go @@ -0,0 +1,46 @@ +package dashscope + +var MessageStore ChatMessages + +func init() { + MessageStore = make(ChatMessages, 0) + MessageStore.Clear() //清理和初始化 + +} + +type ChatMessages []Message + +// 枚举出角色 +const ( + RoleUser = "user" + RoleAssistant = "assistant" + RoleSystem = "system" +) + +func (cm *ChatMessages) Clear() { + *cm = make([]Message, 0) //重新初始化 +} + +// 添加角色和对应的prompt +func (cm *ChatMessages) AddFor(msg string, role string) { + *cm = append(*cm, Message{ + Role: role, + Content: msg, + }) +} + +// 添加Assistant角色的prompt +func (cm *ChatMessages) AddForAssistant(msg string) { + cm.AddFor(msg, RoleAssistant) + +} + +// 添加System角色的prompt +func (cm *ChatMessages) AddForSystem(msg string) { + cm.AddFor(msg, RoleSystem) +} + +// 添加User角色的prompt +func (cm *ChatMessages) AddForUser(msg string) { + cm.AddFor(msg, RoleUser) +} diff --git a/plugins/wasm-go/extensions/ai-agent/dashscope/types.go b/plugins/wasm-go/extensions/ai-agent/dashscope/types.go new file mode 100644 index 000000000..7aef7272e --- /dev/null +++ b/plugins/wasm-go/extensions/ai-agent/dashscope/types.go @@ -0,0 +1,75 @@ +package dashscope + +// DashScope embedding service: Request +type Request struct { + Model string `json:"model"` + Input Input `json:"input"` + Parameter Parameter `json:"parameters"` +} + +type Input struct { + Texts []string `json:"texts"` +} + +type Parameter struct { + TextType string `json:"text_type"` +} + +// DashScope embedding service: Response +type Response struct { + Output Output `json:"output"` + Usage Usage `json:"usage"` + RequestID string `json:"request_id"` +} + +type Output struct { + Embeddings []Embedding `json:"embeddings"` +} + +type Embedding struct { + Embedding []float32 `json:"embedding"` + TextIndex int32 `json:"text_index"` +} + +type Usage struct { + TotalTokens int32 `json:"total_tokens"` +} + +// completion +type Completion struct { + Model string `json:"model"` + Messages []Message `json:"messages"` +} + +type Message struct { + Role string `json:"role"` + Content string `json:"content"` +} + +type CompletionResponse struct { + Choices []Choice `json:"choices"` + Object string `json:"object"` + Usage CompletionUsage `json:"usage"` + Created string `json:"created"` + SystemFingerprint string `json:"system_fingerprint"` + Model string `json:"model"` + ID string `json:"id"` +} + +type Choice struct { + Message Message `json:"message"` + FinishReason string `json:"finish_reason"` + Index int `json:"index"` +} + +type CompletionUsage struct { + PromptTokens int `json:"prompt_tokens"` + CompletionTokens int `json:"completion_tokens"` + TotalTokens int `json:"total_tokens"` +} + +type Content struct { + CH_Question string `json:"ch_question"` + Core string `json:"core"` + // EN_Question string `json:"en_question"` +} diff --git a/plugins/wasm-go/extensions/ai-agent/go.mod b/plugins/wasm-go/extensions/ai-agent/go.mod new file mode 100644 index 000000000..352e7a20a --- /dev/null +++ b/plugins/wasm-go/extensions/ai-agent/go.mod @@ -0,0 +1,19 @@ +module github.com/alibaba/higress/plugins/wasm-go/extensions/ai-agent + +go 1.19 + +require ( + github.com/alibaba/higress/plugins/wasm-go v1.4.2 + github.com/higress-group/proxy-wasm-go-sdk v0.0.0-20240711023527-ba358c48772f + github.com/tidwall/gjson v1.17.3 + gopkg.in/yaml.v2 v2.4.0 +) + +require ( + github.com/google/uuid v1.3.0 // indirect + github.com/higress-group/nottinygc v0.0.0-20231101025119-e93c4c2f8520 // indirect + github.com/magefile/mage v1.14.0 // indirect + github.com/tidwall/match v1.1.1 // indirect + github.com/tidwall/pretty v1.2.0 // indirect + github.com/tidwall/resp v0.1.1 // indirect +) diff --git a/plugins/wasm-go/extensions/ai-agent/go.sum b/plugins/wasm-go/extensions/ai-agent/go.sum new file mode 100644 index 000000000..1b2c82d5a --- /dev/null +++ b/plugins/wasm-go/extensions/ai-agent/go.sum @@ -0,0 +1,26 @@ +github.com/alibaba/higress/plugins/wasm-go v1.4.2 h1:gH7OIGXm4wtW5Vo7L2deMPqF7OVWNESDHv1CaaTGu6s= +github.com/alibaba/higress/plugins/wasm-go v1.4.2/go.mod h1:359don/ahMxpfeLMzr29Cjwcu8IywTTDUzWlBPRNLHw= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I= +github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/higress-group/nottinygc v0.0.0-20231101025119-e93c4c2f8520 h1:IHDghbGQ2DTIXHBHxWfqCYQW1fKjyJ/I7W1pMyUDeEA= +github.com/higress-group/nottinygc v0.0.0-20231101025119-e93c4c2f8520/go.mod h1:Nz8ORLaFiLWotg6GeKlJMhv8cci8mM43uEnLA5t8iew= +github.com/higress-group/proxy-wasm-go-sdk v0.0.0-20240711023527-ba358c48772f h1:ZIiIBRvIw62gA5MJhuwp1+2wWbqL9IGElQ499rUsYYg= +github.com/higress-group/proxy-wasm-go-sdk v0.0.0-20240711023527-ba358c48772f/go.mod h1:hNFjhrLUIq+kJ9bOcs8QtiplSQ61GZXtd2xHKx4BYRo= +github.com/magefile/mage v1.14.0 h1:6QDX3g6z1YvJ4olPhT1wksUcSa/V0a1B+pJb73fBjyo= +github.com/magefile/mage v1.14.0/go.mod h1:z5UZb/iS3GoOSn0JgWuiw7dxlurVYTu+/jHXqQg881A= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk= +github.com/tidwall/gjson v1.17.3 h1:bwWLZU7icoKRG+C+0PNwIKC6FCJO/Q3p2pZvuP0jN94= +github.com/tidwall/gjson v1.17.3/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk= +github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA= +github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM= +github.com/tidwall/pretty v1.2.0 h1:RWIZEg2iJ8/g6fDDYzMpobmaoGh5OLl4AXtGUGPcqCs= +github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU= +github.com/tidwall/resp v0.1.1 h1:Ly20wkhqKTmDUPlyM1S7pWo5kk0tDu8OoC/vFArXmwE= +github.com/tidwall/resp v0.1.1/go.mod h1:3/FrruOBAxPTPtundW0VXgmsQ4ZBA0Aw714lVYgwFa0= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= +gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= diff --git a/plugins/wasm-go/extensions/ai-agent/main.go b/plugins/wasm-go/extensions/ai-agent/main.go new file mode 100644 index 000000000..5f0f0fe22 --- /dev/null +++ b/plugins/wasm-go/extensions/ai-agent/main.go @@ -0,0 +1,321 @@ +package main + +import ( + "encoding/json" + "fmt" + "net/http" + "regexp" + "strings" + + "github.com/alibaba/higress/plugins/wasm-go/extensions/ai-agent/dashscope" + prompttpl "github.com/alibaba/higress/plugins/wasm-go/extensions/ai-agent/promptTpl" + "github.com/alibaba/higress/plugins/wasm-go/pkg/wrapper" + "github.com/higress-group/proxy-wasm-go-sdk/proxywasm" + "github.com/higress-group/proxy-wasm-go-sdk/proxywasm/types" + "github.com/tidwall/gjson" +) + +func main() { + wrapper.SetCtx( + "ai-agent", + wrapper.ParseConfigBy(parseConfig), + wrapper.ProcessRequestHeadersBy(onHttpRequestHeaders), + wrapper.ProcessRequestBodyBy(onHttpRequestBody), + wrapper.ProcessResponseHeadersBy(onHttpResponseHeaders), + wrapper.ProcessResponseBodyBy(onHttpResponseBody), + ) +} + +func parseConfig(gjson gjson.Result, c *PluginConfig, log wrapper.Log) error { + initResponsePromptTpl(gjson, c) + + err := initAPIs(gjson, c) + if err != nil { + return err + } + + initReActPromptTpl(gjson, c) + + initLLMClient(gjson, c) + + return nil +} + +func onHttpRequestHeaders(ctx wrapper.HttpContext, config PluginConfig, log wrapper.Log) types.Action { + return types.ActionContinue +} + +func firstReq(config PluginConfig, prompt string, rawRequest Request, log wrapper.Log) types.Action { + log.Debugf("[onHttpRequestBody] firstreq:%s", prompt) + + var userMessage Message + userMessage.Role = "user" + userMessage.Content = prompt + + newMessages := []Message{userMessage} + rawRequest.Messages = newMessages + + //replace old message and resume request qwen + newbody, err := json.Marshal(rawRequest) + if err != nil { + return types.ActionContinue + } else { + log.Debugf("[onHttpRequestBody] newRequestBody: ", string(newbody)) + err := proxywasm.ReplaceHttpRequestBody(newbody) + if err != nil { + log.Debug("替换失败") + proxywasm.SendHttpResponse(200, [][2]string{{"content-type", "application/json; charset=utf-8"}}, []byte(fmt.Sprintf(config.ReturnResponseTemplate, "替换失败"+err.Error())), -1) + } + log.Debug("[onHttpRequestBody] request替换成功") + return types.ActionContinue + } +} + +func onHttpRequestBody(ctx wrapper.HttpContext, config PluginConfig, body []byte, log wrapper.Log) types.Action { + log.Debug("onHttpRequestBody start") + defer log.Debug("onHttpRequestBody end") + + //拿到请求 + var rawRequest Request + err := json.Unmarshal(body, &rawRequest) + if err != nil { + log.Debugf("[onHttpRequestBody] body json umarshal err: ", err.Error()) + return types.ActionContinue + } + log.Debugf("onHttpRequestBody rawRequest: %v", rawRequest) + + //获取用户query + var query string + messageLength := len(rawRequest.Messages) + log.Debugf("[onHttpRequestBody] messageLength: %s\n", messageLength) + if messageLength > 0 { + query = rawRequest.Messages[messageLength-1].Content + log.Debugf("[onHttpRequestBody] query: %s\n", query) + } else { + return types.ActionContinue + } + + if query == "" { + log.Debug("parse query from request body failed") + return types.ActionContinue + } + + //拼装agent prompt模板 + tool_desc := make([]string, 0) + tool_names := make([]string, 0) + for _, api_param := range config.API_Param { + for _, tool_param := range api_param.Tool_Param { + tool_desc = append(tool_desc, fmt.Sprintf(prompttpl.TOOL_DESC, tool_param.ToolName, tool_param.Desciption, tool_param.Desciption, tool_param.Desciption, tool_param.Parameter), "\n") + tool_names = append(tool_names, tool_param.ToolName) + } + } + + var prompt string + if config.PromptTemplate.Language == "CH" { + prompt = fmt.Sprintf(prompttpl.CH_Template, + tool_desc, + config.PromptTemplate.CHTemplate.Question, + config.PromptTemplate.CHTemplate.Thought1, + tool_names, + config.PromptTemplate.CHTemplate.ActionInput, + config.PromptTemplate.CHTemplate.Observation, + config.PromptTemplate.CHTemplate.FinalAnswer, + config.PromptTemplate.CHTemplate.Begin, + query) + } else { + prompt = fmt.Sprintf(prompttpl.EN_Template, + tool_desc, + config.PromptTemplate.ENTemplate.Question, + config.PromptTemplate.ENTemplate.Thought1, + tool_names, + config.PromptTemplate.ENTemplate.ActionInput, + config.PromptTemplate.ENTemplate.Observation, + config.PromptTemplate.ENTemplate.FinalAnswer, + config.PromptTemplate.ENTemplate.Begin, + query) + } + + //将请求加入到历史对话存储器中 + dashscope.MessageStore.AddForUser(prompt) + + //开始第一次请求 + ret := firstReq(config, prompt, rawRequest, log) + + return ret +} + +func onHttpResponseHeaders(ctx wrapper.HttpContext, config PluginConfig, log wrapper.Log) types.Action { + return types.ActionContinue +} + +func toolsCall(config PluginConfig, content string, rawResponse Response, log wrapper.Log) types.Action { + dashscope.MessageStore.AddForAssistant(content) + + //得到最终答案 + regexPattern := regexp.MustCompile(`Final Answer:(.*)`) + finalAnswer := regexPattern.FindStringSubmatch(content) + if len(finalAnswer) > 1 { + return types.ActionContinue + } + + //没得到最终答案 + regexAction := regexp.MustCompile(`Action:\s*(.*?)[.\n]`) + regexActionInput := regexp.MustCompile(`Action Input:\s*(.*)`) + + action := regexAction.FindStringSubmatch(content) + actionInput := regexActionInput.FindStringSubmatch(content) + + if len(action) > 1 && len(actionInput) > 1 { + var url string + var headers [][2]string + var apiClient wrapper.HttpClient + var method string + + for i, api_param := range config.API_Param { + for _, tool_param := range api_param.Tool_Param { + if action[1] == tool_param.ToolName { + log.Infof("calls %s\n", tool_param.ToolName) + log.Infof("actionInput[1]: %s", actionInput[1]) + + //将大模型需要的参数反序列化 + var data map[string]interface{} + if err := json.Unmarshal([]byte(actionInput[1]), &data); err != nil { + log.Debugf("Error: %s\n", err.Error()) + return types.ActionContinue + } + + var args string + for i, param := range tool_param.ParamName { //从参数列表中取出参数 + if i == 0 { + args = "?" + param + "=%s" + args = fmt.Sprintf(args, data[param]) + } else { + args = args + "&" + param + "=%s" + args = fmt.Sprintf(args, data[param]) + } + } + + url = api_param.URL + tool_param.Path + args + + if api_param.APIKey.Name != "" { + if api_param.APIKey.In == "query" { + headers = nil + key := "&" + api_param.APIKey.Name + "=" + api_param.APIKey.Value + url += key + } else if api_param.APIKey.In == "header" { + headers = [][2]string{{"Content-Type", "application/json"}, {"Authorization", api_param.APIKey.Name + " " + api_param.APIKey.Value}} + } + } + + log.Infof("url: %s\n", url) + + method = tool_param.Method + + apiClient = config.APIClient[i] + break + } + } + } + + if method == "get" { + //调用工具 + err := apiClient.Get( + url, + headers, + func(statusCode int, responseHeaders http.Header, responseBody []byte) { + if statusCode != http.StatusOK { + log.Debugf("statusCode: %d\n", statusCode) + } + log.Info("========函数返回结果========") + log.Infof(string(responseBody)) + + Observation := "Observation: " + string(responseBody) + + dashscope.MessageStore.AddForUser(Observation) + + completion := dashscope.Completion{ + Model: config.LLMInfo.Model, + Messages: dashscope.MessageStore, + } + + headers := [][2]string{{"Content-Type", "application/json"}, {"Authorization", "Bearer " + config.LLMInfo.APIKey}} + completionSerialized, _ := json.Marshal(completion) + err := config.LLMClient.Post( + config.LLMInfo.Path, + headers, + completionSerialized, + func(statusCode int, responseHeaders http.Header, responseBody []byte) { + //得到gpt的返回结果 + var responseCompletion dashscope.CompletionResponse + _ = json.Unmarshal(responseBody, &responseCompletion) + log.Infof("[toolsCall] content: ", responseCompletion.Choices[0].Message.Content) + + if responseCompletion.Choices[0].Message.Content != "" { + retType := toolsCall(config, responseCompletion.Choices[0].Message.Content, rawResponse, log) + if retType == types.ActionContinue { + //得到了Final Answer + var assistantMessage Message + assistantMessage.Role = "assistant" + startIndex := strings.Index(responseCompletion.Choices[0].Message.Content, "Final Answer:") + if startIndex != -1 { + startIndex += len("Final Answer:") // 移动到"Final Answer:"之后的位置 + extractedText := responseCompletion.Choices[0].Message.Content[startIndex:] + assistantMessage.Content = extractedText + } + //assistantMessage.Content = responseCompletion.Choices[0].Message.Content + rawResponse.Choices[0].Message = assistantMessage + + newbody, err := json.Marshal(rawResponse) + if err != nil { + proxywasm.ResumeHttpResponse() + return + } else { + log.Infof("[onHttpResponseBody] newResponseBody: ", string(newbody)) + proxywasm.ReplaceHttpResponseBody(newbody) + + log.Debug("[onHttpResponseBody] response替换成功") + proxywasm.ResumeHttpResponse() + } + } + } else { + proxywasm.ResumeHttpRequest() + } + }, 50000) + if err != nil { + log.Debugf("[onHttpRequestBody] completion err: %s", err.Error()) + proxywasm.ResumeHttpRequest() + } + }, 50000) + if err != nil { + log.Debugf("tool calls error: %s\n", err.Error()) + proxywasm.ResumeHttpRequest() + } + } else { + return types.ActionContinue + } + + } + return types.ActionPause +} + +// 从response接收到firstreq的大模型返回 +func onHttpResponseBody(ctx wrapper.HttpContext, config PluginConfig, body []byte, log wrapper.Log) types.Action { + log.Debugf("onHttpResponseBody start") + defer log.Debugf("onHttpResponseBody end") + + //初始化接收gpt返回内容的结构体 + var rawResponse Response + err := json.Unmarshal(body, &rawResponse) + if err != nil { + log.Debugf("[onHttpResponseBody] body to json err: %s", err.Error()) + return types.ActionContinue + } + log.Infof("first content: %s\n", rawResponse.Choices[0].Message.Content) + //如果gpt返回的内容不是空的 + if rawResponse.Choices[0].Message.Content != "" { + //进入agent的循环思考，工具调用的过程中 + return toolsCall(config, rawResponse.Choices[0].Message.Content, rawResponse, log) + } else { + return types.ActionContinue + } +} diff --git a/plugins/wasm-go/extensions/ai-agent/promptTpl/prompt.go b/plugins/wasm-go/extensions/ai-agent/promptTpl/prompt.go new file mode 100644 index 000000000..ab8bffc57 --- /dev/null +++ b/plugins/wasm-go/extensions/ai-agent/promptTpl/prompt.go @@ -0,0 +1,93 @@ +package prompttpl + +// input param +// {name_for_model} +// {description_for_model} +// {description_for_model} +// {description_for_model} +// {parameters} +const TOOL_DESC = ` +%s: Call this tool to interact with the %s API. What is the %s API useful for? %s +Parameters: +%s +Format the arguments as a JSON object.` + +/* +Answer the following questions as best you can, but speaking as a pirate might speak. You have access to the following tools: + +%s + +Use the following format: + +Question: the input question you must answer +Thought: you should always think about what to do +Action: the action to take, should be one of %s +Action Input: the input to the action +Observation: the result of the action +... (this Thought/Action/Action Input/Observation can repeat N times) +Thought: I now know the final answer +Final Answer: the final answer to the original input question, please give the most direct answer directly in Chinese, not English, and do not add extra content. + +Begin! Remember to speak as a pirate when giving your final answer. Use lots of "Arg"s + +Question: %s +*/ +const EN_Template = ` +Answer the following questions as best you can, but speaking as a pirate might speak. You have access to the following tools: + +%s + +Use the following format: + +Question: %s +Thought: %s +Action: the action to take, should be one of %s +Action Input: %s +Observation: %s +... (this Thought/Action/Action Input/Observation can repeat N times) +Thought: %s +Final Answer: %s + +%s + +Question: %s +` + +/* +尽你所能回答以下问题。你可以使用以下工具： + +%s + +请使用以下格式，其中Action字段后必须跟着Action Input字段，并且不要将Action Input替换成Input或者tool等字段，不能出现格式以外的字段名，每个字段在每个轮次只出现一次： +Question: 你需要回答的输入问题 +Thought: 你应该总是思考该做什么 +Action: 要采取的动作，动作只能是%s中的一个 ，一定不要加入其它内容 +Action Input: 行动的输入，必须出现在Action后。 +Observation: 行动的结果 +...（这个Thought/Action/Action Input/Observation可以重复N次） +Thought: 我现在知道最终答案 +Final Answer: 对原始输入问题的最终答案 + +再次重申，不要修改以上模板的字段名称，开始吧！ + +Question: %s +*/ +const CH_Template = ` +尽你所能回答以下问题。你可以使用以下工具： + +%s + +请使用以下格式，其中Action字段后必须跟着Action Input字段，并且不要将Action Input替换成Input或者tool等字段，不能出现格式以外的字段名，每个字段在每个轮次只出现一次： +Question: %s +Thought: %s +Action: 要采取的动作，动作只能是%s中的一个 ，一定不要加入其它内容 +Action Input: %s +Observation: %s +...（这个Thought/Action/Action Input/Observation可以重复N次） +Thought: %s +Final Answer: %s + +%s + +Question: %s +`