jiazhizhong/higress
1
0
Fork 0
mirror of https://github.com/alibaba/higress.git synced 2026-04-21 03:57:28 +08:00
Code Issues Packages Projects Releases Wiki Activity

ai cache plugin support set skip ai cache header (#1380)

Browse Source
This commit is contained in:
澄潭
2024-10-21 15:43:01 +08:00
committed by GitHub
parent fc6902ded2
commit badf4b7101
3 changed files with 21 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
plugins/wasm-go/extensions/ai-cache/README_EN.md
View File

@@ -6,6 +6,10 @@ description: AI Cache Plugin Configuration Reference
## Function Description
LLM result caching plugin, the default configuration can be directly used for result caching under the OpenAI protocol, and it supports caching of both streaming and non-streaming responses.
**Tips**
When carrying the request header `x-higress-skip-ai-cache: on`, the current request will not use content from the cache but will be directly forwarded to the backend service. Additionally, the response content from this request will not be cached.
## Runtime Properties
Plugin Execution Phase: `Authentication Phase`
Plugin Execution Priority: `10`