jiazhizhong/higress
1
0
Fork 0
mirror of https://github.com/alibaba/higress.git synced 2026-05-24 04:37:25 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat/ai proxy vertex ai compatible (#3324)

Browse Source
This commit is contained in:
woody
2026-01-14 10:13:00 +08:00
committed by GitHub
parent e7010256fe
commit f1a5f18c78
7 changed files with 802 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

79
plugins/wasm-go/extensions/ai-proxy/README_EN.md
View File

@@ -277,6 +277,20 @@ Express Mode is a simplified access mode introduced by Vertex AI. You can quickl
| `apiTokens` | array of string | Required | - | API Key for Express Mode, obtained from Google Cloud Console under API & Services > Credentials |
| `vertexGeminiSafetySetting` | map of string | Optional | - | Gemini model content safety filtering settings. |
**OpenAI Compatible Mode** (using Vertex AI Chat Completions API):
Vertex AI provides an OpenAI-compatible Chat Completions API endpoint, allowing you to use OpenAI format requests and responses directly without protocol conversion. See [Vertex AI OpenAI Compatibility documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/migrate/openai/overview).
| Name | Data Type | Requirement | Default | Description |
|-----------------------------|------------------|---------------| ------ |-------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `vertexOpenAICompatible` | boolean | Optional | false | Enable OpenAI compatible mode. When enabled, uses Vertex AI's OpenAI-compatible Chat Completions API |
| `vertexAuthKey` | string | Required | - | Google Service Account JSON Key for authentication |
| `vertexRegion` | string | Required | - | Google Cloud region (e.g., us-central1, europe-west4) |
| `vertexProjectId` | string | Required | - | Google Cloud Project ID |
| `vertexAuthServiceName` | string | Required | - | Service name for OAuth2 authentication |
**Note**: OpenAI Compatible Mode and Express Mode are mutually exclusive. You cannot configure both `apiTokens` and `vertexOpenAICompatible` at the same time.
#### AWS Bedrock
For AWS Bedrock, the corresponding `type` is `bedrock`. It supports two authentication methods:
@@ -1848,6 +1862,71 @@ provider:
}
```
### Utilizing OpenAI Protocol Proxy for Google Vertex Services (OpenAI Compatible Mode)
OpenAI Compatible Mode uses Vertex AI's OpenAI-compatible Chat Completions API. Both requests and responses use OpenAI format, requiring no protocol conversion.
**Configuration Information**
```yaml
provider:
type: vertex
vertexOpenAICompatible: true
vertexAuthKey: |
{
"type": "service_account",
"project_id": "your-project-id",
"private_key_id": "your-private-key-id",
"private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n",
"client_email": "your-service-account@your-project.iam.gserviceaccount.com",
"token_uri": "https://oauth2.googleapis.com/token"
}
vertexRegion: us-central1
vertexProjectId: your-project-id
vertexAuthServiceName: your-auth-service-name
modelMapping:
"gpt-4": "gemini-2.0-flash"
"*": "gemini-1.5-flash"
```
**Request Example**
```json
{
"model": "gpt-4",
"messages": [
{
"role": "user",
"content": "Hello, who are you?"
}
],
"stream": false
}
```
**Response Example**
```json
{
"id": "chatcmpl-abc123",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Hello! I am Gemini, an AI model developed by Google. I can help answer questions, provide information, and engage in conversations. How can I assist you today?"
},
"finish_reason": "stop"
}
],
"created": 1729986750,
"model": "gemini-2.0-flash",
"object": "chat.completion",
"usage": {
"prompt_tokens": 12,
"completion_tokens": 35,
"total_tokens": 47
}
}
```
### Utilizing OpenAI Protocol Proxy for AWS Bedrock Services
AWS Bedrock supports two authentication methods: