mirror of
https://github.com/alibaba/higress.git
synced 2026-05-26 13:47:27 +08:00
feat: ai-token-ratelimit support setting global rate limit thresholds for routes (#2667)
This commit is contained in:
韩贤涛
GitHub
@@ -3,56 +3,95 @@ title: AI Token Rate Limiting
|
||||
keywords: [ AI Gateway, AI Token Rate Limiting ]
|
||||
description: AI Token Rate Limiting Plugin Configuration Reference
|
||||
---
|
||||
## Function Description
|
||||
The `ai-token-ratelimit` plugin implements token rate limiting based on specific key values. The key values can come from URL parameters, HTTP request headers, client IP addresses, consumer names, or key names in cookies.
|
||||
|
||||
## Runtime Attributes
|
||||
Plugin execution phase: `default phase`
|
||||
## Function Description
|
||||
|
||||
The `ai-token-ratelimit` plugin implements AI Token rate limiting based on Redis, supporting the following two rate limiting modes:
|
||||
|
||||
- **Rule-level Global Rate Limiting**: Sets a global token rate limit threshold for custom rule groups based on the same `rule_name` and `global_threshold` configurations.
|
||||
- **Key-level Dynamic Rate Limiting**: Performs grouped token rate limiting based on dynamic keys in requests (including URL parameters, request headers, client IP, Consumer name, or Cookie fields, etc.).
|
||||
|
||||
|
||||
## Runtime Properties
|
||||
|
||||
Plugin execution phase: `Default Phase`
|
||||
Plugin execution priority: `600`
|
||||
|
||||
|
||||
## Configuration Description
|
||||
| Configuration Item | Type | Required | Default Value | Description |
|
||||
| ----------------------- | ----------------- | -------- | ------------- | ----------------------------------------------------------------------------- |
|
||||
| rule_name | string | Yes | - | Name of the rate limiting rule, used to assemble the redis key based on the rule name + rate limiting type + rate limiting key name + actual value corresponding to the rate limiting key |
|
||||
| rule_items | array of object | Yes | - | Rate limiting rule items. After matching the first rule_item, subsequent rules will be ignored based on the order in `rule_items` |
|
||||
| rejected_code | int | No | 429 | The HTTP status code returned when the request is rate limited |
|
||||
| rejected_msg | string | No | Too many requests | The response body returned when the request is rate limited |
|
||||
| redis | object | Yes | - | Redis related configuration |
|
||||
|
||||
Field descriptions for each item in `rule_items`
|
||||
| Configuration Item | Type | Required | Default Value | Description |
|
||||
| ------------------------ | ----------------- | --------------------------- | ------------- | ------------------------------------------------------------ |
|
||||
| limit_by_header | string | No, optionally select one in `limit_by_*` | - | Configure the source HTTP header name for obtaining the rate limiting key value |
|
||||
| limit_by_param | string | No, optionally select one in `limit_by_*` | - | Configure the source URL parameter name for obtaining the rate limiting key value |
|
||||
| limit_by_consumer | string | No, optionally select one in `limit_by_*` | - | Rate limit by consumer name, no actual value needs to be added |
|
||||
| limit_by_cookie | string | No, optionally select one in `limit_by_*` | - | Configure the source key name in cookies for obtaining the rate limiting key value |
|
||||
| limit_by_per_header | string | No, optionally select one in `limit_by_*` | - | Match specific HTTP request headers according to rules and calculate rate limiting separately for each header. Configure the source HTTP header name for obtaining the rate limiting key value. Supports regular expressions or `*` when configuring `limit_keys` |
|
||||
| limit_by_per_param | string | No, optionally select one in `limit_by_*` | - | Match specific URL parameters according to rules and calculate rate limiting separately for each parameter. Configure the source URL parameter name for obtaining the rate limiting key value. Supports regular expressions or `*` when configuring `limit_keys` |
|
||||
| limit_by_per_consumer | string | No, optionally select one in `limit_by_*` | - | Match specific consumers according to rules and calculate rate limiting separately for each consumer. Rate limit by consumer name, no actual value needs to be added. Supports regular expressions or `*` when configuring `limit_keys` |
|
||||
| limit_by_per_cookie | string | No, optionally select one in `limit_by_*` | - | Match specific cookies according to rules and calculate rate limiting separately for each cookie. Configure the source key name in cookies for obtaining the rate limiting key value. Supports regular expressions or `*` when configuring `limit_keys` |
|
||||
| limit_by_per_ip | string | No, optionally select one in `limit_by_*` | - | Match specific IPs according to rules and calculate rate limiting separately for each IP. Configure the source IP parameter name for obtaining the rate limiting key value from request headers, `from-header-<header name>`, such as `from-header-x-forwarded-for`. Directly get the remote socket IP by configuring `from-remote-addr` |
|
||||
| limit_keys | array of object | Yes | - | Configure the number of rate limit requests after matching keys |
|
||||
| Configuration Item | Type | Required | Default Value | Description |
|
||||
|--------------------------|----------------|----------|---------------|-------------------------------------------------------------------------------------------------|
|
||||
| rule_name | string | Yes | - | Name of the rate limiting rule. The Redis key is assembled based on the rate limiting rule name + rate limiting type + rate limiting key name + actual value corresponding to the rate limiting key. |
|
||||
| global_threshold | Object | No, either `global_threshold` or `rule_items` is required | - | Rate limits the entire custom rule group |
|
||||
| rule_items | array of object| No, either `global_threshold` or `rule_items` is required | - | Rate limiting rule items. The first matching `rule_item` in the order of `rule_items` triggers the rate limiting rule, and subsequent rules are ignored. |
|
||||
| rejected_code | int | No | 429 | HTTP status code returned when a request is rate-limited |
|
||||
| rejected_msg | string | No | Too many requests | Response body returned when a request is rate-limited |
|
||||
| redis | object | Yes | - | Redis-related configurations |
|
||||
|
||||
Field descriptions for each item in `limit_keys`
|
||||
| Configuration Item | Type | Required | Default Value | Description |
|
||||
| ----------------------- | ----------------- | ------------------------------------------- | ------------- | ----------------------------------------------- |
|
||||
| key | string | Yes | - | Matched key value. Types `limit_by_per_header`, `limit_by_per_param`, `limit_by_per_consumer`, `limit_by_per_cookie` support configuring regular expressions (beginning with regexp: followed by the regex) or `*` (representing all). Example regex: `regexp:^d.*` (all strings starting with d); `limit_by_per_ip` supports configuring IP addresses or IP segments |
|
||||
| token_per_second | int | No, optionally select one in `token_per_second`, `token_per_minute`, `token_per_hour`, `token_per_day` | - | Allowed number of token requests per second |
|
||||
| token_per_minute | int | No, optionally select one in `token_per_second`, `token_per_minute`, `token_per_hour`, `token_per_day` | - | Allowed number of token requests per minute |
|
||||
| token_per_hour | int | No, optionally select one in `token_per_second`, `token_per_minute`, `token_per_hour`, `token_per_day` | - | Allowed number of token requests per hour |
|
||||
| token_per_day | int | No, optionally select one in `token_per_second`, `token_per_minute`, `token_per_hour`, `token_per_day` | - | Allowed number of token requests per day |
|
||||
|
||||
Field descriptions for each item in `redis`
|
||||
| Configuration Item | Type | Required | Default Value | Description |
|
||||
| ----------------------- | ----------------- | -------- | --------------------------------------------------------------- | ----------------------------------------------- |
|
||||
| service_name | string | Required | - | Full FQDN name of the redis service, including service type, e.g., my-redis.dns, redis.my-ns.svc.cluster.local |
|
||||
| service_port | int | No | Default value for static addresses (static service) is 80; otherwise, it is 6379 | Input the service port of the redis service |
|
||||
| username | string | No | - | Redis username |
|
||||
| password | string | No | - | Redis password |
|
||||
| timeout | int | No | 1000 | Redis connection timeout in milliseconds |
|
||||
| database | int | No | 0 | The database ID used, for example, configured as 1, corresponds to `SELECT 1`. |
|
||||
### Description of Configuration Fields in `global_threshold`
|
||||
|
||||
| Configuration Item | Type | Required | Default Value | Description |
|
||||
|-----------------------|------|----------|---------------|-----------------------------------------------|
|
||||
| token_per_second | int | No, one of `token_per_second`, `token_per_minute`, `token_per_hour`, `token_per_day` is required | - | Allowed number of request tokens per second |
|
||||
| token_per_minute | int | No, one of `token_per_second`, `token_per_minute`, `token_per_hour`, `token_per_day` is required | - | Allowed number of request tokens per minute |
|
||||
| token_per_hour | int | No, one of `token_per_second`, `token_per_minute`, `token_per_hour`, `token_per_day` is required | - | Allowed number of request tokens per hour |
|
||||
| token_per_day | int | No, one of `token_per_second`, `token_per_minute`, `token_per_hour`, `token_per_day` is required | - | Allowed number of request tokens per day |
|
||||
|
||||
|
||||
### Description of Configuration Fields in `rule_items`
|
||||
|
||||
| Configuration Item | Type | Required | Default Value | Description |
|
||||
|-----------------------------|-----------------|----------|---------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| limit_by_header | string | No, one of `limit_by_*` is required | - | Configures the source of the rate limiting key value as the HTTP request header name |
|
||||
| limit_by_param | string | No, one of `limit_by_*` is required | - | Configures the source of the rate limiting key value as the URL parameter name |
|
||||
| limit_by_consumer | string | No, one of `limit_by_*` is required | - | Performs rate limiting based on the consumer name; no actual value needs to be added |
|
||||
| limit_by_cookie | string | No, one of `limit_by_*` is required | - | Configures the source of the rate limiting key value as the key name in the Cookie |
|
||||
| limit_by_per_header | string | No, one of `limit_by_*` is required | - | Matches specific HTTP request headers by rule and calculates rate limits for each header separately. Configures the source of the rate limiting key value as the HTTP request header name. Regular expressions or `*` are supported when configuring `limit_keys`. |
|
||||
| limit_by_per_param | string | No, one of `limit_by_*` is required | - | Matches specific URL parameters by rule and calculates rate limits for each parameter separately. Configures the source of the rate limiting key value as the URL parameter name. Regular expressions or `*` are supported when configuring `limit_keys`. |
|
||||
| limit_by_per_consumer | string | No, one of `limit_by_*` is required | - | Matches specific consumers by rule and calculates rate limits for each consumer separately. Performs rate limiting based on the consumer name; no actual value needs to be added. Regular expressions or `*` are supported when configuring `limit_keys`. |
|
||||
| limit_by_per_cookie | string | No, one of `limit_by_*` is required | - | Matches specific Cookies by rule and calculates rate limits for each Cookie separately. Configures the source of the rate limiting key value as the key name in the Cookie. Regular expressions or `*` are supported when configuring `limit_keys`. |
|
||||
| limit_by_per_ip | string | No, one of `limit_by_*` is required | - | Matches specific IPs by rule and calculates rate limits for each IP separately. Configures the source of the rate limiting key value as the IP parameter name, obtained from the request header in the format `from-header-corresponding_header_name` (e.g., `from-header-x-forwarded-for`), or directly obtains the peer socket IP by configuring `from-remote-addr`. |
|
||||
| limit_keys | array of object | Yes | - | Configures the rate limiting count after matching the key value |
|
||||
|
||||
|
||||
### Description of Configuration Fields in `limit_keys`
|
||||
|
||||
| Configuration Item | Type | Required | Default Value | Description |
|
||||
|-----------------------|--------|----------|---------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| key | string | Yes | - | The matched key value. For types `limit_by_per_header`, `limit_by_per_param`, `limit_by_per_consumer`, and `limit_by_per_cookie`, regular expressions (starting with `regexp:` followed by the regular expression, e.g., `regexp:^d.*` for all strings starting with "d") or `*` (representing all) are supported. For `limit_by_per_ip`, IP addresses or IP segments are supported. |
|
||||
| token_per_second | int | No, one of `token_per_second`, `token_per_minute`, `token_per_hour`, `token_per_day` is required | - | Allowed number of request tokens per second |
|
||||
| token_per_minute | int | No, one of `token_per_second`, `token_per_minute`, `token_per_hour`, `token_per_day` is required | - | Allowed number of request tokens per minute |
|
||||
| token_per_hour | int | No, one of `token_per_second`, `token_per_minute`, `token_per_hour`, `token_per_day` is required | - | Allowed number of request tokens per hour |
|
||||
| token_per_day | int | No, one of `token_per_second`, `token_per_minute`, `token_per_hour`, `token_per_day` is required | - | Allowed number of request tokens per day |
|
||||
|
||||
|
||||
### Description of Configuration Fields in `redis`
|
||||
|
||||
| Configuration Item | Type | Required | Default Value | Description |
|
||||
|--------------------|--------|----------|---------------|-------------------------------------------------------------------------------------------------|
|
||||
| service_name | string | Yes | - | Redis service name, a complete FQDN with service type, e.g., my-redis.dns, redis.my-ns.svc.cluster.local |
|
||||
| service_port | int | No | 80 for static services, 6379 for others | Enter the service port of the Redis service |
|
||||
| username | string | No | - | Redis username |
|
||||
| password | string | No | - | Redis password |
|
||||
| timeout | int | No | 1000 | Redis connection timeout in milliseconds |
|
||||
| database | int | No | 0 | The database ID to use, e.g., configuring 1 corresponds to `SELECT 1` |
|
||||
|
||||
|
||||
## Configuration Example
|
||||
|
||||
### Custom Rule Group Global Rate Limiting
|
||||
|
||||
```yaml
|
||||
rule_name: routeA-global-limit-rule
|
||||
global_threshold:
|
||||
token_per_minute: 1000 # 1000 tokens per minute for the custom rule group
|
||||
redis:
|
||||
service_name: redis.static
|
||||
show_limit_quota_header: true
|
||||
```
|
||||
|
||||
## Configuration Examples
|
||||
### Identify request parameter apikey for differentiated rate limiting
|
||||
```yaml
|
||||
rule_name: default_rule
|
||||
@@ -83,7 +122,7 @@ rule_name: default_rule
|
||||
rule_items:
|
||||
- limit_by_header: x-ca-key
|
||||
limit_keys:
|
||||
- key: 102234
|
||||
- key: 102234
|
||||
token_per_minute: 10
|
||||
- key: 308239
|
||||
token_per_hour: 10
|
||||
|
||||
Reference in New Issue
Block a user