Compare commits

...

14 Commits

Author SHA1 Message Date
澄潭
8165ea6a99 add helm for kind (#57) 2022-11-14 19:34:18 +08:00
澄潭
b15604e9aa Update README.md 2022-11-14 10:02:34 +08:00
澄潭
4d707e960d add istio patch to fix xds push (#52) 2022-11-11 18:10:28 +08:00
realJackSun
33a8573dba Update README.md (#54) 2022-11-11 12:43:29 +08:00
realJackSun
48ad85386e Update README.md (#53) 2022-11-11 11:42:29 +08:00
Yang
da93352a30 Update higress ingress annotation (#49) 2022-11-10 20:53:23 +08:00
澄潭
268c73301e change route name (#51) 2022-11-10 17:02:43 +08:00
澄潭
2b55b6530a fix deadlock caused by xds (#50) 2022-11-10 16:00:15 +08:00
jimin
cdad934567 optimize: add codecov.yml and badge (#42) 2022-11-10 10:04:35 +08:00
澄潭
6415be9537 Update README.md 2022-11-10 09:59:01 +08:00
澄潭
14b3c92cef Rename CONTRIBUTING.md to README.md 2022-11-10 09:57:59 +08:00
澄潭
201c43105d Add wasm plugin contribution introduction (#47) 2022-11-10 09:52:54 +08:00
Yang
ecba3a0265 Move codes to pkg (#46) 2022-11-09 20:37:40 +08:00
Asher Liu
b09b68c1e0 optimize higress installed notes (#41)
Signed-off-by: iutx <root@viper.run>
Co-authored-by: 澄潭 <zty98751@alibaba-inc.com>
2022-11-09 15:05:33 +08:00
113 changed files with 2347 additions and 1773 deletions

View File

@@ -30,7 +30,7 @@ jobs:
run: make build
- name: "run go test and out codecov"
run: make prebuild; go test ./cmd/... ./ingress/... -race -coverprofile=coverage.out -covermode=atomic
run: make prebuild; go test ./cmd/... ./pkg/... -race -coverprofile=coverage.out -covermode=atomic
- name: "upload coverage"
uses: codecov/codecov-action@v3

1
.gitignore vendored
View File

@@ -1,5 +1,6 @@
external
out
*.out
*.tgz
*.wasm
.idea/

View File

@@ -6,6 +6,7 @@ header:
paths-ignore:
- '.gitignore'
- '*.md'
- '*.yml'
- 'LICENSE'
- '.github/**'
- '.licenserc.yaml'

View File

@@ -102,6 +102,12 @@ helm-push-istio:
cd helm/istio; helm dependency update
cd helm; tar -zcf istio.tgz istio; helm push istio.tgz "oci://$(CHARTS)"
helm-push-kind:
cd helm/kind/higress; helm dependency update
cd helm/kind; tar -zcf higress.tgz higress; helm push higress.tgz "oci://$(CHARTS)"
cd helm/kind/istio; helm dependency update
cd helm/kind; tar -zcf istio.tgz istio; helm push istio.tgz "oci://$(CHARTS)"
DIRS_TO_CLEAN := $(OUT)
DIRS_TO_CLEAN += $(OUT_LINUX)

View File

@@ -4,10 +4,13 @@
Next-generation Cloud Native Gateway
</h1>
[![Build Status](https://github.com/alibaba/higress/workflows/build%20and%20codecov/badge.svg?branch=main)](https://github.com/alibaba/higress/actions)
[![license](https://img.shields.io/github/license/alibaba/higress.svg)](https://www.apache.org/licenses/LICENSE-2.0.html)
[**官网**](https://higress.io/) &nbsp; |
&nbsp; [**文档**](https://higress.io/zh-cn/docs/overview/what-is-higress.html) &nbsp; |
&nbsp; [**博客**](https://higress.io/zh-cn/blog/index.html) &nbsp; |
&nbsp; [**开发**](https://higress.io/zh-cn/docs/developers/developers_dev.html) &nbsp;
&nbsp; [**开发指引**](https://higress.io/zh-cn/docs/dev/code.html) &nbsp;
<p>
@@ -36,7 +39,7 @@ Higress 是基于阿里内部两年多的 Envoy Gateway 实践沉淀,以开源
- **微服务网关**:
Higress 可以作为微服务网关, 能够对接多种类型的注册中心发现服务配置路由,例如 Nacos, ZooKeeper, Consul 等。
Higress 可以作为微服务网关, 能够对接多种类型的注册中心发现服务配置路由,例如 Nacos, ZooKeeper, Consul, Eureka 等。
并且深度集成了 [Dubbo](https://github.com/apache/dubbo), [Nacos](https://github.com/alibaba/nacos), [Sentinel](https://github.com/alibaba/Sentinel) 等微服务技术栈,基于 Envoy C++ 网关内核的出色性能,相比传统 Java 类微服务网关,可以显著降低资源使用率,减少成本。
@@ -48,27 +51,27 @@ Higress 是基于阿里内部两年多的 Envoy Gateway 实践沉淀,以开源
- **生产等级**
脱胎于阿里巴巴2年多生产验证的内部产品支持每秒请求量达数十万级的大规模场景
脱胎于阿里巴巴2年多生产验证的内部产品支持每秒请求量达数十万级的大规模场景
彻底摆脱 reload 引起的流量抖动,配置变更毫秒级生效且业务无感
彻底摆脱 reload 引起的流量抖动,配置变更毫秒级生效且业务无感
- **平滑演进**
支持 Nacos/Zookeeper 等多种注册中心,可以不依赖 K8s Service 进行服务发现,支持非容器架构平滑演进到云原生架构
支持 Nacos/Zookeeper/Eureka 等多种注册中心,可以不依赖 K8s Service 进行服务发现,支持非容器架构平滑演进到云原生架构
支持从 Nginx Ingress Controller 平滑迁移,支持平滑过渡到 Gateway API支持业务架构平滑演进到 ServiceMesh
支持从 Nginx Ingress Controller 平滑迁移,支持平滑过渡到 Gateway API支持业务架构平滑演进到 ServiceMesh
- **兼收并蓄**
兼容 Nginx Ingress Annotation 80%+ 的使用场景,且提供功能更丰富的 Higress Annotation 注解
兼容 Nginx Ingress Annotation 80%+ 的使用场景,且提供功能更丰富的 Higress Annotation 注解
兼容 Ingress API/Gateway API/Istio API可以组合多种 CRD 实现流量精细化管理
兼容 Ingress API/Gateway API/Istio API可以组合多种 CRD 实现流量精细化管理
- **便于扩展**
提供 Wasm、Lua、进程外三种插件扩展机制支持多语言编写插件生效粒度支持全局级、域名级路由级
提供 Wasm、Lua、进程外三种插件扩展机制支持多语言编写插件生效粒度支持全局级、域名级路由级
插件支持热更新,变更插件逻辑和配置都对流量无损
插件支持热更新,变更插件逻辑和配置都对流量无损
## Quick Start

View File

@@ -34,7 +34,7 @@ Powered by [Istio](https://github.com/istio/istio) and [Envoy](https://github.co
- **Microservice gateway**:
Higress can function as a microservice gateway, which can discovery microservices from various service registries, such as Nacos, ZooKeeper, Consul, etc.
Higress can function as a microservice gateway, which can discovery microservices from various service registries, such as Nacos, ZooKeeper, Consul, Eureka, etc.
It deeply integrates of [Dubbo](https://github.com/apache/dubbo), [Nacos](https://github.com/alibaba/nacos), [Sentinel](https://github.com/alibaba/Sentinel) and other microservice technology stacks.

View File

@@ -15,6 +15,7 @@
package main
import (
"fmt"
"os"
"time"
@@ -26,13 +27,10 @@ import (
"istio.io/pkg/log"
"istio.io/pkg/version"
"github.com/alibaba/higress/cmd/higress/bootstrap"
"github.com/alibaba/higress/pkg/bootstrap"
)
var (
probeAddr string
enableLeaderElection bool
serverArgs *bootstrap.ServerArgs
loggingOptions = log.DefaultOptions()
@@ -43,13 +41,10 @@ var (
serveCmd = &cobra.Command{
Use: "serve",
Aliases: []string{"serve"},
Short: "Starts the ingress controller",
Short: "Starts the higress ingress controller",
Example: "higress serve",
PreRunE: func(c *cobra.Command, args []string) error {
if err := log.Configure(loggingOptions); err != nil {
return err
}
return nil
return log.Configure(loggingOptions)
},
RunE: func(c *cobra.Command, args []string) error {
cmd.PrintFlags(c.Flags())
@@ -59,10 +54,12 @@ var (
server, err := bootstrap.NewServer(serverArgs)
if err != nil {
return err
return fmt.Errorf("fail to create higress server: %v", err)
}
server.Start(stop)
if err := server.Start(stop); err != nil {
return fmt.Errorf("fail to start higress server: %v", err)
}
cmd.WaitSignal(stop)

20
codecov.yml Normal file
View File

@@ -0,0 +1,20 @@
codecov:
require_ci_to_pass: yes
coverage:
status:
patch: no
project:
default:
target: auto
threshold: 1%
if_not_found: success
changes: no
precision: 2
round: down
range: 50..100
ignore:
- "helm/**"
comment:
layout: "reach,diff,flags,tree"
behavior: default
require_changes: no

View File

@@ -1,5 +1,5 @@
apiVersion: v2
appVersion: 0.5.1
appVersion: 0.5.2
description: Helm chart for deploying higress gateways
icon: https://higress.io/img/higress_logo_small.png
keywords:
@@ -9,4 +9,4 @@ name: higress
sources:
- http://github.com/alibaba/higress
type: application
version: 0.5.1
version: 0.5.2

View File

@@ -1,5 +1,5 @@
Higress successfully installed!
To learn more about the release, try:
$ helm status {{ .Release.Name }}
$ helm get all {{ .Release.Name }}
$ helm status {{ .Release.Name }} -n {{ .Release.Namespace }}
$ helm get all {{ .Release.Name }} -n {{ .Release.Namespace }}

View File

@@ -73,8 +73,10 @@ spec:
{{- end }}
readinessProbe:
{{- toYaml .Values.controller.probe | nindent 12 }}
{{- if not .Values.global.kind }}
resources:
{{- toYaml .Values.controller.resources | nindent 12 }}
{{- end }}
{{- with .Values.controller.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}

View File

@@ -9,7 +9,11 @@ metadata:
{{- .Values.gateway.annotations | toYaml | nindent 4 }}
spec:
{{- if not .Values.gateway.autoscaling.enabled }}
{{- if not .Values.global.kind }}
replicas: {{ .Values.gateway.replicaCount }}
{{- else }}
replicas: 1
{{- end }}
{{- end }}
selector:
matchLabels:
@@ -137,6 +141,16 @@ spec:
- containerPort: 15090
protocol: TCP
name: http-envoy-prom
{{- if .Values.global.kind }}
- containerPort: 80
hostPort: 80
name: http
protocol: TCP
- containerPort: 443
hostPort: 443
name: https
protocol: TCP
{{- end }}
readinessProbe:
failureThreshold: 30
httpGet:
@@ -147,8 +161,10 @@ spec:
periodSeconds: 2
successThreshold: 1
timeoutSeconds: 3
{{- if not .Values.global.kind }}
resources:
{{- toYaml .Values.gateway.resources | nindent 12 }}
{{- end }}
volumeMounts:
- name: config
mountPath: /etc/istio/config

View File

@@ -2,6 +2,7 @@ global:
# whether to use autoscaling/v2 template for HPA settings
# for internal usage only, not to be configured by users.
autoscalingv2API: true
kind: false
hub: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress
ingressClass: ""
watchNamespace: ""
@@ -13,7 +14,7 @@ gateway:
name: "higress-gateway"
replicaCount: 2
image: gateway
tag: "04cfef254aade907478c2b70cf576bb5eb310047"
tag: "268c73301e78514552fe0576fb0059c42c854a5c"
# revision declares which revision this gateway is a part of
revision: ""
@@ -104,7 +105,7 @@ controller:
name: "higress-controller"
replicaCount: 1
image: higress
tag: "04cfef254aade907478c2b70cf576bb5eb310047"
tag: "268c73301e78514552fe0576fb0059c42c854a5c"
env: {}
replicaCount: 1

View File

@@ -1,5 +1,5 @@
apiVersion: v2
appVersion: 1.12.1
appVersion: 1.12.2
description: Helm chart for deploying higress istio
name: istio
sources:
@@ -12,4 +12,4 @@ dependencies:
repository: "file://../istiod"
version: 1.12.0
type: application
version: 1.12.1
version: 1.12.2

View File

@@ -159,12 +159,14 @@ spec:
value: "{{ .Values.global.istiod.enableAnalysis }}"
- name: CLUSTER_ID
value: "{{ $.Values.global.multiCluster.clusterName | default `Kubernetes` }}"
{{- if not .Values.global.kind }}
resources:
{{- if .Values.pilot.resources }}
{{ toYaml .Values.pilot.resources | trim | indent 12 }}
{{- else }}
{{ toYaml .Values.global.defaultResources | trim | indent 12 }}
{{- end }}
{{- end }}
securityContext:
readOnlyRootFilesystem: true
runAsUser: 1337

View File

@@ -10,7 +10,7 @@ pilot:
rollingMaxUnavailable: 25%
hub: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress
tag: 04cfef254aade907478c2b70cf576bb5eb310047
tag: 268c73301e78514552fe0576fb0059c42c854a5c
# Can be a full hub/image:tag
image: pilot
@@ -221,6 +221,7 @@ meshConfig:
# No hurry to do this in 1.6, we're trying to prove the code.
global:
kind: false
# whether to use autoscaling/v2 template for HPA settings
# for internal usage only, not to be configured by users.
autoscalingv2API: true
@@ -253,7 +254,7 @@ global:
# Dev builds from prow are on gcr.io
hub: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress
# Default tag for Istio images.
tag: 1.12
tag: 268c73301e78514552fe0576fb0059c42c854a5c
# Specify image pull policy if default behavior isn't desired.
# Default behavior: latest images will be Always else IfNotPresent.

View File

@@ -0,0 +1,6 @@
dependencies:
- name: higress
repository: file://../../higress
version: 0.5.2
digest: sha256:414ef7a78a25d8f25d1f643edf4c8e59f74ecaa51a5ae4e4bff3000472b08fdb
generated: "2022-11-13T13:39:25.948896+08:00"

View File

@@ -0,0 +1,16 @@
apiVersion: v2
appVersion: 0.5.2
description: Helm chart for deploying higress gateways
icon: https://higress.io/img/higress_logo_small.png
keywords:
- higress
- gateways
name: higress-local
sources:
- http://github.com/alibaba/higress
dependencies:
- name: higress
repository: "file://../../higress"
version: 0.5.2
type: application
version: 0.5.2

View File

@@ -0,0 +1,2 @@
global:
kind: true

View File

@@ -0,0 +1,9 @@
dependencies:
- name: base
repository: file://../../base
version: 1.12.0
- name: istiod
repository: file://../../istiod
version: 1.12.0
digest: sha256:ce57290cabf7b0a85a150017cfdfcb2e25f18de7e8c93f5f3020750f9a65ece3
generated: "2022-11-13T13:50:54.129765+08:00"

View File

@@ -0,0 +1,15 @@
apiVersion: v2
appVersion: 1.12.2
description: Helm chart for deploying higress istio
name: istio-local
sources:
- http://github.com/alibaba/higress
dependencies:
- name: base
repository: "file://../../base"
version: 1.12.0
- name: istiod
repository: "file://../../istiod"
version: 1.12.0
type: application
version: 1.12.2

View File

@@ -0,0 +1,2 @@
global:
kind: true

View File

@@ -1,160 +0,0 @@
// Copyright (c) 2022 Alibaba Group Holding Ltd.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package annotations
import (
"strings"
networking "istio.io/api/networking/v1alpha3"
. "github.com/alibaba/higress/ingress/log"
)
const (
// request
requestHeaderAdd = "request-header-control-add"
requestHeaderUpdate = "request-header-control-update"
requestHeaderRemove = "request-header-control-remove"
// response
responseHeaderAdd = "response-header-control-add"
responseHeaderUpdate = "response-header-control-update"
responseHeaderRemove = "response-header-control-remove"
)
var (
_ Parser = headerControl{}
_ RouteHandler = headerControl{}
)
type HeaderOperation struct {
Add map[string]string
Update map[string]string
Remove []string
}
// HeaderControlConfig enforces header operations on route level.
// Note: Canary route don't use header control applied on the normal route.
type HeaderControlConfig struct {
Request *HeaderOperation
Response *HeaderOperation
}
type headerControl struct{}
func (h headerControl) Parse(annotations Annotations, config *Ingress, _ *GlobalContext) error {
if !needHeaderControlConfig(annotations) {
return nil
}
config.HeaderControl = &HeaderControlConfig{}
var requestAdd map[string]string
var requestUpdate map[string]string
var requestRemove []string
if add, err := annotations.ParseStringForMSE(requestHeaderAdd); err == nil {
requestAdd = convertAddOrUpdate(add)
}
if update, err := annotations.ParseStringForMSE(requestHeaderUpdate); err == nil {
requestUpdate = convertAddOrUpdate(update)
}
if remove, err := annotations.ParseStringForMSE(requestHeaderRemove); err == nil {
requestRemove = splitBySeparator(remove, ",")
}
if len(requestAdd) > 0 || len(requestUpdate) > 0 || len(requestRemove) > 0 {
config.HeaderControl.Request = &HeaderOperation{
Add: requestAdd,
Update: requestUpdate,
Remove: requestRemove,
}
}
var responseAdd map[string]string
var responseUpdate map[string]string
var responseRemove []string
if add, err := annotations.ParseStringForMSE(responseHeaderAdd); err == nil {
responseAdd = convertAddOrUpdate(add)
}
if update, err := annotations.ParseStringForMSE(responseHeaderUpdate); err == nil {
responseUpdate = convertAddOrUpdate(update)
}
if remove, err := annotations.ParseStringForMSE(responseHeaderRemove); err == nil {
responseRemove = splitBySeparator(remove, ",")
}
if len(responseAdd) > 0 || len(responseUpdate) > 0 || len(responseRemove) > 0 {
config.HeaderControl.Response = &HeaderOperation{
Add: responseAdd,
Update: responseUpdate,
Remove: responseRemove,
}
}
return nil
}
func (h headerControl) ApplyRoute(route *networking.HTTPRoute, config *Ingress) {
headerControlConfig := config.HeaderControl
if headerControlConfig == nil {
return
}
headers := &networking.Headers{
Request: &networking.Headers_HeaderOperations{},
Response: &networking.Headers_HeaderOperations{},
}
if headerControlConfig.Request != nil {
headers.Request.Add = headerControlConfig.Request.Add
headers.Request.Set = headerControlConfig.Request.Update
headers.Request.Remove = headerControlConfig.Request.Remove
}
if headerControlConfig.Response != nil {
headers.Response.Add = headerControlConfig.Response.Add
headers.Response.Set = headerControlConfig.Response.Update
headers.Response.Remove = headerControlConfig.Response.Remove
}
route.Headers = headers
}
func needHeaderControlConfig(annotations Annotations) bool {
return annotations.HasMSE(requestHeaderAdd) ||
annotations.HasMSE(requestHeaderUpdate) ||
annotations.HasMSE(requestHeaderRemove) ||
annotations.HasMSE(responseHeaderAdd) ||
annotations.HasMSE(responseHeaderUpdate) ||
annotations.HasMSE(responseHeaderRemove)
}
func convertAddOrUpdate(headers string) map[string]string {
result := map[string]string{}
parts := strings.Split(headers, "\n")
for _, part := range parts {
part = strings.TrimSpace(part)
if part == "" {
continue
}
keyValue := strings.Fields(part)
if len(keyValue) != 2 {
IngressLog.Infof("Header format %s is invalid.", keyValue)
continue
}
key := strings.TrimSpace(keyValue[0])
value := strings.TrimSpace(keyValue[1])
result[key] = value
}
return result
}

View File

@@ -1,235 +0,0 @@
// Copyright (c) 2022 Alibaba Group Holding Ltd.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package annotations
import (
"reflect"
"testing"
networking "istio.io/api/networking/v1alpha3"
)
func TestHeaderControlParse(t *testing.T) {
headerControl := &headerControl{}
inputCases := []struct {
input map[string]string
expect *HeaderControlConfig
}{
{},
{
input: map[string]string{
buildMSEAnnotationKey(requestHeaderAdd): "one 1",
buildMSEAnnotationKey(responseHeaderAdd): "A a",
},
expect: &HeaderControlConfig{
Request: &HeaderOperation{
Add: map[string]string{
"one": "1",
},
},
Response: &HeaderOperation{
Add: map[string]string{
"A": "a",
},
},
},
},
{
input: map[string]string{
buildMSEAnnotationKey(requestHeaderAdd): "one 1\n two 2\nthree 3 \n",
buildMSEAnnotationKey(requestHeaderUpdate): "two 2",
buildMSEAnnotationKey(requestHeaderRemove): "one, two,three\n",
buildMSEAnnotationKey(responseHeaderAdd): "A a\nB b\n",
buildMSEAnnotationKey(responseHeaderUpdate): "X x\nY y\n",
buildMSEAnnotationKey(responseHeaderRemove): "x",
},
expect: &HeaderControlConfig{
Request: &HeaderOperation{
Add: map[string]string{
"one": "1",
"two": "2",
"three": "3",
},
Update: map[string]string{
"two": "2",
},
Remove: []string{"one", "two", "three"},
},
Response: &HeaderOperation{
Add: map[string]string{
"A": "a",
"B": "b",
},
Update: map[string]string{
"X": "x",
"Y": "y",
},
Remove: []string{"x"},
},
},
},
}
for _, inputCase := range inputCases {
t.Run("", func(t *testing.T) {
config := &Ingress{}
_ = headerControl.Parse(inputCase.input, config, nil)
if !reflect.DeepEqual(inputCase.expect, config.HeaderControl) {
t.Fatal("Should be equal")
}
})
}
}
func TestHeaderControlApplyRoute(t *testing.T) {
headerControl := headerControl{}
inputCases := []struct {
config *Ingress
input *networking.HTTPRoute
expect *networking.HTTPRoute
}{
{
config: &Ingress{},
input: &networking.HTTPRoute{},
expect: &networking.HTTPRoute{},
},
{
config: &Ingress{
HeaderControl: &HeaderControlConfig{},
},
input: &networking.HTTPRoute{},
expect: &networking.HTTPRoute{
Headers: &networking.Headers{
Request: &networking.Headers_HeaderOperations{},
Response: &networking.Headers_HeaderOperations{},
},
},
},
{
config: &Ingress{
HeaderControl: &HeaderControlConfig{
Request: &HeaderOperation{
Add: map[string]string{
"one": "1",
"two": "2",
"three": "3",
},
Update: map[string]string{
"two": "2",
},
Remove: []string{"one", "two", "three"},
},
},
},
input: &networking.HTTPRoute{},
expect: &networking.HTTPRoute{
Headers: &networking.Headers{
Request: &networking.Headers_HeaderOperations{
Add: map[string]string{
"one": "1",
"two": "2",
"three": "3",
},
Set: map[string]string{
"two": "2",
},
Remove: []string{"one", "two", "three"},
},
Response: &networking.Headers_HeaderOperations{},
},
},
},
{
config: &Ingress{
HeaderControl: &HeaderControlConfig{
Response: &HeaderOperation{
Add: map[string]string{
"A": "a",
"B": "b",
},
Update: map[string]string{
"X": "x",
"Y": "y",
},
Remove: []string{"x"},
},
},
},
input: &networking.HTTPRoute{},
expect: &networking.HTTPRoute{
Headers: &networking.Headers{
Request: &networking.Headers_HeaderOperations{},
Response: &networking.Headers_HeaderOperations{
Add: map[string]string{
"A": "a",
"B": "b",
},
Set: map[string]string{
"X": "x",
"Y": "y",
},
Remove: []string{"x"},
},
},
},
},
{
config: &Ingress{
HeaderControl: &HeaderControlConfig{
Request: &HeaderOperation{
Update: map[string]string{
"two": "2",
},
Remove: []string{"one", "two", "three"},
},
Response: &HeaderOperation{
Add: map[string]string{
"A": "a",
"B": "b",
},
Remove: []string{"x"},
},
},
},
input: &networking.HTTPRoute{},
expect: &networking.HTTPRoute{
Headers: &networking.Headers{
Request: &networking.Headers_HeaderOperations{
Set: map[string]string{
"two": "2",
},
Remove: []string{"one", "two", "three"},
},
Response: &networking.Headers_HeaderOperations{
Add: map[string]string{
"A": "a",
"B": "b",
},
Remove: []string{"x"},
},
},
},
},
}
for _, inputCase := range inputCases {
t.Run("", func(t *testing.T) {
headerControl.ApplyRoute(inputCase.input, inputCase.config)
if !reflect.DeepEqual(inputCase.input, inputCase.expect) {
t.Fatal("Should be equal")
}
})
}
}

View File

@@ -1,241 +0,0 @@
// Copyright (c) 2022 Alibaba Group Holding Ltd.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package annotations
import (
"reflect"
"testing"
networking "istio.io/api/networking/v1alpha3"
)
func TestIPAccessControlParse(t *testing.T) {
parser := ipAccessControl{}
testCases := []struct {
input map[string]string
expect *IPAccessControlConfig
}{
{},
{
input: map[string]string{
DefaultAnnotationsPrefix + "/" + whitelist: "1.1.1.1",
MSEAnnotationsPrefix + "/" + blacklist: "2.2.2.2",
},
expect: &IPAccessControlConfig{
Route: &IPAccessControl{
isWhite: true,
remoteIp: []string{"1.1.1.1"},
},
},
},
{
input: map[string]string{
MSEAnnotationsPrefix + "/" + blacklist: "2.2.2.2",
},
expect: &IPAccessControlConfig{
Route: &IPAccessControl{
isWhite: false,
remoteIp: []string{"2.2.2.2"},
},
},
},
{
input: map[string]string{
MSEAnnotationsPrefix + "/" + domainWhitelist: "1.1.1.1",
},
expect: &IPAccessControlConfig{
Domain: &IPAccessControl{
isWhite: true,
remoteIp: []string{"1.1.1.1"},
},
},
},
{
input: map[string]string{
MSEAnnotationsPrefix + "/" + whitelist: "1.1.1.1, 3.3.3.3",
MSEAnnotationsPrefix + "/" + domainBlacklist: "2.2.2.2",
},
expect: &IPAccessControlConfig{
Route: &IPAccessControl{
isWhite: true,
remoteIp: []string{"1.1.1.1", "3.3.3.3"},
},
Domain: &IPAccessControl{
isWhite: false,
remoteIp: []string{"2.2.2.2"},
},
},
},
}
for _, testCase := range testCases {
t.Run("", func(t *testing.T) {
config := &Ingress{}
_ = parser.Parse(testCase.input, config, nil)
if !reflect.DeepEqual(testCase.expect, config.IPAccessControl) {
t.Fatalf("Should be equal")
}
})
}
}
func TestIpAccessControl_ApplyVirtualServiceHandler(t *testing.T) {
parser := ipAccessControl{}
testCases := []struct {
config *Ingress
input *networking.VirtualService
expect *networking.HTTPFilter
}{
{
config: &Ingress{},
input: &networking.VirtualService{},
expect: nil,
},
{
config: &Ingress{
IPAccessControl: &IPAccessControlConfig{
Domain: &IPAccessControl{
isWhite: true,
remoteIp: []string{"1.1.1.1"},
},
},
},
input: &networking.VirtualService{},
expect: &networking.HTTPFilter{
Name: "ip-access-control",
Disable: false,
Filter: &networking.HTTPFilter_IpAccessControl{
IpAccessControl: &networking.IPAccessControl{
RemoteIpBlocks: []string{"1.1.1.1"},
},
},
},
},
{
config: &Ingress{
IPAccessControl: &IPAccessControlConfig{
Domain: &IPAccessControl{
isWhite: false,
remoteIp: []string{"2.2.2.2"},
},
},
},
input: &networking.VirtualService{},
expect: &networking.HTTPFilter{
Name: "ip-access-control",
Disable: false,
Filter: &networking.HTTPFilter_IpAccessControl{
IpAccessControl: &networking.IPAccessControl{
NotRemoteIpBlocks: []string{"2.2.2.2"},
},
},
},
},
}
for _, testCase := range testCases {
t.Run("", func(t *testing.T) {
parser.ApplyVirtualServiceHandler(testCase.input, testCase.config)
if testCase.config.IPAccessControl == nil {
if len(testCase.input.HostHTTPFilters) != 0 {
t.Fatalf("Should be empty")
}
} else {
if len(testCase.input.HostHTTPFilters) == 0 {
t.Fatalf("Should be not empty")
}
if !reflect.DeepEqual(testCase.expect, testCase.input.HostHTTPFilters[0]) {
t.Fatalf("Should be equal")
}
}
})
}
}
func TestIpAccessControl_ApplyRoute(t *testing.T) {
parser := ipAccessControl{}
testCases := []struct {
config *Ingress
input *networking.HTTPRoute
expect *networking.HTTPFilter
}{
{
config: &Ingress{},
input: &networking.HTTPRoute{},
expect: nil,
},
{
config: &Ingress{
IPAccessControl: &IPAccessControlConfig{
Route: &IPAccessControl{
isWhite: true,
remoteIp: []string{"1.1.1.1"},
},
},
},
input: &networking.HTTPRoute{},
expect: &networking.HTTPFilter{
Name: "ip-access-control",
Disable: false,
Filter: &networking.HTTPFilter_IpAccessControl{
IpAccessControl: &networking.IPAccessControl{
RemoteIpBlocks: []string{"1.1.1.1"},
},
},
},
},
{
config: &Ingress{
IPAccessControl: &IPAccessControlConfig{
Route: &IPAccessControl{
isWhite: false,
remoteIp: []string{"2.2.2.2"},
},
},
},
input: &networking.HTTPRoute{},
expect: &networking.HTTPFilter{
Name: "ip-access-control",
Disable: false,
Filter: &networking.HTTPFilter_IpAccessControl{
IpAccessControl: &networking.IPAccessControl{
NotRemoteIpBlocks: []string{"2.2.2.2"},
},
},
},
},
}
for _, testCase := range testCases {
t.Run("", func(t *testing.T) {
parser.ApplyRoute(testCase.input, testCase.config)
if testCase.config.IPAccessControl == nil {
if len(testCase.input.RouteHTTPFilters) != 0 {
t.Fatalf("Should be empty")
}
} else {
if len(testCase.input.RouteHTTPFilters) == 0 {
t.Fatalf("Should be not empty")
}
if !reflect.DeepEqual(testCase.expect, testCase.input.RouteHTTPFilters[0]) {
t.Fatalf("Should be equal")
}
}
})
}
}

View File

@@ -1,110 +0,0 @@
// Copyright (c) 2022 Alibaba Group Holding Ltd.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package annotations
import (
"github.com/gogo/protobuf/types"
networking "istio.io/api/networking/v1alpha3"
"istio.io/istio/pilot/pkg/networking/core/v1alpha3/mseingress"
)
const (
limitRPM = "route-limit-rpm"
limitRPS = "route-limit-rps"
limitBurstMultiplier = "route-limit-burst-multiplier"
defaultBurstMultiplier = 5
defaultStatusCode = 503
)
var (
_ Parser = localRateLimit{}
_ RouteHandler = localRateLimit{}
second = &types.Duration{
Seconds: 1,
}
minute = &types.Duration{
Seconds: 60,
}
)
type localRateLimitConfig struct {
TokensPerFill uint32
MaxTokens uint32
FillInterval *types.Duration
}
type localRateLimit struct{}
func (l localRateLimit) Parse(annotations Annotations, config *Ingress, _ *GlobalContext) error {
if !needLocalRateLimitConfig(annotations) {
return nil
}
var local *localRateLimitConfig
defer func() {
config.localRateLimit = local
}()
var multiplier uint32 = defaultBurstMultiplier
if m, err := annotations.ParseUint32ForMSE(limitBurstMultiplier); err == nil {
multiplier = m
}
if rpm, err := annotations.ParseUint32ForMSE(limitRPM); err == nil {
local = &localRateLimitConfig{
MaxTokens: rpm * multiplier,
TokensPerFill: rpm,
FillInterval: minute,
}
} else if rps, err := annotations.ParseUint32ForMSE(limitRPS); err == nil {
local = &localRateLimitConfig{
MaxTokens: rps * multiplier,
TokensPerFill: rps,
FillInterval: second,
}
}
return nil
}
func (l localRateLimit) ApplyRoute(route *networking.HTTPRoute, config *Ingress) {
localRateLimitConfig := config.localRateLimit
if localRateLimitConfig == nil {
return
}
route.RouteHTTPFilters = append(route.RouteHTTPFilters, &networking.HTTPFilter{
Name: mseingress.LocalRateLimit,
Filter: &networking.HTTPFilter_LocalRateLimit{
LocalRateLimit: &networking.LocalRateLimit{
TokenBucket: &networking.TokenBucket{
MaxTokens: localRateLimitConfig.MaxTokens,
TokensPefFill: localRateLimitConfig.TokensPerFill,
FillInterval: localRateLimitConfig.FillInterval,
},
StatusCode: defaultStatusCode,
},
},
})
}
func needLocalRateLimitConfig(annotations Annotations) bool {
return annotations.HasMSE(limitRPM) ||
annotations.HasMSE(limitRPS)
}

View File

@@ -1,127 +0,0 @@
// Copyright (c) 2022 Alibaba Group Holding Ltd.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package annotations
import (
"reflect"
"testing"
networking "istio.io/api/networking/v1alpha3"
"istio.io/istio/pilot/pkg/networking/core/v1alpha3/mseingress"
)
func TestLocalRateLimitParse(t *testing.T) {
localRateLimit := localRateLimit{}
inputCases := []struct {
input map[string]string
expect *localRateLimitConfig
}{
{},
{
input: map[string]string{
buildMSEAnnotationKey(limitRPM): "2",
},
expect: &localRateLimitConfig{
MaxTokens: 10,
TokensPerFill: 2,
FillInterval: minute,
},
},
{
input: map[string]string{
buildMSEAnnotationKey(limitRPM): "2",
buildMSEAnnotationKey(limitRPS): "3",
buildMSEAnnotationKey(limitBurstMultiplier): "10",
},
expect: &localRateLimitConfig{
MaxTokens: 20,
TokensPerFill: 2,
FillInterval: minute,
},
},
{
input: map[string]string{
buildMSEAnnotationKey(limitRPS): "3",
buildMSEAnnotationKey(limitBurstMultiplier): "10",
},
expect: &localRateLimitConfig{
MaxTokens: 30,
TokensPerFill: 3,
FillInterval: second,
},
},
}
for _, inputCase := range inputCases {
t.Run("", func(t *testing.T) {
config := &Ingress{}
_ = localRateLimit.Parse(inputCase.input, config, nil)
if !reflect.DeepEqual(inputCase.expect, config.localRateLimit) {
t.Fatal("Should be equal")
}
})
}
}
func TestLocalRateLimitApplyRoute(t *testing.T) {
localRateLimit := localRateLimit{}
inputCases := []struct {
config *Ingress
input *networking.HTTPRoute
expect *networking.HTTPRoute
}{
{
config: &Ingress{},
input: &networking.HTTPRoute{},
expect: &networking.HTTPRoute{},
},
{
config: &Ingress{
localRateLimit: &localRateLimitConfig{
MaxTokens: 60,
TokensPerFill: 20,
FillInterval: second,
},
},
input: &networking.HTTPRoute{},
expect: &networking.HTTPRoute{
RouteHTTPFilters: []*networking.HTTPFilter{
{
Name: mseingress.LocalRateLimit,
Filter: &networking.HTTPFilter_LocalRateLimit{
LocalRateLimit: &networking.LocalRateLimit{
TokenBucket: &networking.TokenBucket{
MaxTokens: 60,
TokensPefFill: 20,
FillInterval: second,
},
StatusCode: defaultStatusCode,
},
},
},
},
},
},
}
for _, inputCase := range inputCases {
t.Run("", func(t *testing.T) {
localRateLimit.ApplyRoute(inputCase.input, inputCase.config)
if !reflect.DeepEqual(inputCase.input, inputCase.expect) {
t.Fatal("Should be equal")
}
})
}
}

View File

@@ -1,62 +0,0 @@
// Copyright (c) 2022 Alibaba Group Holding Ltd.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package annotations
import (
"github.com/gogo/protobuf/types"
networking "istio.io/api/networking/v1alpha3"
)
const timeoutAnnotation = "timeout"
var (
_ Parser = timeout{}
_ RouteHandler = timeout{}
)
type TimeoutConfig struct {
time *types.Duration
}
type timeout struct{}
func (t timeout) Parse(annotations Annotations, config *Ingress, _ *GlobalContext) error {
if !needTimeoutConfig(annotations) {
return nil
}
if time, err := annotations.ParseIntForMSE(timeoutAnnotation); err == nil {
config.Timeout = &TimeoutConfig{
time: &types.Duration{
Seconds: int64(time),
},
}
}
return nil
}
func (t timeout) ApplyRoute(route *networking.HTTPRoute, config *Ingress) {
timeout := config.Timeout
if timeout == nil || timeout.time == nil || timeout.time.Seconds == 0 {
return
}
route.Timeout = timeout.time
}
func needTimeoutConfig(annotations Annotations) bool {
return annotations.HasMSE(timeoutAnnotation)
}

View File

@@ -1,121 +0,0 @@
// Copyright (c) 2022 Alibaba Group Holding Ltd.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package annotations
import (
"reflect"
"testing"
"github.com/gogo/protobuf/types"
networking "istio.io/api/networking/v1alpha3"
)
func TestTimeoutParse(t *testing.T) {
timeout := timeout{}
inputCases := []struct {
input map[string]string
expect *TimeoutConfig
}{
{},
{
input: map[string]string{
MSEAnnotationsPrefix + "/" + timeoutAnnotation: "",
},
},
{
input: map[string]string{
MSEAnnotationsPrefix + "/" + timeoutAnnotation: "0",
},
expect: &TimeoutConfig{
time: &types.Duration{},
},
},
{
input: map[string]string{
MSEAnnotationsPrefix + "/" + timeoutAnnotation: "10",
},
expect: &TimeoutConfig{
time: &types.Duration{
Seconds: 10,
},
},
},
}
for _, c := range inputCases {
t.Run("", func(t *testing.T) {
config := &Ingress{}
_ = timeout.Parse(c.input, config, nil)
if !reflect.DeepEqual(c.expect, config.Timeout) {
t.Fatalf("Should be equal.")
}
})
}
}
func TestTimeoutApplyRoute(t *testing.T) {
timeout := timeout{}
inputCases := []struct {
config *Ingress
input *networking.HTTPRoute
expect *networking.HTTPRoute
}{
{
config: &Ingress{},
input: &networking.HTTPRoute{},
expect: &networking.HTTPRoute{},
},
{
config: &Ingress{
Timeout: &TimeoutConfig{},
},
input: &networking.HTTPRoute{},
expect: &networking.HTTPRoute{},
},
{
config: &Ingress{
Timeout: &TimeoutConfig{
time: &types.Duration{},
},
},
input: &networking.HTTPRoute{},
expect: &networking.HTTPRoute{},
},
{
config: &Ingress{
Timeout: &TimeoutConfig{
time: &types.Duration{
Seconds: 10,
},
},
},
input: &networking.HTTPRoute{},
expect: &networking.HTTPRoute{
Timeout: &types.Duration{
Seconds: 10,
},
},
},
}
for _, inputCase := range inputCases {
t.Run("", func(t *testing.T) {
timeout.ApplyRoute(inputCase.input, inputCase.config)
if !reflect.DeepEqual(inputCase.input, inputCase.expect) {
t.Fatalf("Should be equal")
}
})
}
}

View File

@@ -0,0 +1,185 @@
diff --git a/pkg/istio-agent/xds_proxy.go b/pkg/istio-agent/xds_proxy.go
index 51766751e1..5b33234024 100644
--- a/pkg/istio-agent/xds_proxy.go
+++ b/pkg/istio-agent/xds_proxy.go
@@ -47,6 +47,7 @@ import (
"istio.io/istio/pilot/pkg/features"
istiogrpc "istio.io/istio/pilot/pkg/grpc"
v3 "istio.io/istio/pilot/pkg/xds/v3"
+ "istio.io/istio/pkg/channels"
"istio.io/istio/pkg/config/constants"
dnsProto "istio.io/istio/pkg/dns/proto"
"istio.io/istio/pkg/istio-agent/health"
@@ -233,24 +234,14 @@ func initXdsProxy(ia *Agent) (*XdsProxy, error) {
// PersistRequest sends a request to the currently connected proxy. Additionally, on any reconnection
// to the upstream XDS request we will resend this request.
func (p *XdsProxy) PersistRequest(req *discovery.DiscoveryRequest) {
- var ch chan *discovery.DiscoveryRequest
- var stop chan struct{}
-
p.connectedMutex.Lock()
- if p.connected != nil {
- ch = p.connected.requestsChan
- stop = p.connected.stopChan
+ // Immediately send if we are currently connect
+ if p.connected != nil && p.connected.requestsChan != nil {
+ p.connected.requestsChan.Put(req)
}
+ // Otherwise place it as our initial request for new connections
p.initialRequest = req
p.connectedMutex.Unlock()
-
- // Immediately send if we are currently connect
- if ch != nil {
- select {
- case ch <- req:
- case <-stop:
- }
- }
}
func (p *XdsProxy) UnregisterStream(c *ProxyConnection) {
@@ -276,9 +267,9 @@ type ProxyConnection struct {
conID uint32
upstreamError chan error
downstreamError chan error
- requestsChan chan *discovery.DiscoveryRequest
+ requestsChan *channels.Unbounded
responsesChan chan *discovery.DiscoveryResponse
- deltaRequestsChan chan *discovery.DeltaDiscoveryRequest
+ deltaRequestsChan *channels.Unbounded
deltaResponsesChan chan *discovery.DeltaDiscoveryResponse
stopChan chan struct{}
downstream adsStream
@@ -290,10 +281,7 @@ type ProxyConnection struct {
// sendRequest is a small wrapper around sending to con.requestsChan. This ensures that we do not
// block forever on
func (con *ProxyConnection) sendRequest(req *discovery.DiscoveryRequest) {
- select {
- case con.requestsChan <- req:
- case <-con.stopChan:
- }
+ con.requestsChan.Put(req)
}
type adsStream interface {
@@ -315,10 +303,29 @@ func (p *XdsProxy) handleStream(downstream adsStream) error {
conID: connectionNumber.Inc(),
upstreamError: make(chan error, 2), // can be produced by recv and send
downstreamError: make(chan error, 2), // can be produced by recv and send
- requestsChan: make(chan *discovery.DiscoveryRequest, 10),
- responsesChan: make(chan *discovery.DiscoveryResponse, 10),
- stopChan: make(chan struct{}),
- downstream: downstream,
+ // Requests channel is unbounded. The Envoy<->XDS Proxy<->Istiod system produces a natural
+ // looping of Recv and Send. Due to backpressure introduce by gRPC natively (that is, Send() can
+ // only send so much data without being Recv'd before it starts blocking), along with the
+ // backpressure provided by our channels, we have a risk of deadlock where both xdsproxy and
+ // Istiod are trying to Send, but both are blocked by gRPC backpressure until Recv() is called.
+ // However, Recv can fail to be called by Send being blocked. This can be triggered by the two
+ // sources in our system (Envoy request and Istiod pushes) producing more events than we can keep
+ // up with.
+ // See https://github.com/istio/istio/issues/39209 for more information
+ //
+ // To prevent these issues, we need to either:
+ // 1. Apply backpressure directly to Envoy requests or Istiod pushes
+ // 2. Make part of the system unbounded
+ //
+ // (1) is challenging because we cannot do a conditional Recv (for Envoy requests), and changing
+ // the control plane requires substantial changes. Instead, we make the requests channel
+ // unbounded. This is the least likely to cause issues as the messages we store here are the
+ // smallest relative to other channels.
+ requestsChan: channels.NewUnbounded(),
+ // Allow a buffer of 1. This ensures we queue up at most 2 (one in process, 1 pending) responses before forwarding.
+ responsesChan: make(chan *discovery.DiscoveryResponse, 1),
+ stopChan: make(chan struct{}),
+ downstream: downstream,
}
p.RegisterStream(con)
@@ -452,7 +459,9 @@ func (p *XdsProxy) handleUpstreamRequest(con *ProxyConnection) {
defer con.upstream.CloseSend() // nolint
for {
select {
- case req := <-con.requestsChan:
+ case requ := <-con.requestsChan.Get():
+ con.requestsChan.Load()
+ req := requ.(*discovery.DiscoveryRequest)
proxyLog.Debugf("request for type url %s", req.TypeUrl)
metrics.XdsProxyRequests.Increment()
if req.TypeUrl == v3.ExtensionConfigurationType {
diff --git a/pkg/istio-agent/xds_proxy_delta.go b/pkg/istio-agent/xds_proxy_delta.go
index 2cd82b4059..c2245f9918 100644
--- a/pkg/istio-agent/xds_proxy_delta.go
+++ b/pkg/istio-agent/xds_proxy_delta.go
@@ -27,6 +27,7 @@ import (
"istio.io/istio/pilot/pkg/features"
istiogrpc "istio.io/istio/pilot/pkg/grpc"
v3 "istio.io/istio/pilot/pkg/xds/v3"
+ "istio.io/istio/pkg/channels"
"istio.io/istio/pkg/istio-agent/metrics"
"istio.io/istio/pkg/wasm"
)
@@ -34,10 +35,7 @@ import (
// sendDeltaRequest is a small wrapper around sending to con.requestsChan. This ensures that we do not
// block forever on
func (con *ProxyConnection) sendDeltaRequest(req *discovery.DeltaDiscoveryRequest) {
- select {
- case con.deltaRequestsChan <- req:
- case <-con.stopChan:
- }
+ con.deltaRequestsChan.Put(req)
}
// requests from envoy
@@ -48,10 +46,11 @@ func (p *XdsProxy) DeltaAggregatedResources(downstream discovery.AggregatedDisco
proxyLog.Debugf("accepted delta xds connection from envoy, forwarding to upstream")
con := &ProxyConnection{
- upstreamError: make(chan error, 2), // can be produced by recv and send
- downstreamError: make(chan error, 2), // can be produced by recv and send
- deltaRequestsChan: make(chan *discovery.DeltaDiscoveryRequest, 10),
- deltaResponsesChan: make(chan *discovery.DeltaDiscoveryResponse, 10),
+ upstreamError: make(chan error, 2), // can be produced by recv and send
+ downstreamError: make(chan error, 2), // can be produced by recv and send
+ deltaRequestsChan: channels.NewUnbounded(),
+ // Allow a buffer of 1. This ensures we queue up at most 2 (one in process, 1 pending) responses before forwarding.
+ deltaResponsesChan: make(chan *discovery.DeltaDiscoveryResponse, 1),
stopChan: make(chan struct{}),
downstreamDeltas: downstream,
}
@@ -190,7 +189,9 @@ func (p *XdsProxy) handleUpstreamDeltaRequest(con *ProxyConnection) {
}()
for {
select {
- case req := <-con.deltaRequestsChan:
+ case requ := <-con.deltaRequestsChan.Get():
+ con.deltaRequestsChan.Load()
+ req := requ.(*discovery.DeltaDiscoveryRequest)
proxyLog.Debugf("delta request for type url %s", req.TypeUrl)
metrics.XdsProxyRequests.Increment()
if req.TypeUrl == v3.ExtensionConfigurationType {
@@ -296,22 +297,10 @@ func sendDownstreamDelta(deltaDownstream discovery.AggregatedDiscoveryService_De
}
func (p *XdsProxy) PersistDeltaRequest(req *discovery.DeltaDiscoveryRequest) {
- var ch chan *discovery.DeltaDiscoveryRequest
- var stop chan struct{}
-
p.connectedMutex.Lock()
if p.connected != nil {
- ch = p.connected.deltaRequestsChan
- stop = p.connected.stopChan
+ p.connected.deltaRequestsChan.Put(req)
}
p.initialDeltaRequest = req
p.connectedMutex.Unlock()
-
- // Immediately send if we are currently connect
- if ch != nil {
- select {
- case ch <- req:
- case <-stop:
- }
- }
}

View File

@@ -0,0 +1,139 @@
diff -Naur base/pilot/pkg/xds/xdsgen.go istio/pilot/pkg/xds/xdsgen.go
--- base/pilot/pkg/xds/xdsgen.go 2022-11-10 20:45:14.000000000 +0800
+++ istio/pilot/pkg/xds/xdsgen.go 2022-11-10 20:16:35.000000000 +0800
@@ -96,40 +96,23 @@
return nil
}
-// End added by ingress
-
-// Push an XDS resource for the given connection. Configuration will be generated
-// based on the passed in generator. Based on the updates field, generators may
-// choose to send partial or even no response if there are no changes.
-func (s *DiscoveryServer) pushXds(con *Connection, push *model.PushContext,
- w *model.WatchedResource, req *model.PushRequest) error {
+func (s *DiscoveryServer) pushMcpXds(con *Connection, push *model.PushContext, w *model.WatchedResource, req *model.PushRequest) error {
if w == nil {
return nil
}
t0 := time.Now()
- // Modified by ingress
+
var (
- res []*any.Any
logdata model.XdsLogDetails
err error
)
- if s.Env.MCPMode {
- res = make([]*any.Any, 0)
- gen := s.findMcpGenerator(w.TypeUrl, con)
- if gen != nil {
- res, logdata, err = gen.Generate(con.proxy, push, w, req)
- }
- } else {
- gen := s.findGenerator(w.TypeUrl, con)
- if gen == nil {
- return nil
- }
- var resource model.Resources
- resource, logdata, err = gen.Generate(con.proxy, push, w, req)
- res = model.ResourcesToAny(resource)
+ res := make([]*any.Any, 0)
+ gen := s.findMcpGenerator(w.TypeUrl, con)
+ if gen != nil {
+ res, logdata, err = gen.Generate(con.proxy, push, w, req)
}
- if err != nil || res == nil {
+ if err != nil {
// If we have nothing to send, report that we got an ACK for this version.
if s.StatusReporter != nil {
s.StatusReporter.RegisterEvent(con.ConID, w.TypeUrl, push.LedgerVersion)
@@ -181,7 +164,86 @@
log.Infof("%s: %s%s for node:%s resources:%d size:%v%s%s", v3.GetShortType(w.TypeUrl), ptype, req.PushReason(), con.proxy.ID, len(res),
util.ByteCount(AnyResourceSize(res)), info, debug)
}
- // End modified by ingress
+ return nil
+}
+
+// End added by ingress
+
+// Push an XDS resource for the given connection. Configuration will be generated
+// based on the passed in generator. Based on the updates field, generators may
+// choose to send partial or even no response if there are no changes.
+func (s *DiscoveryServer) pushXds(con *Connection, push *model.PushContext,
+ w *model.WatchedResource, req *model.PushRequest) error {
+ // Added by ingress
+ if s.Env.MCPMode {
+ return s.pushMcpXds(con, push, w, req)
+ }
+ // End added by ingress
+ if w == nil {
+ return nil
+ }
+ gen := s.findGenerator(w.TypeUrl, con)
+ if gen == nil {
+ return nil
+ }
+
+ t0 := time.Now()
+
+ res, logdata, err := gen.Generate(con.proxy, push, w, req)
+ if err != nil || res == nil {
+ // If we have nothing to send, report that we got an ACK for this version.
+ if s.StatusReporter != nil {
+ s.StatusReporter.RegisterEvent(con.ConID, w.TypeUrl, push.LedgerVersion)
+ }
+ return err
+ }
+ defer func() { recordPushTime(w.TypeUrl, time.Since(t0)) }()
+
+ resp := &discovery.DiscoveryResponse{
+ ControlPlane: ControlPlane(),
+ TypeUrl: w.TypeUrl,
+ // TODO: send different version for incremental eds
+ VersionInfo: push.PushVersion,
+ Nonce: nonce(push.LedgerVersion),
+ Resources: model.ResourcesToAny(res),
+ }
+
+ configSize := ResourceSize(res)
+ configSizeBytes.With(typeTag.Value(w.TypeUrl)).Record(float64(configSize))
+
+ ptype := "PUSH"
+ info := ""
+ if logdata.Incremental {
+ ptype = "PUSH INC"
+ }
+ if len(logdata.AdditionalInfo) > 0 {
+ info = " " + logdata.AdditionalInfo
+ }
+
+ if err := con.send(resp); err != nil {
+ if recordSendError(w.TypeUrl, err) {
+ log.Warnf("%s: Send failure for node:%s resources:%d size:%s%s: %v",
+ v3.GetShortType(w.TypeUrl), con.proxy.ID, len(res), util.ByteCount(configSize), info, err)
+ }
+ return err
+ }
+
+ switch {
+ case logdata.Incremental:
+ if log.DebugEnabled() {
+ log.Debugf("%s: %s%s for node:%s resources:%d size:%s%s",
+ v3.GetShortType(w.TypeUrl), ptype, req.PushReason(), con.proxy.ID, len(res), util.ByteCount(configSize), info)
+ }
+ default:
+ debug := ""
+ if log.DebugEnabled() {
+ // Add additional information to logs when debug mode enabled.
+ debug = " nonce:" + resp.Nonce + " version:" + resp.VersionInfo
+ }
+ log.Infof("%s: %s%s for node:%s resources:%d size:%v%s%s", v3.GetShortType(w.TypeUrl), ptype, req.PushReason(), con.proxy.ID, len(res),
+ util.ByteCount(ResourceSize(res)), info, debug)
+ }
+
return nil
}

View File

@@ -20,7 +20,6 @@ import (
"net/http"
"time"
"github.com/alibaba/higress/ingress/kube/common"
prometheus "github.com/grpc-ecosystem/go-grpc-prometheus"
"google.golang.org/grpc"
"google.golang.org/grpc/reflection"
@@ -46,8 +45,9 @@ import (
"k8s.io/client-go/rest"
"k8s.io/client-go/tools/cache"
ingressconfig "github.com/alibaba/higress/ingress/config"
"github.com/alibaba/higress/ingress/mcp"
ingressconfig "github.com/alibaba/higress/pkg/ingress/config"
"github.com/alibaba/higress/pkg/ingress/kube/common"
"github.com/alibaba/higress/pkg/ingress/mcp"
)
type XdsOptions struct {

View File

@@ -37,13 +37,13 @@ import (
listersv1 "k8s.io/client-go/listers/core/v1"
"k8s.io/client-go/tools/cache"
"github.com/alibaba/higress/ingress/kube/annotations"
"github.com/alibaba/higress/ingress/kube/common"
"github.com/alibaba/higress/ingress/kube/ingress"
"github.com/alibaba/higress/ingress/kube/ingressv1"
secretkube "github.com/alibaba/higress/ingress/kube/secret/kube"
"github.com/alibaba/higress/ingress/kube/util"
. "github.com/alibaba/higress/ingress/log"
"github.com/alibaba/higress/pkg/ingress/kube/annotations"
"github.com/alibaba/higress/pkg/ingress/kube/common"
"github.com/alibaba/higress/pkg/ingress/kube/ingress"
"github.com/alibaba/higress/pkg/ingress/kube/ingressv1"
secretkube "github.com/alibaba/higress/pkg/ingress/kube/secret/kube"
"github.com/alibaba/higress/pkg/ingress/kube/util"
. "github.com/alibaba/higress/pkg/ingress/log"
)
var (

View File

@@ -28,10 +28,10 @@ import (
ingress "k8s.io/api/networking/v1"
ingressv1beta1 "k8s.io/api/networking/v1beta1"
"github.com/alibaba/higress/ingress/kube/annotations"
"github.com/alibaba/higress/ingress/kube/common"
controllerv1beta1 "github.com/alibaba/higress/ingress/kube/ingress"
controllerv1 "github.com/alibaba/higress/ingress/kube/ingressv1"
"github.com/alibaba/higress/pkg/ingress/kube/annotations"
"github.com/alibaba/higress/pkg/ingress/kube/common"
controllerv1beta1 "github.com/alibaba/higress/pkg/ingress/kube/ingress"
controllerv1 "github.com/alibaba/higress/pkg/ingress/kube/ingressv1"
)
func TestNormalizeWeightedCluster(t *testing.T) {
@@ -182,8 +182,7 @@ func TestConvertGatewaysForIngress(t *testing.T) {
},
AnnotationsConfig: &annotations.Ingress{
DownstreamTLS: &annotations.DownstreamTLSConfig{
TlsMinVersion: annotations.TLSProtocolVersion("TLSv1.1"),
CipherSuites: []string{"ECDHE-RSA-AES128-GCM-SHA256", "AES256-SHA"},
CipherSuites: []string{"ECDHE-RSA-AES128-GCM-SHA256", "AES256-SHA"},
},
},
},
@@ -249,8 +248,7 @@ func TestConvertGatewaysForIngress(t *testing.T) {
},
AnnotationsConfig: &annotations.Ingress{
DownstreamTLS: &annotations.DownstreamTLSConfig{
TlsMinVersion: annotations.TLSProtocolVersion("TLSv1.2"),
CipherSuites: []string{"ECDHE-RSA-AES128-GCM-SHA256"},
CipherSuites: []string{"ECDHE-RSA-AES128-GCM-SHA256"},
},
},
},
@@ -284,10 +282,9 @@ func TestConvertGatewaysForIngress(t *testing.T) {
},
Hosts: []string{"foo.com"},
Tls: &networking.ServerTLSSettings{
Mode: networking.ServerTLSSettings_SIMPLE,
CredentialName: "kubernetes-ingress://ingress-v1beta1__/wakanda/foo-com",
MinProtocolVersion: networking.ServerTLSSettings_TLSV1_1,
CipherSuites: []string{"ECDHE-RSA-AES128-GCM-SHA256", "AES256-SHA"},
Mode: networking.ServerTLSSettings_SIMPLE,
CredentialName: "kubernetes-ingress://ingress-v1beta1__/wakanda/foo-com",
CipherSuites: []string{"ECDHE-RSA-AES128-GCM-SHA256", "AES256-SHA"},
},
},
},
@@ -321,10 +318,9 @@ func TestConvertGatewaysForIngress(t *testing.T) {
},
Hosts: []string{"test.com"},
Tls: &networking.ServerTLSSettings{
Mode: networking.ServerTLSSettings_SIMPLE,
CredentialName: "kubernetes-ingress://ingress-v1beta1__/wakanda/test-com",
MinProtocolVersion: networking.ServerTLSSettings_TLSV1_1,
CipherSuites: []string{"ECDHE-RSA-AES128-GCM-SHA256", "AES256-SHA"},
Mode: networking.ServerTLSSettings_SIMPLE,
CredentialName: "kubernetes-ingress://ingress-v1beta1__/wakanda/test-com",
CipherSuites: []string{"ECDHE-RSA-AES128-GCM-SHA256", "AES256-SHA"},
},
},
},
@@ -466,8 +462,7 @@ func TestConvertGatewaysForIngress(t *testing.T) {
},
AnnotationsConfig: &annotations.Ingress{
DownstreamTLS: &annotations.DownstreamTLSConfig{
TlsMinVersion: annotations.TLSProtocolVersion("TLSv1.2"),
CipherSuites: []string{"ECDHE-RSA-AES128-GCM-SHA256"},
CipherSuites: []string{"ECDHE-RSA-AES128-GCM-SHA256"},
},
},
},
@@ -501,10 +496,9 @@ func TestConvertGatewaysForIngress(t *testing.T) {
},
Hosts: []string{"foo.com"},
Tls: &networking.ServerTLSSettings{
Mode: networking.ServerTLSSettings_SIMPLE,
CredentialName: "kubernetes-ingress://ingress-v1__/wakanda/foo-com",
MinProtocolVersion: networking.ServerTLSSettings_TLSV1_2,
CipherSuites: []string{"ECDHE-RSA-AES128-GCM-SHA256"},
Mode: networking.ServerTLSSettings_SIMPLE,
CredentialName: "kubernetes-ingress://ingress-v1__/wakanda/foo-com",
CipherSuites: []string{"ECDHE-RSA-AES128-GCM-SHA256"},
},
},
},
@@ -538,10 +532,9 @@ func TestConvertGatewaysForIngress(t *testing.T) {
},
Hosts: []string{"test.com"},
Tls: &networking.ServerTLSSettings{
Mode: networking.ServerTLSSettings_SIMPLE,
CredentialName: "kubernetes-ingress://ingress-v1__/wakanda/test-com",
MinProtocolVersion: networking.ServerTLSSettings_TLSV1_2,
CipherSuites: []string{"ECDHE-RSA-AES128-GCM-SHA256"},
Mode: networking.ServerTLSSettings_SIMPLE,
CredentialName: "kubernetes-ingress://ingress-v1__/wakanda/test-com",
CipherSuites: []string{"ECDHE-RSA-AES128-GCM-SHA256"},
},
},
},

View File

@@ -54,16 +54,10 @@ type Ingress struct {
IPAccessControl *IPAccessControlConfig
HeaderControl *HeaderControlConfig
Timeout *TimeoutConfig
Retry *RetryConfig
LoadBalance *LoadBalanceConfig
localRateLimit *localRateLimitConfig
Fallback *FallbackConfig
Auth *AuthConfig
@@ -105,22 +99,6 @@ func (i *Ingress) NeedTrafficPolicy() bool {
i.LoadBalance != nil
}
func (i *Ingress) MergeHostIPAccessControlIfNotExist(ac *IPAccessControlConfig) {
if i.IPAccessControl != nil && i.IPAccessControl.Domain != nil {
return
}
if ac != nil && ac.Domain != nil {
if i.IPAccessControl == nil {
i.IPAccessControl = &IPAccessControlConfig{
Domain: ac.Domain,
}
} else {
i.IPAccessControl.Domain = ac.Domain
}
}
}
type AnnotationHandler interface {
Parser
GatewayHandler
@@ -147,11 +125,8 @@ func NewAnnotationHandlerManager() AnnotationHandler {
rewrite{},
upstreamTLS{},
ipAccessControl{},
headerControl{},
timeout{},
retry{},
loadBalance{},
localRateLimit{},
fallback{},
auth{},
},
@@ -166,10 +141,7 @@ func NewAnnotationHandlerManager() AnnotationHandler {
redirect{},
rewrite{},
ipAccessControl{},
headerControl{},
timeout{},
retry{},
localRateLimit{},
fallback{},
},
trafficPolicyHandlers: []TrafficPolicyHandler{

View File

@@ -21,8 +21,8 @@ import (
corev1 "k8s.io/api/core/v1"
"github.com/alibaba/higress/ingress/kube/util"
. "github.com/alibaba/higress/ingress/log"
"github.com/alibaba/higress/pkg/ingress/kube/util"
. "github.com/alibaba/higress/pkg/ingress/log"
)
const (

View File

@@ -29,7 +29,7 @@ import (
listerv1 "k8s.io/client-go/listers/core/v1"
"k8s.io/client-go/tools/cache"
"github.com/alibaba/higress/ingress/kube/util"
"github.com/alibaba/higress/pkg/ingress/kube/util"
)
func TestAuthParse(t *testing.T) {
@@ -68,8 +68,8 @@ func TestAuthParse(t *testing.T) {
},
{
input: map[string]string{
buildNginxAnnotationKey(authType): defaultAuthType,
buildMSEAnnotationKey(authSecretAnn): "foo/bar",
buildNginxAnnotationKey(authType): defaultAuthType,
buildHigressAnnotationKey(authSecretAnn): "foo/bar",
},
secret: &v1.Secret{
ObjectMeta: metav1.ObjectMeta{
@@ -96,7 +96,7 @@ func TestAuthParse(t *testing.T) {
{
input: map[string]string{
buildNginxAnnotationKey(authType): defaultAuthType,
buildMSEAnnotationKey(authSecretAnn): "foo/bar",
buildHigressAnnotationKey(authSecretAnn): "foo/bar",
buildNginxAnnotationKey(authSecretTypeAnn): string(authMapAuthSecretType),
},
secret: &v1.Secret{
@@ -125,7 +125,7 @@ func TestAuthParse(t *testing.T) {
{
input: map[string]string{
buildNginxAnnotationKey(authType): defaultAuthType,
buildMSEAnnotationKey(authSecretAnn): "bar",
buildHigressAnnotationKey(authSecretAnn): "bar",
buildNginxAnnotationKey(authSecretTypeAnn): string(authFileAuthSecretType),
},
secret: &v1.Secret{

View File

@@ -106,8 +106,6 @@ func ApplyByWeight(canary, route *networking.HTTPRoute, canaryIngress *Ingress)
// We will process total weight in the end.
route.Route = append(route.Route, canary.Route[0])
// canary route use the header control applied on itself.
headerControl{}.ApplyRoute(canary, canaryIngress)
// Move route level to destination level
canary.Route[0].Headers = canary.Headers
@@ -168,10 +166,6 @@ func ApplyByHeader(canary, route *networking.HTTPRoute, canaryIngress *Ingress)
}
}
canary.Headers = nil
// canary route use the header control applied on itself.
headerControl{}.ApplyRoute(canary, canaryIngress)
// First add normal route cluster
canary.Route[0].FallbackClusters = append(canary.Route[0].FallbackClusters,
route.Route[0].Destination.DeepCopy())

View File

@@ -166,89 +166,3 @@ func TestApplyWeight(t *testing.T) {
t.Fatal("Should be equal")
}
}
func TestApplyHeader(t *testing.T) {
route := &networking.HTTPRoute{
Headers: &networking.Headers{
Request: &networking.Headers_HeaderOperations{
Add: map[string]string{
"normal": "true",
},
},
},
Route: []*networking.HTTPRouteDestination{
{
Destination: &networking.Destination{
Host: "normal",
Port: &networking.PortSelector{
Number: 80,
},
},
},
},
}
canary := &networking.HTTPRoute{
Headers: &networking.Headers{
Request: &networking.Headers_HeaderOperations{
Add: map[string]string{
"canary": "true",
},
},
},
Route: []*networking.HTTPRouteDestination{
{
Destination: &networking.Destination{
Host: "canary",
Port: &networking.PortSelector{
Number: 80,
},
},
},
},
}
ApplyByHeader(canary, route, &Ingress{
Canary: &CanaryConfig{},
HeaderControl: &HeaderControlConfig{
Request: &HeaderOperation{
Add: map[string]string{
"canary": "true",
},
},
},
})
expect := &networking.HTTPRoute{
Headers: &networking.Headers{
Request: &networking.Headers_HeaderOperations{
Add: map[string]string{
"canary": "true",
},
},
Response: &networking.Headers_HeaderOperations{},
},
Route: []*networking.HTTPRouteDestination{
{
Destination: &networking.Destination{
Host: "canary",
Port: &networking.PortSelector{
Number: 80,
},
},
FallbackClusters: []*networking.Destination{
{
Host: "normal",
Port: &networking.PortSelector{
Number: 80,
},
},
},
},
},
}
if !reflect.DeepEqual(canary, expect) {
t.Fatal("Should be equal")
}
}

View File

@@ -131,9 +131,9 @@ func TestCorsParse(t *testing.T) {
},
{
input: Annotations{
buildMSEAnnotationKey(enableCors): "true",
buildHigressAnnotationKey(enableCors): "true",
buildNginxAnnotationKey(allowOrigin): "https://origin-site.com:4443, http://origin-site.com, https://example.org:1199",
buildMSEAnnotationKey(allowMethods): "GET, PUT",
buildHigressAnnotationKey(allowMethods): "GET, PUT",
buildNginxAnnotationKey(allowHeaders): "foo,bar",
buildNginxAnnotationKey(allowCredentials): "false",
buildNginxAnnotationKey(maxAge): "100",

View File

@@ -20,8 +20,8 @@ import (
networking "istio.io/api/networking/v1alpha3"
"istio.io/istio/pilot/pkg/model"
"github.com/alibaba/higress/ingress/kube/util"
. "github.com/alibaba/higress/ingress/log"
"github.com/alibaba/higress/pkg/ingress/kube/util"
. "github.com/alibaba/higress/pkg/ingress/log"
)
const (

View File

@@ -73,7 +73,7 @@ func TestFallbackParse(t *testing.T) {
},
{
input: map[string]string{
buildMSEAnnotationKey(annDefaultBackend): "app",
buildHigressAnnotationKey(annDefaultBackend): "app",
},
expect: &FallbackConfig{
DefaultBackend: model.NamespacedName{
@@ -85,13 +85,13 @@ func TestFallbackParse(t *testing.T) {
},
{
input: map[string]string{
buildMSEAnnotationKey(annDefaultBackend): "foo/app",
buildHigressAnnotationKey(annDefaultBackend): "foo/app",
},
},
{
input: map[string]string{
buildMSEAnnotationKey(annDefaultBackend): "test/app",
buildNginxAnnotationKey(customHTTPError): "404,503",
buildHigressAnnotationKey(annDefaultBackend): "test/app",
buildNginxAnnotationKey(customHTTPError): "404,503",
},
expect: &FallbackConfig{
DefaultBackend: model.NamespacedName{
@@ -104,8 +104,8 @@ func TestFallbackParse(t *testing.T) {
},
{
input: map[string]string{
buildMSEAnnotationKey(annDefaultBackend): "test/app",
buildNginxAnnotationKey(customHTTPError): "404,5ac",
buildHigressAnnotationKey(annDefaultBackend): "test/app",
buildNginxAnnotationKey(customHTTPError): "404,5ac",
},
expect: &FallbackConfig{
DefaultBackend: model.NamespacedName{

View File

@@ -18,58 +18,29 @@ import (
"strings"
networking "istio.io/api/networking/v1alpha3"
"github.com/alibaba/higress/ingress/kube/util"
. "github.com/alibaba/higress/ingress/log"
"istio.io/istio/pilot/pkg/credentials/kube"
"istio.io/istio/pilot/pkg/model"
gatewaytool "istio.io/istio/pkg/config/gateway"
"istio.io/istio/pkg/config/security"
"github.com/alibaba/higress/pkg/ingress/kube/util"
. "github.com/alibaba/higress/pkg/ingress/log"
)
const (
authTLSSecret = "auth-tls-secret"
tlsMinVersion = "tls-min-protocol-version"
tlsMaxVersion = "tls-max-protocol-version"
sslCipher = "ssl-cipher"
)
type TLSProtocolVersion string
const (
tlsV10 TLSProtocolVersion = "TLSv1.0"
tlsV11 TLSProtocolVersion = "TLSv1.1"
tlsV12 TLSProtocolVersion = "TLSv1.2"
tlsV13 TLSProtocolVersion = "TLSv1.3"
)
var (
_ Parser = &downstreamTLS{}
_ GatewayHandler = &downstreamTLS{}
tlsProtocol = map[TLSProtocolVersion]networking.ServerTLSSettings_TLSProtocol{
tlsV10: networking.ServerTLSSettings_TLSV1_0,
tlsV11: networking.ServerTLSSettings_TLSV1_1,
tlsV12: networking.ServerTLSSettings_TLSV1_2,
tlsV13: networking.ServerTLSSettings_TLSV1_3,
}
)
func isValidTLSProtocolVersion(protocol string) bool {
tls := TLSProtocolVersion(protocol)
_, exist := tlsProtocol[tls]
return exist
}
func Convert(protocol string) networking.ServerTLSSettings_TLSProtocol {
return tlsProtocol[TLSProtocolVersion(protocol)]
}
type DownstreamTLSConfig struct {
TlsMinVersion TLSProtocolVersion
TlsMaxVersion TLSProtocolVersion
CipherSuites []string
Mode networking.ServerTLSSettings_TLSmode
CASecretName model.NamespacedName
CipherSuites []string
Mode networking.ServerTLSSettings_TLSmode
CASecretName model.NamespacedName
}
type downstreamTLS struct{}
@@ -99,16 +70,6 @@ func (d downstreamTLS) Parse(annotations Annotations, config *Ingress, _ *Global
}
}
if minVersion, err := annotations.ParseStringForMSE(tlsMinVersion); err == nil &&
isValidTLSProtocolVersion(minVersion) {
downstreamTLSConfig.TlsMinVersion = TLSProtocolVersion(minVersion)
}
if maxVersion, err := annotations.ParseStringForMSE(tlsMaxVersion); err == nil &&
isValidTLSProtocolVersion(maxVersion) {
downstreamTLSConfig.TlsMaxVersion = TLSProtocolVersion(maxVersion)
}
if rawTlsCipherSuite, err := annotations.ParseStringASAP(sslCipher); err == nil {
var validCipherSuite []string
cipherList := strings.Split(rawTlsCipherSuite, ":")
@@ -143,12 +104,6 @@ func (d downstreamTLS) ApplyGateway(gateway *networking.Gateway, config *Ingress
}
}
if downstreamTLSConfig.TlsMinVersion != "" {
server.Tls.MinProtocolVersion = tlsProtocol[downstreamTLSConfig.TlsMinVersion]
}
if downstreamTLSConfig.TlsMaxVersion != "" {
server.Tls.MaxProtocolVersion = tlsProtocol[downstreamTLSConfig.TlsMaxVersion]
}
if len(downstreamTLSConfig.CipherSuites) != 0 {
server.Tls.CipherSuites = downstreamTLSConfig.CipherSuites
}
@@ -157,8 +112,6 @@ func (d downstreamTLS) ApplyGateway(gateway *networking.Gateway, config *Ingress
}
func needDownstreamTLS(annotations Annotations) bool {
return annotations.HasMSE(tlsMinVersion) ||
annotations.HasMSE(tlsMaxVersion) ||
annotations.HasASAP(sslCipher) ||
return annotations.HasASAP(sslCipher) ||
annotations.HasASAP(authTLSSecret)
}

View File

@@ -32,28 +32,7 @@ func TestParse(t *testing.T) {
{},
{
input: map[string]string{
MSEAnnotationsPrefix + "/" + tlsMinVersion: "TLSv1.0",
},
expect: &DownstreamTLSConfig{
Mode: networking.ServerTLSSettings_SIMPLE,
TlsMinVersion: tlsV10,
},
},
{
input: map[string]string{
MSEAnnotationsPrefix + "/" + tlsMinVersion: "TLSv1.3",
DefaultAnnotationsPrefix + "/" + sslCipher: "ECDHE-RSA-AES256-GCM-SHA384:AES128-SHA",
},
expect: &DownstreamTLSConfig{
Mode: networking.ServerTLSSettings_SIMPLE,
TlsMinVersion: tlsV13,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384", "AES128-SHA"},
},
},
{
input: map[string]string{
MSEAnnotationsPrefix + "/" + tlsMinVersion: "xxx",
DefaultAnnotationsPrefix + "/" + sslCipher: "ECDHE-RSA-AES256-GCM-SHA384:AES128-SHA",
buildNginxAnnotationKey(sslCipher): "ECDHE-RSA-AES256-GCM-SHA384:AES128-SHA",
},
expect: &DownstreamTLSConfig{
Mode: networking.ServerTLSSettings_SIMPLE,
@@ -62,19 +41,8 @@ func TestParse(t *testing.T) {
},
{
input: map[string]string{
MSEAnnotationsPrefix + "/" + tlsMinVersion: "xxx",
MSEAnnotationsPrefix + "/" + sslCipher: "ECDHE-RSA-AES256-GCM-SHA384:AES128-SHA",
},
expect: &DownstreamTLSConfig{
Mode: networking.ServerTLSSettings_SIMPLE,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384", "AES128-SHA"},
},
},
{
input: map[string]string{
buildNginxAnnotationKey(authTLSSecret): "test",
MSEAnnotationsPrefix + "/" + tlsMinVersion: "xxx",
MSEAnnotationsPrefix + "/" + sslCipher: "ECDHE-RSA-AES256-GCM-SHA384:AES128-SHA",
buildNginxAnnotationKey(authTLSSecret): "test",
buildNginxAnnotationKey(sslCipher): "ECDHE-RSA-AES256-GCM-SHA384:AES128-SHA",
},
expect: &DownstreamTLSConfig{
CASecretName: model.NamespacedName{
@@ -87,8 +55,7 @@ func TestParse(t *testing.T) {
},
{
input: map[string]string{
buildMSEAnnotationKey(authTLSSecret): "test/foo",
MSEAnnotationsPrefix + "/" + tlsMinVersion: "TLSv1.3",
buildHigressAnnotationKey(authTLSSecret): "test/foo",
DefaultAnnotationsPrefix + "/" + sslCipher: "ECDHE-RSA-AES256-GCM-SHA384:AES128-SHA",
},
expect: &DownstreamTLSConfig{
@@ -96,9 +63,8 @@ func TestParse(t *testing.T) {
Namespace: "test",
Name: "foo",
},
Mode: networking.ServerTLSSettings_MUTUAL,
TlsMinVersion: tlsV13,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384", "AES128-SHA"},
Mode: networking.ServerTLSSettings_MUTUAL,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384", "AES128-SHA"},
},
},
}
@@ -124,31 +90,6 @@ func TestApplyGateway(t *testing.T) {
config *Ingress
expect *networking.Gateway
}{
{
input: &networking.Gateway{
Servers: []*networking.Server{
{
Port: &networking.Port{
Protocol: "HTTP",
},
},
},
},
config: &Ingress{
DownstreamTLS: &DownstreamTLSConfig{
TlsMinVersion: tlsV10,
},
},
expect: &networking.Gateway{
Servers: []*networking.Server{
{
Port: &networking.Port{
Protocol: "HTTP",
},
},
},
},
},
{
input: &networking.Gateway{
Servers: []*networking.Server{
@@ -164,7 +105,7 @@ func TestApplyGateway(t *testing.T) {
},
config: &Ingress{
DownstreamTLS: &DownstreamTLSConfig{
TlsMinVersion: tlsV12,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384"},
},
},
expect: &networking.Gateway{
@@ -174,42 +115,8 @@ func TestApplyGateway(t *testing.T) {
Protocol: "HTTPS",
},
Tls: &networking.ServerTLSSettings{
Mode: networking.ServerTLSSettings_SIMPLE,
MinProtocolVersion: networking.ServerTLSSettings_TLSV1_2,
},
},
},
},
},
{
input: &networking.Gateway{
Servers: []*networking.Server{
{
Port: &networking.Port{
Protocol: "HTTPS",
},
Tls: &networking.ServerTLSSettings{
Mode: networking.ServerTLSSettings_SIMPLE,
},
},
},
},
config: &Ingress{
DownstreamTLS: &DownstreamTLSConfig{
TlsMaxVersion: tlsV13,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384"},
},
},
expect: &networking.Gateway{
Servers: []*networking.Server{
{
Port: &networking.Port{
Protocol: "HTTPS",
},
Tls: &networking.ServerTLSSettings{
Mode: networking.ServerTLSSettings_SIMPLE,
MaxProtocolVersion: networking.ServerTLSSettings_TLSV1_3,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384"},
Mode: networking.ServerTLSSettings_SIMPLE,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384"},
},
},
},
@@ -235,9 +142,8 @@ func TestApplyGateway(t *testing.T) {
Namespace: "foo",
Name: "bar",
},
Mode: networking.ServerTLSSettings_MUTUAL,
TlsMaxVersion: tlsV13,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384"},
Mode: networking.ServerTLSSettings_MUTUAL,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384"},
},
},
expect: &networking.Gateway{
@@ -247,10 +153,9 @@ func TestApplyGateway(t *testing.T) {
Protocol: "HTTPS",
},
Tls: &networking.ServerTLSSettings{
CredentialName: "kubernetes-ingress://cluster/foo/bar",
Mode: networking.ServerTLSSettings_MUTUAL,
MaxProtocolVersion: networking.ServerTLSSettings_TLSV1_3,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384"},
CredentialName: "kubernetes-ingress://cluster/foo/bar",
Mode: networking.ServerTLSSettings_MUTUAL,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384"},
},
},
},
@@ -276,9 +181,8 @@ func TestApplyGateway(t *testing.T) {
Namespace: "foo",
Name: "bar-cacert",
},
Mode: networking.ServerTLSSettings_MUTUAL,
TlsMaxVersion: tlsV13,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384"},
Mode: networking.ServerTLSSettings_MUTUAL,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384"},
},
},
expect: &networking.Gateway{
@@ -288,10 +192,9 @@ func TestApplyGateway(t *testing.T) {
Protocol: "HTTPS",
},
Tls: &networking.ServerTLSSettings{
CredentialName: "kubernetes-ingress://cluster/foo/bar",
Mode: networking.ServerTLSSettings_MUTUAL,
MaxProtocolVersion: networking.ServerTLSSettings_TLSV1_3,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384"},
CredentialName: "kubernetes-ingress://cluster/foo/bar",
Mode: networking.ServerTLSSettings_MUTUAL,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384"},
},
},
},
@@ -317,9 +220,8 @@ func TestApplyGateway(t *testing.T) {
Namespace: "bar",
Name: "foo",
},
Mode: networking.ServerTLSSettings_MUTUAL,
TlsMaxVersion: tlsV13,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384"},
Mode: networking.ServerTLSSettings_MUTUAL,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384"},
},
},
expect: &networking.Gateway{
@@ -329,10 +231,9 @@ func TestApplyGateway(t *testing.T) {
Protocol: "HTTPS",
},
Tls: &networking.ServerTLSSettings{
CredentialName: "kubernetes-ingress://cluster/foo/bar",
Mode: networking.ServerTLSSettings_SIMPLE,
MaxProtocolVersion: networking.ServerTLSSettings_TLSV1_3,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384"},
CredentialName: "kubernetes-ingress://cluster/foo/bar",
Mode: networking.ServerTLSSettings_SIMPLE,
CipherSuites: []string{"ECDHE-RSA-AES256-GCM-SHA384"},
},
},
},

View File

@@ -20,10 +20,7 @@ import (
)
const (
domainWhitelist = "domain-whitelist-source-range"
domainBlacklist = "domain-blacklist-source-range"
whitelist = "whitelist-source-range"
blacklist = "blacklist-source-range"
whitelist = "whitelist-source-range"
)
var (
@@ -37,8 +34,7 @@ type IPAccessControl struct {
}
type IPAccessControlConfig struct {
Domain *IPAccessControl
Route *IPAccessControl
Route *IPAccessControl
}
type ipAccessControl struct{}
@@ -53,40 +49,14 @@ func (i ipAccessControl) Parse(annotations Annotations, config *Ingress, _ *Glob
config.IPAccessControl = ipConfig
}()
var domain *IPAccessControl
rawWhitelist, err := annotations.ParseStringForMSE(domainWhitelist)
if err == nil {
domain = &IPAccessControl{
isWhite: true,
remoteIp: splitStringWithSpaceTrim(rawWhitelist),
}
} else {
if rawBlacklist, err := annotations.ParseStringForMSE(domainBlacklist); err == nil {
domain = &IPAccessControl{
isWhite: false,
remoteIp: splitStringWithSpaceTrim(rawBlacklist),
}
}
}
if domain != nil {
ipConfig.Domain = domain
}
var route *IPAccessControl
rawWhitelist, err = annotations.ParseStringASAP(whitelist)
if err == nil {
if rawWhitelist, err := annotations.ParseStringASAP(whitelist); err == nil {
route = &IPAccessControl{
isWhite: true,
remoteIp: splitStringWithSpaceTrim(rawWhitelist),
}
} else {
if rawBlacklist, err := annotations.ParseStringForMSE(blacklist); err == nil {
route = &IPAccessControl{
isWhite: false,
remoteIp: splitStringWithSpaceTrim(rawBlacklist),
}
}
}
if route != nil {
ipConfig.Route = route
}
@@ -94,25 +64,8 @@ func (i ipAccessControl) Parse(annotations Annotations, config *Ingress, _ *Glob
return nil
}
func (i ipAccessControl) ApplyVirtualServiceHandler(virtualService *networking.VirtualService, config *Ingress) {
ac := config.IPAccessControl
if ac == nil || ac.Domain == nil {
return
}
filter := &networking.IPAccessControl{}
if ac.Domain.isWhite {
filter.RemoteIpBlocks = ac.Domain.remoteIp
} else {
filter.NotRemoteIpBlocks = ac.Domain.remoteIp
}
virtualService.HostHTTPFilters = append(virtualService.HostHTTPFilters, &networking.HTTPFilter{
Name: mseingress.IPAccessControl,
Filter: &networking.HTTPFilter_IpAccessControl{
IpAccessControl: filter,
},
})
func (i ipAccessControl) ApplyVirtualServiceHandler(_ *networking.VirtualService, _ *Ingress) {
// DO NOTHING
}
func (i ipAccessControl) ApplyRoute(route *networking.HTTPRoute, config *Ingress) {
@@ -137,8 +90,5 @@ func (i ipAccessControl) ApplyRoute(route *networking.HTTPRoute, config *Ingress
}
func needIPAccessControlConfig(annotations Annotations) bool {
return annotations.HasMSE(domainWhitelist) ||
annotations.HasMSE(domainBlacklist) ||
annotations.HasASAP(whitelist) ||
annotations.HasMSE(blacklist)
return annotations.HasASAP(whitelist)
}

View File

@@ -0,0 +1,108 @@
// Copyright (c) 2022 Alibaba Group Holding Ltd.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package annotations
import (
"reflect"
"testing"
networking "istio.io/api/networking/v1alpha3"
)
func TestIPAccessControlParse(t *testing.T) {
parser := ipAccessControl{}
testCases := []struct {
input map[string]string
expect *IPAccessControlConfig
}{
{},
{
input: map[string]string{
buildNginxAnnotationKey(whitelist): "1.1.1.1",
},
expect: &IPAccessControlConfig{
Route: &IPAccessControl{
isWhite: true,
remoteIp: []string{"1.1.1.1"},
},
},
},
}
for _, testCase := range testCases {
t.Run("", func(t *testing.T) {
config := &Ingress{}
_ = parser.Parse(testCase.input, config, nil)
if !reflect.DeepEqual(testCase.expect, config.IPAccessControl) {
t.Fatalf("Should be equal")
}
})
}
}
func TestIpAccessControl_ApplyRoute(t *testing.T) {
parser := ipAccessControl{}
testCases := []struct {
config *Ingress
input *networking.HTTPRoute
expect *networking.HTTPFilter
}{
{
config: &Ingress{},
input: &networking.HTTPRoute{},
expect: nil,
},
{
config: &Ingress{
IPAccessControl: &IPAccessControlConfig{
Route: &IPAccessControl{
isWhite: true,
remoteIp: []string{"1.1.1.1"},
},
},
},
input: &networking.HTTPRoute{},
expect: &networking.HTTPFilter{
Name: "ip-access-control",
Disable: false,
Filter: &networking.HTTPFilter_IpAccessControl{
IpAccessControl: &networking.IPAccessControl{
RemoteIpBlocks: []string{"1.1.1.1"},
},
},
},
},
}
for _, testCase := range testCases {
t.Run("", func(t *testing.T) {
parser.ApplyRoute(testCase.input, testCase.config)
if testCase.config.IPAccessControl == nil {
if len(testCase.input.RouteHTTPFilters) != 0 {
t.Fatalf("Should be empty")
}
} else {
if len(testCase.input.RouteHTTPFilters) == 0 {
t.Fatalf("Should be not empty")
}
if !reflect.DeepEqual(testCase.expect, testCase.input.RouteHTTPFilters[0]) {
t.Fatalf("Should be equal")
}
}
})
}
}

View File

@@ -34,7 +34,6 @@ const (
sessionCookiePath = "session-cookie-path"
sessionCookieMaxAge = "session-cookie-max-age"
sessionCookieExpires = "session-cookie-expires"
warmup = "warmup"
varIndicator = "$"
headerIndicator = "$http_"
@@ -68,7 +67,6 @@ type consistentHashByCookie struct {
type LoadBalanceConfig struct {
simple networking.LoadBalancerSettings_SimpleLB
warmup *types.Duration
other *consistentHashByOther
cookie *consistentHashByCookie
}
@@ -133,12 +131,6 @@ func (l loadBalance) Parse(annotations Annotations, config *Ingress, _ *GlobalCo
lb = strings.ToUpper(lb)
loadBalanceConfig.simple = networking.LoadBalancerSettings_SimpleLB(networking.LoadBalancerSettings_SimpleLB_value[lb])
}
if warmup, err := annotations.ParseIntForMSE(warmup); err == nil && warmup != 0 {
loadBalanceConfig.warmup = &types.Duration{
Seconds: int64(warmup),
}
}
}
return nil
@@ -190,7 +182,6 @@ func (l loadBalance) ApplyTrafficPolicy(trafficPolicy *networking.TrafficPolicy_
Simple: loadBalanceConfig.simple,
},
}
trafficPolicy.LoadBalancer.WarmupDurationSecs = loadBalanceConfig.warmup
}
}
@@ -206,7 +197,6 @@ func isOtherAffinity(annotations Annotations) bool {
func needLoadBalanceConfig(annotations Annotations) bool {
return annotations.HasASAP(loadBalanceAnnotation) ||
annotations.HasMSE(warmup) ||
isCookieAffinity(annotations) ||
isOtherAffinity(annotations)
}

View File

@@ -127,41 +127,6 @@ func TestLoadBalanceParse(t *testing.T) {
},
},
},
{
input: map[string]string{
buildMSEAnnotationKey(warmup): "100",
},
expect: &LoadBalanceConfig{
simple: networking.LoadBalancerSettings_ROUND_ROBIN,
warmup: &types.Duration{
Seconds: 100,
},
},
},
{
input: map[string]string{
buildNginxAnnotationKey(loadBalanceAnnotation): "LEAST_CONN",
buildMSEAnnotationKey(warmup): "100",
},
expect: &LoadBalanceConfig{
simple: networking.LoadBalancerSettings_LEAST_CONN,
warmup: &types.Duration{
Seconds: 100,
},
},
},
{
input: map[string]string{
buildNginxAnnotationKey(loadBalanceAnnotation): "random",
buildMSEAnnotationKey(warmup): "100",
},
expect: &LoadBalanceConfig{
simple: networking.LoadBalancerSettings_RANDOM,
warmup: &types.Duration{
Seconds: 100,
},
},
},
}
for _, inputCase := range inputCases {
@@ -260,27 +225,6 @@ func TestLoadBalanceApplyTrafficPolicy(t *testing.T) {
},
},
},
{
config: &Ingress{
LoadBalance: &LoadBalanceConfig{
simple: networking.LoadBalancerSettings_ROUND_ROBIN,
warmup: &types.Duration{
Seconds: 100,
},
},
},
input: &networking.TrafficPolicy_PortTrafficPolicy{},
expect: &networking.TrafficPolicy_PortTrafficPolicy{
LoadBalancer: &networking.LoadBalancerSettings{
LbPolicy: &networking.LoadBalancerSettings_Simple{
Simple: networking.LoadBalancerSettings_ROUND_ROBIN,
},
WarmupDurationSecs: &types.Duration{
Seconds: 100,
},
},
},
},
}
for _, inputCase := range inputCases {

View File

@@ -24,8 +24,8 @@ const (
// DefaultAnnotationsPrefix defines the common prefix used in the nginx ingress controller
DefaultAnnotationsPrefix = "nginx.ingress.kubernetes.io"
// MSEAnnotationsPrefix defines the common prefix used in the mse ingress controller
MSEAnnotationsPrefix = "mse.ingress.kubernetes.io"
// HigressAnnotationsPrefix defines the common prefix used in the higress ingress controller
HigressAnnotationsPrefix = "higress.io"
)
var (
@@ -67,12 +67,12 @@ func (a Annotations) ParseBool(key string) (bool, error) {
return false, ErrMissingAnnotations
}
func (a Annotations) ParseBoolForMSE(key string) (bool, error) {
func (a Annotations) ParseBoolForHigress(key string) (bool, error) {
if len(a) == 0 {
return false, ErrMissingAnnotations
}
val, ok := a[buildMSEAnnotationKey(key)]
val, ok := a[buildHigressAnnotationKey(key)]
if ok {
b, err := strconv.ParseBool(val)
if err != nil {
@@ -88,7 +88,7 @@ func (a Annotations) ParseBoolASAP(key string) (bool, error) {
if result, err := a.ParseBool(key); err == nil {
return result, nil
}
return a.ParseBoolForMSE(key)
return a.ParseBoolForHigress(key)
}
func (a Annotations) ParseString(key string) (string, error) {
@@ -108,12 +108,12 @@ func (a Annotations) ParseString(key string) (string, error) {
return "", ErrMissingAnnotations
}
func (a Annotations) ParseStringForMSE(key string) (string, error) {
func (a Annotations) ParseStringForHigress(key string) (string, error) {
if len(a) == 0 {
return "", ErrMissingAnnotations
}
val, ok := a[buildMSEAnnotationKey(key)]
val, ok := a[buildHigressAnnotationKey(key)]
if ok {
s := normalizeString(val)
if s == "" {
@@ -126,12 +126,12 @@ func (a Annotations) ParseStringForMSE(key string) (string, error) {
}
// ParseStringASAP will first extra config from nginx annotation, then will
// try to extra config from mse annotation if the first step fails.
// try to extra config from Higress annotation if the first step fails.
func (a Annotations) ParseStringASAP(key string) (string, error) {
if result, err := a.ParseString(key); err == nil {
return result, nil
}
return a.ParseStringForMSE(key)
return a.ParseStringForHigress(key)
}
func (a Annotations) ParseInt(key string) (int, error) {
@@ -150,12 +150,12 @@ func (a Annotations) ParseInt(key string) (int, error) {
return 0, ErrMissingAnnotations
}
func (a Annotations) ParseIntForMSE(key string) (int, error) {
func (a Annotations) ParseIntForHigress(key string) (int, error) {
if len(a) == 0 {
return 0, ErrMissingAnnotations
}
val, ok := a[buildMSEAnnotationKey(key)]
val, ok := a[buildHigressAnnotationKey(key)]
if ok {
i, err := strconv.Atoi(val)
if err != nil {
@@ -182,12 +182,12 @@ func (a Annotations) ParseInt32(key string) (int32, error) {
return 0, ErrMissingAnnotations
}
func (a Annotations) ParseInt32ForMSE(key string) (int32, error) {
func (a Annotations) ParseInt32ForHigress(key string) (int32, error) {
if len(a) == 0 {
return 0, ErrMissingAnnotations
}
val, ok := a[buildMSEAnnotationKey(key)]
val, ok := a[buildHigressAnnotationKey(key)]
if ok {
i, err := strconv.ParseInt(val, 10, 32)
if err != nil {
@@ -198,12 +198,12 @@ func (a Annotations) ParseInt32ForMSE(key string) (int32, error) {
return 0, ErrMissingAnnotations
}
func (a Annotations) ParseUint32ForMSE(key string) (uint32, error) {
func (a Annotations) ParseUint32ForHigress(key string) (uint32, error) {
if len(a) == 0 {
return 0, ErrMissingAnnotations
}
val, ok := a[buildMSEAnnotationKey(key)]
val, ok := a[buildHigressAnnotationKey(key)]
if ok {
i, err := strconv.ParseUint(val, 10, 32)
if err != nil {
@@ -218,14 +218,14 @@ func (a Annotations) ParseIntASAP(key string) (int, error) {
if result, err := a.ParseInt(key); err == nil {
return result, nil
}
return a.ParseIntForMSE(key)
return a.ParseIntForHigress(key)
}
func (a Annotations) ParseInt32ASAP(key string) (int32, error) {
if result, err := a.ParseInt32(key); err == nil {
return result, nil
}
return a.ParseInt32ForMSE(key)
return a.ParseInt32ForHigress(key)
}
func (a Annotations) Has(key string) bool {
@@ -237,12 +237,12 @@ func (a Annotations) Has(key string) bool {
return exist
}
func (a Annotations) HasMSE(key string) bool {
func (a Annotations) HasHigress(key string) bool {
if len(a) == 0 {
return false
}
_, exist := a[buildMSEAnnotationKey(key)]
_, exist := a[buildHigressAnnotationKey(key)]
return exist
}
@@ -250,15 +250,15 @@ func (a Annotations) HasASAP(key string) bool {
if a.Has(key) {
return true
}
return a.HasMSE(key)
return a.HasHigress(key)
}
func buildNginxAnnotationKey(key string) string {
return DefaultAnnotationsPrefix + "/" + key
}
func buildMSEAnnotationKey(key string) string {
return MSEAnnotationsPrefix + "/" + key
func buildHigressAnnotationKey(key string) string {
return HigressAnnotationsPrefix + "/" + key
}
func normalizeString(input string) string {

View File

@@ -22,7 +22,7 @@ import (
networking "istio.io/api/networking/v1alpha3"
"istio.io/istio/pilot/pkg/model/credentials"
"github.com/alibaba/higress/ingress/kube/util"
"github.com/alibaba/higress/pkg/ingress/kube/util"
)
const (

View File

@@ -24,7 +24,7 @@ import (
listerv1 "k8s.io/client-go/listers/core/v1"
"k8s.io/client-go/tools/cache"
"github.com/alibaba/higress/ingress/kube/annotations"
"github.com/alibaba/higress/pkg/ingress/kube/annotations"
)
type ServiceKey struct {
@@ -73,7 +73,7 @@ type WrapperHTTPRoute struct {
}
func (w *WrapperHTTPRoute) Meta() string {
return strings.Join([]string{w.WrapperConfig.Config.Namespace, w.WrapperConfig.Config.Name}, "-")
return strings.Join([]string{w.WrapperConfig.Config.Namespace, w.WrapperConfig.Config.Name}, "/")
}
func (w *WrapperHTTPRoute) BasePathFormat() string {

View File

@@ -28,7 +28,7 @@ import (
"istio.io/istio/pkg/config/schema/collections"
"k8s.io/apimachinery/pkg/labels"
. "github.com/alibaba/higress/ingress/log"
. "github.com/alibaba/higress/pkg/ingress/log"
)
type PathType string

View File

@@ -28,7 +28,7 @@ import (
v1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/util/version"
. "github.com/alibaba/higress/ingress/log"
. "github.com/alibaba/higress/pkg/ingress/log"
)
// V1Available check if the "networking/v1" Ingress is available.
@@ -267,31 +267,11 @@ func partMd5(raw string) string {
}
func GenerateUniqueRouteName(route *WrapperHTTPRoute) string {
raw := constructRouteName(route)
// meta-part-clusterId
// meta: ingressNamespace-ingressName
meta := route.Meta()
// host-pathType-path-header-queryParam, md5, then before 4 char and end 4 char
part := partMd5(raw)
routeName := CreateConvertedName(meta, part, route.ClusterId)
if route.WrapperConfig.AnnotationsConfig.IsCanary() {
return routeName + "-canary"
}
return routeName
return route.Meta()
}
func GenerateUniqueRouteNameWithSuffix(route *WrapperHTTPRoute, suffix string) string {
raw := constructRouteName(route)
// meta-part-clusterId
// meta: ingressNamespace-ingressName
meta := route.Meta()
// host-pathType-path-header-queryParam, md5, then before 4 char and end 4 char
part := partMd5(raw)
return CreateConvertedName(meta, part, route.ClusterId, suffix)
return CreateConvertedName(route.Meta(), suffix)
}
func SplitServiceFQDN(fqdn string) (string, string, bool) {

View File

@@ -22,7 +22,8 @@ import (
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/alibaba/higress/ingress/kube/annotations"
"github.com/alibaba/higress/pkg/ingress/kube/annotations"
"github.com/stretchr/testify/assert"
)
func TestConstructRouteName(t *testing.T) {
@@ -151,7 +152,7 @@ func TestConstructRouteName(t *testing.T) {
}
func TestGenerateUniqueRouteName(t *testing.T) {
inputWithoutCanary := &WrapperHTTPRoute{
input := &WrapperHTTPRoute{
WrapperConfig: &WrapperConfig{
Config: &config.Config{
Meta: config.Meta{
@@ -197,65 +198,8 @@ func TestGenerateUniqueRouteName(t *testing.T) {
},
}
withoutCanary := GenerateUniqueRouteName(inputWithoutCanary)
t.Log(withoutCanary)
assert.Equal(t, "bar/foo", GenerateUniqueRouteName(input))
inputWithCanary := &WrapperHTTPRoute{
WrapperConfig: &WrapperConfig{
Config: &config.Config{
Meta: config.Meta{
Name: "foo",
Namespace: "bar",
},
},
AnnotationsConfig: &annotations.Ingress{
Canary: &annotations.CanaryConfig{
Enabled: true,
},
},
},
Host: "test.com",
OriginPathType: Prefix,
OriginPath: "/test",
ClusterId: "cluster1",
HTTPRoute: &networking.HTTPRoute{
Match: []*networking.HTTPMatchRequest{
{
Headers: map[string]*networking.StringMatch{
"f": {
MatchType: &networking.StringMatch_Regex{
Regex: "abc?",
},
},
"e": {
MatchType: &networking.StringMatch_Exact{
Exact: "bye",
},
},
},
QueryParams: map[string]*networking.StringMatch{
"b": {
MatchType: &networking.StringMatch_Regex{
Regex: "a?c.*",
},
},
"a": {
MatchType: &networking.StringMatch_Exact{
Exact: "hello",
},
},
},
},
},
},
}
withCanary := GenerateUniqueRouteName(inputWithCanary)
t.Log(withCanary)
if withCanary != withoutCanary+"-canary" {
t.Fatalf("Expect %s, but actual is %s", withCanary, withoutCanary+"-canary")
}
}
func TestGetLbStatusList(t *testing.T) {

View File

@@ -48,11 +48,11 @@ import (
"k8s.io/client-go/tools/cache"
"k8s.io/client-go/util/workqueue"
"github.com/alibaba/higress/ingress/kube/annotations"
"github.com/alibaba/higress/ingress/kube/common"
"github.com/alibaba/higress/ingress/kube/secret"
"github.com/alibaba/higress/ingress/kube/util"
. "github.com/alibaba/higress/ingress/log"
"github.com/alibaba/higress/pkg/ingress/kube/annotations"
"github.com/alibaba/higress/pkg/ingress/kube/common"
"github.com/alibaba/higress/pkg/ingress/kube/secret"
"github.com/alibaba/higress/pkg/ingress/kube/util"
. "github.com/alibaba/higress/pkg/ingress/log"
)
var (
@@ -497,8 +497,6 @@ func (c *controller) ConvertHTTPRoute(convertOptions *common.ConvertOptions, wra
WrapperConfig: wrapper,
}
convertOptions.VirtualServices[rule.Host] = wrapperVS
} else {
wrapperVS.WrapperConfig.AnnotationsConfig.MergeHostIPAccessControlIfNotExist(wrapper.AnnotationsConfig.IPAccessControl)
}
// Record the latest app root for per host.

View File

@@ -20,7 +20,7 @@ import (
"k8s.io/api/networking/v1beta1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/alibaba/higress/ingress/kube/common"
"github.com/alibaba/higress/pkg/ingress/kube/common"
)
func TestShouldProcessIngressUpdate(t *testing.T) {

View File

@@ -16,6 +16,8 @@ package ingress
import (
"context"
common2 "github.com/alibaba/higress/pkg/ingress/kube/common"
. "github.com/alibaba/higress/pkg/ingress/log"
"reflect"
"sort"
"time"
@@ -28,9 +30,6 @@ import (
listerv1 "k8s.io/client-go/listers/core/v1"
ingresslister "k8s.io/client-go/listers/networking/v1beta1"
"k8s.io/client-go/tools/cache"
"github.com/alibaba/higress/ingress/kube/common"
. "github.com/alibaba/higress/ingress/log"
)
// statusSyncer keeps the status IP in each Ingress resource updated
@@ -62,7 +61,7 @@ func newStatusSyncer(localKubeClient, client kubelib.Client, controller *control
func (s *statusSyncer) run(stopCh <-chan struct{}) {
cache.WaitForCacheSync(stopCh, s.controller.HasSynced)
ticker := time.NewTicker(common.DefaultStatusUpdateInterval)
ticker := time.NewTicker(common2.DefaultStatusUpdateInterval)
for {
select {
case <-stopCh:
@@ -77,14 +76,14 @@ func (s *statusSyncer) run(stopCh <-chan struct{}) {
}
func (s *statusSyncer) runUpdateStatus() error {
svcList, err := s.serviceLister.Services(s.watchedNamespace).List(common.SvcLabelSelector)
svcList, err := s.serviceLister.Services(s.watchedNamespace).List(common2.SvcLabelSelector)
if err != nil {
return err
}
IngressLog.Debugf("found number %d of svc", len(svcList))
lbStatusList := common.GetLbStatusList(svcList)
lbStatusList := common2.GetLbStatusList(svcList)
if len(lbStatusList) == 0 {
return nil
}
@@ -111,7 +110,7 @@ func (s *statusSyncer) updateStatus(status []coreV1.LoadBalancerIngress) error {
}
curIPs := ingress.Status.LoadBalancer.Ingress
sort.SliceStable(curIPs, common.SortLbIngressList(curIPs))
sort.SliceStable(curIPs, common2.SortLbIngressList(curIPs))
if reflect.DeepEqual(status, curIPs) {
IngressLog.Debugf("skipping update of Ingress %v/%v within cluster %s (no change)",

View File

@@ -47,11 +47,11 @@ import (
"k8s.io/client-go/tools/cache"
"k8s.io/client-go/util/workqueue"
"github.com/alibaba/higress/ingress/kube/annotations"
"github.com/alibaba/higress/ingress/kube/common"
"github.com/alibaba/higress/ingress/kube/secret"
"github.com/alibaba/higress/ingress/kube/util"
. "github.com/alibaba/higress/ingress/log"
"github.com/alibaba/higress/pkg/ingress/kube/annotations"
"github.com/alibaba/higress/pkg/ingress/kube/common"
"github.com/alibaba/higress/pkg/ingress/kube/secret"
"github.com/alibaba/higress/pkg/ingress/kube/util"
. "github.com/alibaba/higress/pkg/ingress/log"
)
var (
@@ -492,8 +492,6 @@ func (c *controller) ConvertHTTPRoute(convertOptions *common.ConvertOptions, wra
WrapperConfig: wrapper,
}
convertOptions.VirtualServices[rule.Host] = wrapperVS
} else {
wrapperVS.WrapperConfig.AnnotationsConfig.MergeHostIPAccessControlIfNotExist(wrapper.AnnotationsConfig.IPAccessControl)
}
// Record the latest app root for per host.

View File

@@ -20,7 +20,7 @@ import (
v1 "k8s.io/api/networking/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/alibaba/higress/ingress/kube/common"
"github.com/alibaba/higress/pkg/ingress/kube/common"
)
func TestShouldProcessIngressUpdate(t *testing.T) {

View File

@@ -29,8 +29,8 @@ import (
ingresslister "k8s.io/client-go/listers/networking/v1"
"k8s.io/client-go/tools/cache"
"github.com/alibaba/higress/ingress/kube/common"
. "github.com/alibaba/higress/ingress/log"
"github.com/alibaba/higress/pkg/ingress/kube/common"
. "github.com/alibaba/higress/pkg/ingress/log"
)
// statusSyncer keeps the status IP in each Ingress resource updated

View File

@@ -33,10 +33,10 @@ import (
"k8s.io/client-go/tools/cache"
"k8s.io/client-go/util/workqueue"
"github.com/alibaba/higress/ingress/kube/common"
"github.com/alibaba/higress/ingress/kube/secret"
"github.com/alibaba/higress/ingress/kube/util"
. "github.com/alibaba/higress/ingress/log"
"github.com/alibaba/higress/pkg/ingress/kube/common"
"github.com/alibaba/higress/pkg/ingress/kube/secret"
"github.com/alibaba/higress/pkg/ingress/kube/util"
. "github.com/alibaba/higress/pkg/ingress/log"
)
var _ secret.Controller = &controller{}

View File

@@ -18,7 +18,7 @@ import (
listerv1 "k8s.io/client-go/listers/core/v1"
"k8s.io/client-go/tools/cache"
"github.com/alibaba/higress/ingress/kube/util"
"github.com/alibaba/higress/pkg/ingress/kube/util"
)
type Controller interface {

53
plugins/README.md Normal file
View File

@@ -0,0 +1,53 @@
## Wasm 插件
目前 Higress 提供了 c++ 和 golang 两种 Wasm 插件开发框架,支持 Wasm 插件路由&域名级匹配生效。
同时提供了多个内置插件,用户可以基于 Higress 提供的官方镜像仓库直接使用这些插件:
[basic-auth](./wasm-cpp/basic_auth)Basic Auth 认证鉴权
[key-auth](./wasm-cpp/key_auth)Key 认证鉴权
[hmac-auth](./wasm-cpp/hmac_auth)Hmac 认证鉴权
[jwt-auth](./wasm-cpp/jwt_auth) JWT 认证鉴权
[bot-detect](./wasm-cpp/bot_detect):防互联网爬虫
[custom-response](./wasm-cpp/custom_response):自定义应答
[key-rate-limit](./wasm-cpp/key_rate_limit):针对参数的限流
[request-block](./wasm-cpp/request_block):自定义请求屏蔽
使用方式具体可以参考此[文档](./wasm-go/README.md) 中相关说明。
所有内置插件都已上传至 Higress 的官方镜像仓库higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins
例如用如下配置使用 request-block 插件 的 1.0.0 版本:
```yaml
apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
name: request-block
namespace: higress-system
spec:
selector:
matchLabels:
higress: higress-system-higress-gateway
pluginConfig:
block_urls:
- "swagger.html"
url: oci://higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/request-block:1.0.0
```
## 贡献 Wasm 插件
如果您想要为 Higress 贡献插件请参考下述说明。
根据你选择的开发语言,将插件代码放到 [wasm-cpp/extensions](./wasm-cpp/extensions) ,或者 [go-cpp/extensions](./wasm-go/extensions) 目录下。
除了代码以外,需要额外提供一个 README.md 文件说明插件配置方式,以及 VERSION 文件用于记录插件版本,用作推送镜像时的 tag。
提交 PR 后,我们将评估插件的通用性,并对代码逻辑进行审查,确认无误后,会将插件镜像推送到官方仓库,后面将出现在社区的插件市场中。

View File

@@ -0,0 +1,110 @@
# 功能说明
`basic-auth`插件实现了基于 HTTP Basic Auth 标准进行认证鉴权的功能
# 配置字段
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| ----------- | --------------- | -------- | ------ | ---------------------------------------------------- |
| `consumers` | array of object | 必填 | - | 配置服务的调用者,用于对请求进行认证 |
| `_rules_` | array of object | 选填 | - | 配置特定路由或域名的访问权限列表,用于对请求进行鉴权 |
`consumers`中每一项的配置字段说明如下:
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| ------------ | -------- | -------- | ------ | ------------------------ |
| `credential` | string | 必填 | - | 配置该consumer的访问凭证 |
| `name` | string | 必填 | - | 配置该consumer的名称 |
`_rules_` 中每一项的配置字段说明如下:
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| ---------------- | --------------- | ------------------------------------------------- | ------ | -------------------------------------------------- |
| `_match_route_` | array of string | 选填,`_match_route_``_match_domain_`中选填一项 | - | 配置要匹配的路由名称 |
| `_match_domain_` | array of string | 选填,`_match_route_``_match_domain_`中选填一项 | - | 配置要匹配的域名 |
| `allow` | array of string | 必填 | - | 对于符合匹配条件的请求配置允许访问的consumer名称 |
**注意:**
- 若不配置`_rules_`字段,则默认对当前网关实例的所有路由开启认证;
- 对于通过认证鉴权的请求请求的header会被添加一个`X-Mse-Consumer`字段,用以标识调用者的名称。
# 配置示例
## 对特定路由或域名开启认证和鉴权
以下配置将对网关特定路由或域名开启 Basic Auth 认证和鉴权,注意凭证信息中的用户名和密码之间使用":"分隔,`credential`字段不能重复
```yaml
# 使用 _rules_ 字段进行细粒度规则配置
consumers:
- credential: 'admin:123456'
name: consumer1
- credential: 'guest:abc'
name: consumer2
_rules_:
# 规则一:按路由名称匹配生效
- _match_route_:
- route-a
- route-b
allow:
- consumer1
# 规则二:按域名匹配生效
- _match_domain_:
- "*.example.com"
- test.com
allow:
- consumer2
```
此例 `_match_route_` 中指定的 `route-a``route-b` 即在创建网关路由时填写的路由名称,当匹配到这两个路由时,将允许`name``consumer1`的调用者访问,其他调用者不允许访问;
此例 `_match_domain_` 中指定的 `*.example.com``test.com` 用于匹配请求的域名,当发现域名匹配时,将允许`name``consumer2`的调用者访问,其他调用者不允许访问。
### 根据该配置,下列请求可以允许访问:
**请求指定用户名密码**
```bash
# 假设以下请求将会匹配到route-a路由
# 使用 curl 的 -u 参数指定
curl -u admin:123456 http://xxx.hello.com/test
# 或者直接指定 Authorization 请求头,用户名密码使用 base64 编码
curl -H 'Authorization: Basic YWRtaW46MTIzNDU2' http://xxx.hello.com/test
```
认证鉴权通过后请求的header中会被添加一个`X-Mse-Consumer`字段,在此例中其值为`consumer1`,用以标识调用方的名称
### 下列请求将拒绝访问:
**请求未提供用户名密码返回401**
```bash
curl http://xxx.hello.com/test
```
**请求提供的用户名密码错误返回401**
```bash
curl -u admin:abc http://xxx.hello.com/test
```
**根据请求的用户名和密码匹配到的调用者无访问权限返回403**
```bash
# consumer2不在route-a的allow列表里
curl -u guest:abc http://xxx.hello.com/test
```
## 网关实例级别开启
以下配置未指定`_rules_`字段,因此将对网关实例级别开启 Basic Auth 认证
```yaml
consumers:
- credential: 'admin:123456'
name: consumer1
- credential: 'guest:abc'
name: consumer2
```
# 相关错误码
| HTTP 状态码 | 出错信息 | 原因说明 |
| ----------- | ------------------------------------------------------------------------------ | ---------------------- |
| 401 | Request denied by Basic Auth check. No Basic Authentication information found. | 请求未提供凭证 |
| 401 | Request denied by Basic Auth check. Invalid username and/or password | 请求凭证无效 |
| 403 | Request denied by Basic Auth check. Unauthorized consumer | 请求的调用方无访问权限 |

View File

@@ -0,0 +1 @@
1.0.0

View File

@@ -0,0 +1,75 @@
# 功能说明
`bot-detect`插件可以用于识别并阻止互联网爬虫对站点资源的爬取
# 配置字段
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| -------- | -------- | -------- | -------- | -------- |
| allow | array of string | 选填 | - | 配置匹配 User-Agent 请求头的正则表达式,匹配命中时将允许其访问 |
| deny | array of string | 选填 | - | 配置匹配 User-Agent 请求头的正则表达式,匹配命中时将屏蔽请求 |
| blocked_code | number | 选填 | 403 | 配置请求被屏蔽时返回的 HTTP 状态码 |
| blocked_message | string | 选填 | - | 配置请求被屏蔽时返回的 HTTP 应答 Body |
`allow``deny` 字段可以均不配置,则执行默认的爬虫判断逻辑,通过配置 `allow` 字段可以将原本命中默认爬虫判断逻辑的请求放行,通过配置 `deny` 字段可以增加额外的爬虫判断逻辑。
默认的爬虫判断正则表达式集合如下:
```bash
# Bots General matcher 'name/0.0'
(?:\/[A-Za-z0-9\.]+|) {0,5}([A-Za-z0-9 \-_\!\[\]:]{0,50}(?:[Aa]rchiver|[Ii]ndexer|[Ss]craper|[Bb]ot|[Ss]pider|[Cc]rawl[a-z]{0,50}))[/ ](\d+)(?:\.(\d+)(?:\.(\d+)|)|)
# Bots General matcher 'name 0.0'
(?:\/[A-Za-z0-9\.]+|) {0,5}([A-Za-z0-9 \-_\!\[\]:]{0,50}(?:[Aa]rchiver|[Ii]ndexer|[Ss]craper|[Bb]ot|[Ss]pider|[Cc]rawl[a-z]{0,50})) (\d+)(?:\.(\d+)(?:\.(\d+)|)|)
# Bots containing spider|scrape|bot(but not CUBOT)|Crawl
((?:[A-z0-9]{1,50}|[A-z\-]{1,50} ?|)(?: the |)(?:[Ss][Pp][Ii][Dd][Ee][Rr]|[Ss]crape|[Cc][Rr][Aa][Ww][Ll])[A-z0-9]{0,50})(?:(?:[ /]| v)(\d+)(?:\.(\d+)|)(?:\.(\d+)|)|)
# Bots Pattern '/name-0.0'
/((?:Ant-)?Nutch|[A-z]+[Bb]ot|[A-z]+[Ss]pider|Axtaris|fetchurl|Isara|ShopSalad|Tailsweep)[ \-](\d+)(?:\.(\d+)(?:\.(\d+))?)?
# Bots Pattern 'name/0.0'
\b(008|Altresium|Argus|BaiduMobaider|BoardReader|DNSGroup|DataparkSearch|EDI|Goodzer|Grub|INGRID|Infohelfer|LinkedInBot|LOOQ|Nutch|OgScrper|PathDefender|Peew|PostPost|Steeler|Twitterbot|VSE|WebCrunch|WebZIP|Y!J-BR[A-Z]|YahooSeeker|envolk|sproose|wminer)/(\d+)(?:\.(\d+)|)(?:\.(\d+)|)
# More bots
(CSimpleSpider|Cityreview Robot|CrawlDaddy|CrawlFire|Finderbots|Index crawler|Job Roboter|KiwiStatus Spider|Lijit Crawler|QuerySeekerSpider|ScollSpider|Trends Crawler|USyd-NLP-Spider|SiteCat Webbot|BotName\/\$BotVersion|123metaspider-Bot|1470\.net crawler|50\.nu|8bo Crawler Bot|Aboundex|Accoona-[A-z]{1,30}-Agent|AdsBot-Google(?:-[a-z]{1,30}|)|altavista|AppEngine-Google|archive.{0,30}\.org_bot|archiver|Ask Jeeves|[Bb]ai[Dd]u[Ss]pider(?:-[A-Za-z]{1,30})(?:-[A-Za-z]{1,30}|)|bingbot|BingPreview|blitzbot|BlogBridge|Bloglovin|BoardReader Blog Indexer|BoardReader Favicon Fetcher|boitho.com-dc|BotSeer|BUbiNG|\b\w{0,30}favicon\w{0,30}\b|\bYeti(?:-[a-z]{1,30}|)|Catchpoint(?: bot|)|[Cc]harlotte|Checklinks|clumboot|Comodo HTTP\(S\) Crawler|Comodo-Webinspector-Crawler|ConveraCrawler|CRAWL-E|CrawlConvera|Daumoa(?:-feedfetcher|)|Feed Seeker Bot|Feedbin|findlinks|Flamingo_SearchEngine|FollowSite Bot|furlbot|Genieo|gigabot|GomezAgent|gonzo1|(?:[a-zA-Z]{1,30}-|)Googlebot(?:-[a-zA-Z]{1,30}|)|Google SketchUp|grub-client|gsa-crawler|heritrix|HiddenMarket|holmes|HooWWWer|htdig|ia_archiver|ICC-Crawler|Icarus6j|ichiro(?:/mobile|)|IconSurf|IlTrovatore(?:-Setaccio|)|InfuzApp|Innovazion Crawler|InternetArchive|IP2[a-z]{1,30}Bot|jbot\b|KaloogaBot|Kraken|Kurzor|larbin|LEIA|LesnikBot|Linguee Bot|LinkAider|LinkedInBot|Lite Bot|Llaut|lycos|Mail\.RU_Bot|masscan|masidani_bot|Mediapartners-Google|Microsoft .{0,30} Bot|mogimogi|mozDex|MJ12bot|msnbot(?:-media {0,2}|)|msrbot|Mtps Feed Aggregation System|netresearch|Netvibes|NewsGator[^/]{0,30}|^NING|Nutch[^/]{0,30}|Nymesis|ObjectsSearch|OgScrper|Orbiter|OOZBOT|PagePeeker|PagesInventory|PaxleFramework|Peeplo Screenshot Bot|PlantyNet_WebRobot|Pompos|Qwantify|Read%20Later|Reaper|RedCarpet|Retreiver|Riddler|Rival IQ|scooter|Scrapy|Scrubby|searchsight|seekbot|semanticdiscovery|SemrushBot|Simpy|SimplePie|SEOstats|SimpleRSS|SiteCon|Slackbot-LinkExpanding|Slack-ImgProxy|Slurp|snappy|Speedy Spider|Squrl Java|Stringer|TheUsefulbot|ThumbShotsBot|Thumbshots\.ru|Tiny Tiny RSS|Twitterbot|WhatsApp|URL2PNG|Vagabondo|VoilaBot|^vortex|Votay bot|^voyager|WASALive.Bot|Web-sniffer|WebThumb|WeSEE:[A-z]{1,30}|WhatWeb|WIRE|WordPress|Wotbox|www\.almaden\.ibm\.com|Xenu(?:.s|) Link Sleuth|Xerka [A-z]{1,30}Bot|yacy(?:bot|)|YahooSeeker|Yahoo! Slurp|Yandex\w{1,30}|YodaoBot(?:-[A-z]{1,30}|)|YottaaMonitor|Yowedo|^Zao|^Zao-Crawler|ZeBot_www\.ze\.bz|ZooShot|ZyBorg)(?:[ /]v?(\d+)(?:\.(\d+)(?:\.(\d+)|)|)|)
```
# 配置示例
## 放行原本命中爬虫规则的请求
```yaml
allow:
- ".*Go-http-client.*"
```
若不作该配置,默认的 Golang 网络库请求会被视做爬虫,被禁止访问
## 增加爬虫判断
```yaml
deny:
- "spd-tools.*"
```
根据该配置,下列请求将被禁止访问:
```bash
curl http://example.com -H 'User-Agent: spd-tools/1.1'
curl http://exmaple.com -H 'User-Agent: spd-tools'
```
## 对特定路由或域名开启
```yaml
# 使用 _rules_ 字段进行细粒度规则配置
_rules_:
# 规则一:按路由名称匹配生效
- _match_route_:
- route-a
- route-b
# 规则二:按域名匹配生效
- _match_domain_:
- "*.example.com"
- test.com
allow:
- ".*Go-http-client.*"
```
此例 `_match_route_` 中指定的 `route-a``route-b` 即在创建网关路由时填写的路由名称,当匹配到这两个路由时,将使用此段配置;
此例 `_match_domain_` 中指定的 `*.example.com``test.com` 用于匹配请求的域名,当发现域名匹配时,将使用此段配置;
配置的匹配生效顺序,将按照 `_rules_` 下规则的排列顺序,匹配第一个规则后生效对应配置,后续规则将被忽略。

View File

@@ -0,0 +1 @@
1.0.0

View File

@@ -0,0 +1,80 @@
# 功能说明
`custom-response`插件支持配置自定义的响应,包括自定义 HTTP 应答状态码、HTTP 应答头,以及 HTTP 应答 Body。可以用于 Mock 响应,也可以用于判断特定状态码后给出自定义应答,例如在触发网关限流策略时实现自定义响应。
# 配置字段
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| -------- | -------- | -------- | -------- | -------- |
| status_code | number | 选填 | 200 | 自定义 HTTP 应答状态码 |
| headers | array of string | 选填 | - | 自定义 HTTP 应答头key 和 value 用`=`分隔 |
| body | string | 选填 | - | 自定义 HTTP 应答 Body |
| enable_on_status | array of number | 选填 | - | 匹配原始状态码,生成自定义响应,不填写时,不判断原始状态码 |
# 配置示例
## Mock 应答场景
```yaml
status_code: 200
headers:
- Content-Type=application/json
- Hello=World
body: "{\"hello\":\"world\"}"
```
根据该配置,请求将返回自定义应答如下:
```text
HTTP/1.1 200 OK
Content-Type: application/json
Hello: World
Content-Length: 17
{"hello":"world"}
```
## 触发限流时自定义响应
```yaml
enable_on_status:
- 429
status_code: 302
headers:
- Location=https://example.com
```
触发网关限流时一般会返回 `429` 状态码,这时请求将返回自定义应答如下:
```text
HTTP/1.1 302 Found
Location: https://example.com
```
从而实现基于浏览器 302 重定向机制,将限流后的用户引导到其他页面,比如可以是一个 CDN 上的静态页面。
如果希望触发限流时,正常返回其他应答,参考 Mock 应答场景配置相应的字段即可。
## 对特定路由或域名开启
```yaml
# 使用 _rules_ 字段进行细粒度规则配置
_rules_:
# 规则一:按路由名称匹配生效
- _match_route_:
- route-a
- route-b
body: "{\"hello\":\"world\"}"
# 规则二:按域名匹配生效
- _match_domain_:
- "*.example.com"
- test.com
enable_on_status:
- 429
status_code: 200
headers:
- Content-Type=application/json
body: "{\"errmsg\": \"rate limited\"}"
```
此例 `_match_route_` 中指定的 `route-a``route-b` 即在创建网关路由时填写的路由名称,当匹配到这两个路由时,将使用此段配置;
此例 `_match_domain_` 中指定的 `*.example.com``test.com` 用于匹配请求的域名,当发现域名匹配时,将使用此段配置;
配置的匹配生效顺序,将按照 `_rules_` 下规则的排列顺序,匹配第一个规则后生效对应配置,后续规则将被忽略。

View File

@@ -0,0 +1 @@
1.0.0

View File

@@ -0,0 +1,285 @@
# 功能说明
`hmac-auth`插件实现了基于 HMAC 算法为 HTTP 请求生成不可伪造的签名,并基于签名实现身份认证和鉴权
# 配置字段
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| ------------- | --------------- | -------- | ------ | ------------------------------------------------------------------------------------------------------------------- |
| `consumers` | array of object | 必填 | - | 配置服务的调用者,用于对请求进行认证 |
| `date_offset` | number | 选填 | - | 配置允许的客户端最大时间偏移,单位为秒,根据请求头`Date`解析客户端 UTC 时间,可用于避免请求重放;未配置时,不做校验 |
| `_rules_` | array of object | 选填 | - | 配置特定路由或域名的访问权限列表,用于对请求进行鉴权 |
`consumers`中每一项的配置字段说明如下:
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| -------- | -------- | -------- | ------ | ----------------------------------- |
| `key` | string | 必填 | - | 配置从请求的`x-ca-key`头中提取的key |
| `secret` | string | 必填 | - | 配置用于生成签名的secret |
| `name` | string | 必填 | - | 配置该consumer的名称 |
`_rules_` 中每一项的配置字段说明如下:
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| ---------------- | --------------- | ------------------------------------------------- | ------ | -------------------------------------------------- |
| `_match_route_` | array of string | 选填,`_match_route_``_match_domain_`中选填一项 | - | 配置要匹配的路由名称 |
| `_match_domain_` | array of string | 选填,`_match_route_``_match_domain_`中选填一项 | - | 配置要匹配的域名 |
| `allow` | array of string | 必填 | - | 对于符合匹配条件的请求配置允许访问的consumer名称 |
**注意:**
- 若不配置`_rules_`字段,则默认对当前网关实例的所有路由开启认证;
- 对于通过认证鉴权的请求请求的header会被添加一个`X-Mse-Consumer`字段,用以标识调用者的名称。
# 配置示例
以下配置将对网关特定路由或域名开启 Hmac Auth 认证和鉴权,注意`key`字段不能重复
## 对特定路由或域名开启
```yaml
consumers:
- key: appKey-example-1
secret: appSecret-example-1
name: consumer-1
- key: appKey-example-2
secret: appSecret-example-2
name: consumer-2
# 使用 _rules_ 字段进行细粒度规则配置
_rules_:
# 规则一:按路由名称匹配生效
- _match_route_:
- route-a
- route-b
allow:
- consumer-1
# 规则二:按域名匹配生效
- _match_domain_:
- "*.example.com"
- test.com
allow:
- consumer-2
```
每条匹配规则下的`allow`字段用于指定该匹配条件下允许访问的调用者列表;
此例 `_match_route_` 中指定的 `route-a``route-b` 即在创建网关路由时填写的路由名称,当匹配到这两个路由时,将允许`name``consumer-1`的调用者访问,其他调用者不允许访问;
此例 `_match_domain_` 中指定的 `*.example.com``test.com` 用于匹配请求的域名,当发现域名匹配时,将允许`name``consumer-2`的调用者访问,其他调用者不允许访问;
认证成功后请求的header中会被添加一个`X-Mse-Consumer`字段,其值为调用方的名称,例如`consumer-1`
## 网关实例级别开启
以下配置将对网关实例级别开启 Hamc Auth 认证
```yaml
consumers:
- key: appKey-example-1
secret: appSecret-example-1
name: consumer-1
- key: appKey-example-2
secret: appSecret-example-2
name: consumer-2
```
# 签名机制说明
## 配置准备
如上指引,在插件配置中配置生成和验证签名需要用的凭证配置
- key: 用于请求头 `x-ca-key` 中设置
- secret: 用于生成请求签名
## 客户端签名生成方式
### 流程简介
客户端生成签名一共分三步处理:
1. 从原始请求中提取关键数据,得到一个用来签名的字符串
2. 使用加密算法和配置的 `secret` 对关键数据签名串进行加密处理,得到签名
3. 将签名所相关的所有头加入到原始HTTP请求中得到最终HTTP请求
如下图所示:
![](https://help-static-aliyun-doc.aliyuncs.com/assets/img/zh-CN/1745707061/p188113.png)
### 签名串提取流程
客户端需要从Http请求中提取出关键数据组合成一个签名串生成的签名串的格式如下
```text
HTTPMethod
Accept
Content-MD5
Content-Type
Date
Headers
PathAndParameters
```
以上7个字段构成整个签名串字段之间使用\n间隔如果Headers为空则不需要加\n其他字段如果为空都需要保留\n。签名大小写敏感。下面介绍下每个字段的提取规则
- HTTPMethodHTTP的方法全部大写比如POST
- Accept请求中的Accept头的值可为空。建议显式设置 Accept Header。当 Accept 为空时,部分 Http 客户端会给 Accept 设置默认值为 `*/*`,导致签名校验失败。
- Content-MD5请求中的Content-MD5头的值可为空只有在请求存在Body且Body为非Form形式时才计算Content-MD5头下面是Java的Content-MD5值的参考计算方式
```java
String content-MD5 = Base64.encodeBase64(MD5(bodyStream.getbytes("UTF-8")));
```
- Content-Type请求中的Content-Type头的值可为空
- Date请求中的Date头的值当未开启`date_offset`配置时,可为空,否则将用于时间偏移校验
- Headers用户可以选取指定的header参与签名关于header的签名串拼接方式有以下规则
- 参与签名计算的Header的Key按照字典排序后使用如下方式拼接
```text
HeaderKey1 + ":" + HeaderValue1 + "\n"\+
HeaderKey2 + ":" + HeaderValue2 + "\n"\+
...
HeaderKeyN + ":" + HeaderValueN + "\n"
```
- 某个Header的Value为空则使用HeaderKey+":"+"\n"参与签名需要保留Key和英文冒号
- 所有参与签名的Header的Key的集合使用英文逗号分割放到Key为X-Ca-Signature-Headers的Header中
- 以下Header不参与Header签名计算X-Ca-Signature、X-Ca-Signature-Headers、Accept、Content-MD5、Content-Type、Date
- PathAndParameters: 这个字段包含PathQuery和Form中的所有参数具体组织形式如下
```text
Path + "?" + Key1 + "=" + Value1 + "&" + Key2 + "=" + Value2 + ... "&" + KeyN + "=" + ValueN
```
注意:
1. Query和Form参数对的Key按照字典排序后使用上面的方式拼接
2. Query和Form参数为空时则直接使用Path不需要添加?
3. 参数的Value为空时只保留Key参与签名等号不需要再加入签名
4. Query和Form存在数组参数时key相同value不同的参数 取第一个Value参与签名计算
### 签名串提取示例
初始的HTTP请求
```text
POST /http2test/test?param1=test HTTP/1.1
host:api.aliyun.com
accept:application/json; charset=utf-8
ca_version:1
content-type:application/x-www-form-urlencoded; charset=utf-8
x-ca-timestamp:1525872629832
date:Wed, 09 May 2018 13:30:29 GMT+00:00
user-agent:ALIYUN-ANDROID-DEMO
x-ca-nonce:c9f15cbf-f4ac-4a6c-b54d-f51abf4b5b44
content-length:33
username=xiaoming&password=123456789
```
生成的正确签名串为:
```text
POST
application/json; charset=utf-8
application/x-www-form-urlencoded; charset=utf-8
Wed, 09 May 2018 13:30:29 GMT+00:00
x-ca-key:203753385
x-ca-nonce:c9f15cbf-f4ac-4a6c-b54d-f51abf4b5b44
x-ca-signature-method:HmacSHA256
x-ca-timestamp:1525872629832
/http2test/test?param1=test&password=123456789&username=xiaoming
```
### 签名计算流程
客户端从HTTP请求中提取出关键数据组装成签名串后需要对签名串进行加密及编码处理形成最终的签名
具体的加密形式如下,其中 `stringToSign` 是提取出来的签名串,`secret` 就是插件配置中填写的,`sign` 是最终生成的签名:
```java
Mac hmacSha256 = Mac.getInstance("HmacSHA256");
byte[] secretBytes = secret.getBytes("UTF-8");
hmacSha256.init(new SecretKeySpec(secretBytes, 0, secretBytes.length, "HmacSHA256"));
byte[] result = hmacSha256.doFinal(stringToSign.getBytes("UTF-8"));
String sign = Base64.encodeBase64String(result);
```
总结一下,就是将 `stringToSign` 使用UTF-8解码后得到Byte数组然后使用加密算法对Byte数组进行加密然后使用Base64算法进行编码形成最终的签名。
### 添加签名流程
客户端需要将以下四个Header放在HTTP请求中传输给API网关进行签名校验
- x-ca-key取值APP Key必选
- x-ca-signature-method签名算法取值HmacSHA256或者HmacSHA1可选默认值为HmacSHA256
- x-ca-signature-headers所有签名头的Key的集合使用英文逗号分隔可选
- x-ca-signature签名必选
下面是携带签名的整个HTTP请求的示例
```text
POST /http2test/test?param1=test HTTP/1.1
host:api.aliyun.com
accept:application/json; charset=utf-8
ca_version:1
content-type:application/x-www-form-urlencoded; charset=utf-8
x-ca-timestamp:1525872629832
date:Wed, 09 May 2018 13:30:29 GMT+00:00
user-agent:ALIYUN-ANDROID-DEMO
x-ca-nonce:c9f15cbf-f4ac-4a6c-b54d-f51abf4b5b44
x-ca-key:203753385
x-ca-signature-method:HmacSHA256
x-ca-signature-headers:x-ca-timestamp,x-ca-key,x-ca-nonce,x-ca-signature-method
x-ca-signature:xfX+bZxY2yl7EB/qdoDy9v/uscw3Nnj1pgoU+Bm6xdM=
content-length:33
username=xiaoming&password=123456789
```
## 服务端签名验证方式
### 流程简介
服务器验证客户端签名一共分四步处理:
1. 从接收到的请求中提取关键数据,得到一个用来签名的字符串
2. 从接收到的请求中读取 `key` ,通过 `key` 查询到对应的 `secret`
3. 使用加密算法和 `secret` 对关键数据签名串进行加密处理,得到签名
4. 从接收到的请求中读取客户端签名,对比服务器端签名和客户端签名的一致性
如下图所示:
![](https://help-static-aliyun-doc.aliyuncs.com/assets/img/zh-CN/1745707061/p188116.png)
## 签名排错方法
网关签名校验失败时会将服务端的签名串StringToSign放到HTTP Response的Header中返回到客户端Key为X-Ca-Error-Message用户只需要将本地计算的签名串StringToSign与服务端返回的签名串进行对比即可找到问题
如果服务端与客户端的StringToSign一致请检查用于签名计算的APP Secret是否正确
因为HTTP Header中无法表示换行因此StringToSign中的换行符都被替换成`#`,如下所示:
```text
X-Ca-Error-Message: Server StringToSign:`GET#application/json##application/json##X-Ca-Key:200000#X-Ca-Timestamp:1589458000000#/app/v1/config/keys?keys=TEST`
```
# 相关错误码
| HTTP 状态码 | 出错信息 | 原因说明 |
| ----------- | ---------------------- | -------------------------------------------------------------------------------- |
| 401 | Invalid Key | 请求头未提供 x-ca-key或者 x-ca-key 无效 |
| 401 | Empty Signature | 请求头未提供 x-ca-signature 签名串 |
| 400 | Invalid Signature | 请求头 x-ca-signature 签名串,与服务端计算得到签名不一致 |
| 400 | Invalid Content-MD5 | 请求头 content-md5 不正确 |
| 400 | Invalid Date | 根据请求头 date 计算时间偏移超过配置的 date_offset |
| 413 | Request Body Too Large | 请求 Body 超过限制大小32 MB |
| 413 | Payload Too Large | 请求 Body 超过全局配置 DownstreamConnectionBufferLimits |
| 403 | Unauthorized Consumer | 请求的调用方无访问权限 |

View File

@@ -0,0 +1 @@
1.0.0

View File

@@ -0,0 +1,400 @@
# 功能说明
`jwt-auth`插件实现了基于JWT(JSON Web Tokens)进行认证鉴权的功能支持从HTTP请求的URL参数、请求头、Cookie字段解析JWT同时验证该Token是否有权限访问。
本插件和`安全能力->认证鉴权`中JWT认证的区别是额外提供了调用方身份识别的能力支持对不同调用方配置不同的JWT凭证。
# 详细说明
## 1、基于token的认证
### 1.1 简介
很多对外开放的API需要识别请求者的身份并据此判断所请求的资源是否可以返回给请求者。token就是一种用于身份验证的机制基于这种机制应用不需要在服务端保留用户的认证信息或者会话信息可实现无状态、分布式的Web应用授权为应用的扩展提供了便利。
### 1.2 流程描述
![](https://help-static-aliyun-doc.aliyuncs.com/assets/img/zh-CN/2336348951/p135822.png)
上图是网关利用JWT实现认证的整个业务流程时序图下面我们用文字来详细描述图中标注的步骤
1. 客户端向API网关发起认证请求请求中一般会携带终端用户的用户名和密码
2. 网关将请求直接转发给后端服务;
3. 后端服务读取请求中的验证信息比如用户名、密码进行验证验证通过后使用私钥生成标准的token返回给网关
4. 网关将携带token的应答返回给客户端客户端需要将这个token缓存到本地
5. 客户端向API网关发送业务请求请求中携带token
6. 网关使用用户设定的公钥对请求中的token进行验证验证通过后将请求透传给后端服务
7. 后端服务进行业务处理后应答;
8. 网关将业务应答返回给客户端。
在这个整个过程中, 网关利用token认证机制实现了用户使用自己的用户体系对自己API进行授权的能力。下面我们就要介绍网关实现token认证所使用的结构化令牌Json Web Token(JWT)。
### 1.3 JWT
#### 1.3.1 简介
Json Web TokeJWT是为了在网络应用环境间传递声明而执行的一种基于JSON的开放标准RFC7519。JWT一般可以用作独立的身份验证令牌可以包含用户标识、用户角色和权限等信息以便于从资源服务器获取资源也可以增加一些额外的其它业务逻辑所必须的声明信息特别适用于分布式站点的登录场景。
#### 1.3.2 JWT的构成
`eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ`
如上面的例子所示JWT就是一个字符串由三部分构成
- Header头部
- Payload数据
- Signature签名
**Header**
JWT的头部承载两个信息
- 声明类型这里是JWT
- 声明加密的算法
网关支持的加密算法如下:
```text
ES256, ES384, ES512,
HS256, HS384, HS512,
RS256, RS384, RS512,
PS256, PS384, PS512,
EdDSA
```
完整的头部就像下面这样的JSON
```js
{
'typ': 'JWT',
'alg': 'HS256'
}
```
然后将头部进行Base64编码该编码是可以对称解码的构成了第一部分。
`eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9`
**Payload**
载荷就是存放有效信息的地方。定义细节如下:
```text
iss令牌颁发者。表示该令牌由谁创建该声明是一个字符串
sub: Subject Identifieriss提供的终端用户的标识在iss范围内唯一最长为255个ASCII个字符区分大小写
audAudience(s),令牌的受众,分大小写的字符串数组
expExpiration time令牌的过期时间戳。超过此时间的token会作废 该声明是一个整数是1970年1月1日以来的秒数
iat: 令牌的颁发时间该声明是一个整数是1970年1月1日以来的秒数
jti: 令牌的唯一标识,该声明的值在令牌颁发者创建的每一个令牌中都是唯一的,为了防止冲突,它通常是一个密码学随机值。这个值相当于向结构化令牌中加入了一个攻击者无法获得的随机熵组件,有利于防止令牌猜测攻击和重放攻击。
```
也可以新增用户系统需要使用的自定义字段比如下面的例子添加了name 用户昵称:
```js
{
"sub": "1234567890",
"name": "John Doe"
}
```
然后将其进行Base64编码得到JWT的第二部分
`JTdCJTBBJTIwJTIwJTIyc3ViJTIyJTNBJTIwJTIyMTIzNDU2Nzg5MCUyMiUyQyUwQSUyMCUyMCUyMm5hbWUlMjIlM0ElMjAlMjJKb2huJTIwRG9lJTIyJTBBJTdE`
**Signature**
这个部分需要Base64编码后的Header和Base64编码后的Payload使用 . 连接组成的字符串然后通过Header中声明的加密方式进行加密$secret 表示用户的私钥然后就构成了jwt的第三部分。
```js
var encodedString = base64UrlEncode(header) + '.' + base64UrlEncode(payload);
var signature = HMACSHA256(encodedString, '$secret');
```
将这三部分用 . 连接成一个完整的字符串,就构成了 1.3.2 节最开始的JWT示例。
#### 1.3.3 时效
网关会验证token中的exp字段一旦这个字段过期了网关会认为这个token无效而将请求直接打回。过期时间这个值必须设置。
#### 1.3.4 JWT的几个特点
1. JWT 默认是不加密,不能将秘密数据写入 JWT。
2. JWT 不仅可以用于认证,也可以用于交换信息。有效使用 JWT可以降低服务器查询数据库的次数。
3. JWT 的最大缺点是,由于服务器不保存 session 状态,因此无法在使用过程中废止某个 token或者更改 token 的权限。也就是说,一旦 JWT 签发了,在到期之前就会始终有效,除非服务器部署额外的逻辑。
4. JWT 本身包含了认证信息一旦泄露任何人都可以获得该令牌的所有权限。为了减少盗用JWT 的有效期应该设置得比较短。对于一些比较重要的权限,使用时应该再次对用户进行认证。
5. 为了减少盗用JWT 不应该使用 HTTP 协议明码传输要使用HTTPS 协议传输。
## 2、用户系统如何应用JWT插件保护API
### 2.1 生成一对JWKJSON Web 密钥)
**方法一、在线生成:**
用户可以在这个站点https://mkjwk.org 生成用于token生成与验证的私钥与公钥 私钥用于授权服务签发JWT公钥配置到JWT插件中用于网关对请求验签注意网关使用的jwks格式配置下图中Public Key需要放到keys结构体中`{"keys":[{"kty":"RSA","e":"AQAB",...}]}`
<img src="https://help-static-aliyun-doc.aliyuncs.com/assets/img/zh-CN/2336348951/p135823.png" style="zoom:50%" />
**方法二、本地生成:**
本文应用Java示例说明其他语言用户也可以找到相关的工具生成密钥对。 新建一个Maven项目加入如下依赖
```xml
<dependency>
<groupId>org.bitbucket.b_c</groupId>
<artifactId>jose4j</artifactId>
<version>0.7.0</version>
</dependency>
```
使用如下的代码生成一对RSA密钥
```java
RsaJsonWebKey rsaJsonWebKey = RsaJwkGenerator.generateJwk(2048);
final String publicKeyString = rsaJsonWebKey.toJson(JsonWebKey.OutputControlLevel.PUBLIC_ONLY);
final String privateKeyString = rsaJsonWebKey.toJson(JsonWebKey.OutputControlLevel.INCLUDE_PRIVATE);
```
### 2.2 使用JWK中的私钥实现颁发token 的认证服务
需要使用2.1节中在线生成的 Keypair JSON字符串三个方框内的第一个或者本地生成的 privateKeyString JSON字符串作为私钥来颁发token用于授权可信的用户访问受保护的API具体实现可以参考下方示例。 向客户颁发token的形式由用户根据具体的业务场景决定可以将颁发token的功能部署到生产环境配置成普通API后由访问者通过用户名密码获得也可以直接在本地环境生成token 后,直接拷贝给指定用户使用。
```java
import java.security.PrivateKey;
import org.jose4j.json.JsonUtil;
import org.jose4j.jwk.RsaJsonWebKey;
import org.jose4j.jwk.RsaJwkGenerator;
import org.jose4j.jws.AlgorithmIdentifiers;
import org.jose4j.jws.JsonWebSignature;
import org.jose4j.jwt.JwtClaims;
import org.jose4j.jwt.NumericDate;
import org.jose4j.lang.JoseException;
public class GenerateJwtDemo {
public static void main(String[] args) throws JoseException {
String keyId = "uniq_key";
//使用本文2.1节生成的Keypair
String privateKeyJson = "{\n"
+ " \"kty\": \"RSA\",\n"
+ " \"d\": "
+
"\"O9MJSOgcjjiVMNJ4jmBAh0mRHF_TlaVva70Imghtlgwxl8BLfcf1S8ueN1PD7xV6Cnq8YenSKsfiNOhC6yZ_fjW1syn5raWfj68eR7cjHWjLOvKjwVY33GBPNOvspNhVAFzeqfWneRTBbga53Agb6jjN0SUcZdJgnelzz5JNdOGaLzhacjH6YPJKpbuzCQYPkWtoZHDqWTzCSb4mJ3n0NRTsWy7Pm8LwG_Fd3pACl7JIY38IanPQDLoighFfo-Lriv5z3IdlhwbPnx0tk9sBwQBTRdZ8JkqqYkxUiB06phwr7mAnKEpQJ6HvhZBQ1cCnYZ_nIlrX9-I7qomrlE1UoQ\",\n"
+ " \"e\": \"AQAB\",\n"
+ " \"alg\": \"RS256\",\n"
+ " \"n\": \"vCuB8MgwPZfziMSytEbBoOEwxsG7XI3MaVMoocziP4SjzU4IuWuE_DodbOHQwb_thUru57_Efe"
+
"--sfATHEa0Odv5ny3QbByqsvjyeHk6ZE4mSAV9BsHYa6GWAgEZtnDceeeDc0y76utXK2XHhC1Pysi2KG8KAzqDa099Yh7s31AyoueoMnrYTmWfEyDsQL_OAIiwgXakkS5U8QyXmWicCwXntDzkIMh8MjfPskesyli0XQD1AmCXVV3h2Opm1Amx0ggSOOiINUR5YRD6mKo49_cN-nrJWjtwSouqDdxHYP-4c7epuTcdS6kQHiQERBd1ejdpAxV4c0t0FHF7MOy9kw\"\n"
+ "}";
JwtClaims claims = new JwtClaims();
claims.setGeneratedJwtId();
claims.setIssuedAtToNow();
//过期时间一定要设置
NumericDate date = NumericDate.now();
date.addSeconds(120*60);
claims.setExpirationTime(date);
claims.setNotBeforeMinutesInThePast(1);
claims.setSubject("YOUR_SUBJECT");
claims.setAudience("YOUR_AUDIENCE");
//添加自定义参数所有值请都使用String类型
claims.setClaim("userId", "1213234");
claims.setClaim("email", "userEmail@youapp.com");
JsonWebSignature jws = new JsonWebSignature();
jws.setAlgorithmHeaderValue(AlgorithmIdentifiers.RSA_USING_SHA256);
jws.setKeyIdHeaderValue(keyId);
jws.setPayload(claims.toJson());
PrivateKey privateKey = new RsaJsonWebKey(JsonUtil.parseJson(privateKeyJson)).getPrivateKey();
jws.setKey(privateKey);
String jwtResult = jws.getCompactSerialization();
System.out.println("Generate Json Web token , result is " + jwtResult);
}
}
```
# 插件配置说明
## 配置字段
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| ----------- | --------------- | ------------------------------------------- | ------ | ----------------------------------------------------------- |
| `consumers` | array of object | 必填 | - | 配置服务的调用者,用于对请求进行认证 |
| `_rules_` | array of object | 选填 | - | 配置特定路由或域名的访问权限列表,用于对请求进行鉴权 |
`consumers`中每一项的配置字段说明如下:
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| ----------------------- | ----------------- | -------- | ------------------------------------------------- | ------------------------ |
| `name` | string | 必填 | - | 配置该consumer的名称 |
| `jwks` | string | 必填 | - | https://www.rfc-editor.org/rfc/rfc7517 指定的json格式字符串是由验证JWT中签名的公钥或对称密钥组成的Json Web Key Set |
| `issuer` | string | 必填 | - | JWT的签发者需要和payload中的iss字段保持一致 |
| `claims_to_headers` | array of object | 选填 | - | 抽取JWT的payload中指定字段设置到指定的请求头中转发给后端 |
| `from_headers` | array of object | 选填 | {"name":"Authorization","value_prefix":"Bearer "} | 从指定的请求头中抽取JWT |
| `from_params` | array of string | 选填 | access_token | 从指定的URL参数中抽取JWT |
| `from_cookies` | array of string | 选填 | - | 从指定的cookie中抽取JWT |
| `clock_skew_seconds` | number | 选填 | 60 | 校验JWT的exp和iat字段时允许的时钟偏移量单位为秒 |
| `keep_token` | bool | 选填 | ture | 转发给后端时是否保留JWT |
**注意:**
- 只有当`from_headers`,`from_params`,`from_cookies`均未配置时,才会使用默认值
`from_headers` 中每一项的配置字段说明如下:
| 名称 | 数据类型 | 填写要求| 默认值 | 描述 |
| ---------------- | --------------- | ------- | ------ | --------------------------------------------------------- |
| `name` | string | 必填 | - | 抽取JWT的请求header |
| `value_prefix` | string | 必填 | - | 对请求header的value去除此前缀剩余部分作为JWT |
`claims_to_headers` 中每一项的配置字段说明如下:
| 名称 | 数据类型 | 填写要求| 默认值 | 描述 |
| ---------------- | --------------- | ------- | ------ | --------------------------------------------------------- |
| `claim` | string | 必填 | - | JWT payload中的指定字段要求必须是字符串或无符号整数类型 |
| `header` | string | 必填 | - | 从payload取出字段的值设置到这个请求头中转发给后端 |
| `override` | bool | 选填 | true | true时存在同名请求头会进行覆盖false时追加同名请求头 |
`_rules_` 中每一项的配置字段说明如下:
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| ---------------- | --------------- | ------------------------------------------------- | ------ | -------------------------------------------------- |
| `_match_route_` | array of string | 选填,`_match_route_``_match_domain_`中选填一项 | - | 配置要匹配的路由名称 |
| `_match_domain_` | array of string | 选填,`_match_route_``_match_domain_`中选填一项 | - | 配置要匹配的域名 |
| `allow` | array of string | 必填 | - | 对于符合匹配条件的请求配置允许访问的consumer名称 |
**注意:**
- 若不配置`_rules_`字段,则默认对当前网关实例的所有路由开启认证;
- 对于通过认证鉴权的请求请求的header会被添加一个`X-Mse-Consumer`字段,用以标识调用者的名称。
## 配置示例
### 对特定路由或域名开启
以下配置将对网关特定路由或域名开启 Jwt Auth 认证和鉴权注意如果一个JWT能匹配多个`jwks`,则按照配置顺序命中第一个匹配的`consumer`
```yaml
consumers:
- name: consumer1
issuer: abcd
jwks: |
{
"keys": [
{
"kty": "oct",
"kid": "123",
"k": "hM0k3AbXBPpKOGg__Ql2Obcq7s60myWDpbHXzgKUQdYo7YCRp0gUqkCnbGSvZ2rGEl4YFkKqIqW7mTHdj-bcqXpNr-NOznEyMpVPOIlqG_NWVC3dydBgcsIZIdD-MR2AQceEaxriPA_VmiUCwfwL2Bhs6_i7eolXoY11EapLQtutz0BV6ZxQQ4dYUmct--7PLNb4BWJyQeWu0QfbIthnvhYllyl2dgeLTEJT58wzFz5HeNMNz8ohY5K0XaKAe5cepryqoXLhA-V-O1OjSG8lCNdKS09OY6O0fkyweKEtuDfien5tHHSsHXoAxYEHPFcSRL4bFPLZ0orTt1_4zpyfew",
"alg": "HS256"
}
]
}
- name: consumer2
issuer: abc
jwks: |
{
"keys": [
{
"kty": "RSA",
"e": "AQAB",
"use": "sig",
"kid": "123",
"alg": "RS256",
"n": "i0B67f1jggT9QJlZ_8QL9QQ56LfurrqDhpuu8BxtVcfxrYmaXaCtqTn7OfCuca7cGHdrJIjq99rz890NmYFZuvhaZ-LMt2iyiSb9LZJAeJmHf7ecguXS_-4x3hvbsrgUDi9tlg7xxbqGYcrco3anmalAFxsbswtu2PAXLtTnUo6aYwZsWA6ksq4FL3-anPNL5oZUgIp3HGyhhLTLdlQcC83jzxbguOim-0OEz-N4fniTYRivK7MlibHKrJfO3xa_6whBS07HW4Ydc37ZN3Rx9Ov3ZyV0idFblU519nUdqp_inXj1eEpynlxH60Ys_aTU2POGZh_25KXGdF_ZC_MSRw"
}
]
}
# 使用 _rules_ 字段进行细粒度规则配置
_rules_:
# 规则一:按路由名称匹配生效
- _match_route_:
- route-a
- route-b
allow:
- consumer1
# 规则二:按域名匹配生效
- _match_domain_:
- "*.example.com"
- test.com
allow:
- consumer2
```
此例 `_match_route_` 中指定的 `route-a``route-b` 即在创建网关路由时填写的路由名称,当匹配到这两个路由时,将允许`name``consumer1`的调用者访问,其他调用者不允许访问;
此例 `_match_domain_` 中指定的 `*.example.com``test.com` 用于匹配请求的域名,当发现域名匹配时,将允许`name``consumer2`的调用者访问,其他调用者不允许访问。
#### 根据该配置,下列请求可以允许访问:
假设以下请求会匹配到route-a这条路由
**将 JWT 设置在 url 参数中**
```bash
curl 'http://xxx.hello.com/test?access_token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjEyMyJ9.eyJpc3MiOiJhYmNkIiwic3ViIjoidGVzdCIsImlhdCI6MTY2NTY2MDUyNywiZXhwIjoxODY1NjczODE5fQ.-vBSV0bKeDwQcuS6eeSZN9dLTUnSnZVk8eVCXdooCQ4'
```
**将 JWT 设置在 http 请求头中**
```bash
curl http://xxx.hello.com/test -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjEyMyJ9.eyJpc3MiOiJhYmNkIiwic3ViIjoidGVzdCIsImlhdCI6MTY2NTY2MDUyNywiZXhwIjoxODY1NjczODE5fQ.-vBSV0bKeDwQcuS6eeSZN9dLTUnSnZVk8eVCXdooCQ4'
```
认证鉴权通过后请求的header中会被添加一个`X-Mse-Consumer`字段,在此例中其值为`consumer1`,用以标识调用方的名称
#### 下列请求将拒绝访问:
**请求未提供JWT返回401**
```bash
curl http://xxx.hello.com/test
```
**根据请求提供的JWT匹配到的调用者无访问权限返回403**
```bash
# consumer1不在*.example.com的allow列表里
curl 'http://xxx.example.com/test' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjEyMyJ9.eyJpc3MiOiJhYmNkIiwic3ViIjoidGVzdCIsImlhdCI6MTY2NTY2MDUyNywiZXhwIjoxODY1NjczODE5fQ.-vBSV0bKeDwQcuS6eeSZN9dLTUnSnZVk8eVCXdooCQ4'
```
### 网关实例级别开启
以下配置未指定`_rules_`字段,因此将对网关实例级别开启 JWT Auth 认证
```yaml
consumers:
- name: consumer1
issuer: abcd
jwks: |
{
"keys": [
{
"kty": "oct",
"kid": "123",
"k": "hM0k3AbXBPpKOGg__Ql2Obcq7s60myWDpbHXzgKUQdYo7YCRp0gUqkCnbGSvZ2rGEl4YFkKqIqW7mTHdj-bcqXpNr-NOznEyMpVPOIlqG_NWVC3dydBgcsIZIdD-MR2AQceEaxriPA_VmiUCwfwL2Bhs6_i7eolXoY11EapLQtutz0BV6ZxQQ4dYUmct--7PLNb4BWJyQeWu0QfbIthnvhYllyl2dgeLTEJT58wzFz5HeNMNz8ohY5K0XaKAe5cepryqoXLhA-V-O1OjSG8lCNdKS09OY6O0fkyweKEtuDfien5tHHSsHXoAxYEHPFcSRL4bFPLZ0orTt1_4zpyfew",
"alg": "HS256"
}
]
}
- name: consumer2
issuer: abc
jwks: |
{
"keys": [
{
"kty": "RSA",
"e": "AQAB",
"use": "sig",
"kid": "123",
"alg": "RS256",
"n": "i0B67f1jggT9QJlZ_8QL9QQ56LfurrqDhpuu8BxtVcfxrYmaXaCtqTn7OfCuca7cGHdrJIjq99rz890NmYFZuvhaZ-LMt2iyiSb9LZJAeJmHf7ecguXS_-4x3hvbsrgUDi9tlg7xxbqGYcrco3anmalAFxsbswtu2PAXLtTnUo6aYwZsWA6ksq4FL3-anPNL5oZUgIp3HGyhhLTLdlQcC83jzxbguOim-0OEz-N4fniTYRivK7MlibHKrJfO3xa_6whBS07HW4Ydc37ZN3Rx9Ov3ZyV0idFblU519nUdqp_inXj1eEpynlxH60Ys_aTU2POGZh_25KXGdF_ZC_MSRw"
}
]
}
```
# 常见错误码说明
| HTTP 状态码 | 出错信息 | 原因说明 |
| ----------- | ---------------------- | -------------------------------------------------------------------------------- |
| 401 | Jwt missing | 请求头未提供JWT |
| 401 | Jwt expired | JWT已经过期 |
| 401 | Jwt verification fails | JWT payload校验失败如iss不匹配 |
| 403 | Access Denied | 无权限访问当前路由 |

View File

@@ -0,0 +1 @@
1.0.0

View File

@@ -0,0 +1,121 @@
# 功能说明
`key-auth`插件实现了基于 API Key 进行认证鉴权的功能,支持从 HTTP 请求的 URL 参数或者请求头解析 API Key同时验证该 API Key 是否有权限访问。
# 配置字段
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| ----------- | --------------- | ------------------------------------------- | ------ | ----------------------------------------------------------- |
| `consumers` | array of object | 必填 | - | 配置服务的调用者,用于对请求进行认证 |
| `keys` | array of string | 必填 | - | API Key 的来源字段名称,可以是 URL 参数或者 HTTP 请求头名称 |
| `in_query` | bool | `in_query``in_header` 至少有一个为 true | true | 配置 true 时,网关会尝试从 URL 参数中解析 API Key |
| `in_header` | bool | `in_query``in_header` 至少有一个为 true | true | 配置 true 时,网关会尝试从 HTTP 请求头中解析 API Key |
| `_rules_` | array of object | 选填 | - | 配置特定路由或域名的访问权限列表,用于对请求进行鉴权 |
`consumers`中每一项的配置字段说明如下:
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| ------------ | -------- | -------- | ------ | ------------------------ |
| `credential` | string | 必填 | - | 配置该consumer的访问凭证 |
| `name` | string | 必填 | - | 配置该consumer的名称 |
`_rules_` 中每一项的配置字段说明如下:
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| ---------------- | --------------- | ------------------------------------------------- | ------ | -------------------------------------------------- |
| `_match_route_` | array of string | 选填,`_match_route_``_match_domain_`中选填一项 | - | 配置要匹配的路由名称 |
| `_match_domain_` | array of string | 选填,`_match_route_``_match_domain_`中选填一项 | - | 配置要匹配的域名 |
| `allow` | array of string | 必填 | - | 对于符合匹配条件的请求配置允许访问的consumer名称 |
**注意:**
- 若不配置`_rules_`字段,则默认对当前网关实例的所有路由开启认证;
- 对于通过认证鉴权的请求请求的header会被添加一个`X-Mse-Consumer`字段,用以标识调用者的名称。
# 配置示例
## 对特定路由或域名开启
以下配置将对网关特定路由或域名开启 Key Auth 认证和鉴权,注意`credential`字段不能重复
```yaml
consumers:
- credential: 2bda943c-ba2b-11ec-ba07-00163e1250b5
name: consumer1
- credential: c8c8e9ca-558e-4a2d-bb62-e700dcc40e35
name: consumer2
keys:
- apikey
- x-api-key
# 使用 _rules_ 字段进行细粒度规则配置
_rules_:
# 规则一:按路由名称匹配生效
- _match_route_:
- route-a
- route-b
allow:
- consumer1
# 规则二:按域名匹配生效
- _match_domain_:
- "*.example.com"
- test.com
allow:
- consumer2
```
此例 `_match_route_` 中指定的 `route-a``route-b` 即在创建网关路由时填写的路由名称,当匹配到这两个路由时,将允许`name``consumer1`的调用者访问,其他调用者不允许访问;
此例 `_match_domain_` 中指定的 `*.example.com``test.com` 用于匹配请求的域名,当发现域名匹配时,将允许`name``consumer2`的调用者访问,其他调用者不允许访问。
### 根据该配置,下列请求可以允许访问:
假设以下请求会匹配到route-a这条路由
**将 API Key 设置在 url 参数中**
```bash
curl http://xxx.hello.com/test?apikey=2bda943c-ba2b-11ec-ba07-00163e1250b5
```
**将 API Key 设置在 http 请求头中**
```bash
curl http://xxx.hello.com/test -H 'x-api-key: 2bda943c-ba2b-11ec-ba07-00163e1250b5'
```
认证鉴权通过后请求的header中会被添加一个`X-Mse-Consumer`字段,在此例中其值为`consumer1`,用以标识调用方的名称
### 下列请求将拒绝访问:
**请求未提供 API Key返回401**
```bash
curl http://xxx.hello.com/test
```
**请求提供的 API Key 无权访问返回401**
```bash
curl http://xxx.hello.com/test?apikey=926d90ac-ba2e-11ec-ab68-00163e1250b5
```
**根据请求提供的 API Key匹配到的调用者无访问权限返回403**
```bash
# consumer2不在route-a的allow列表里
curl http://xxx.hello.com/test?apikey=c8c8e9ca-558e-4a2d-bb62-e700dcc40e35
```
## 网关实例级别开启
以下配置未指定`_rules_`字段,因此将对网关实例级别开启 Key Auth 认证
```yaml
consumers:
- credential: 2bda943c-ba2b-11ec-ba07-00163e1250b5
name: consumer1
- credential: c8c8e9ca-558e-4a2d-bb62-e700dcc40e35
name: consumer2
keys:
- apikey
- x-api-key
```
# 相关错误码
| HTTP 状态码 | 出错信息 | 原因说明 |
| ----------- | --------------------------------------------------------- | ----------------------- |
| 401 | No API key found in request | 请求未提供 API Key |
| 401 | Request denied by Key Auth check. Invalid API key | 不允许当前 API Key 访问 |
| 403 | Request denied by Basic Auth check. Unauthorized consumer | 请求的调用方无访问权限 |

View File

@@ -0,0 +1 @@
1.0.0

View File

@@ -0,0 +1,68 @@
# 功能说明
`key-rate-limit`插件实现了基于特定键值实现限流,键值来源可以是 URL 参数、HTTP 请求头
# 配置字段
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| -------- | -------- | -------- | -------- | -------- |
| limit_by_header | string | 选填,`limit_by_header`,`limit_by_param` 中选填一项 | - | 配置获取限流键值的来源 http 请求头名称 |
| limit_by_param | string | 选填,`limit_by_header`,`limit_by_param` 中选填一项 | - | 配置获取限流键值的来源 URL 参数名称 |
| limit_keys | array of object | 必填 | - | 配置匹配键值后的限流次数 |
`limit_keys`中每一项的配置字段说明
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| -------- | -------- | -------- | -------- | -------- |
| key | string | 必填 | - | 匹配的键值 |
| query_per_second | number | 选填,`query_per_second`,`query_per_minute`,`query_per_hour`,`query_per_day` 中选填一项 | - | 允许每秒请求次数 |
| query_per_minute | number | 选填,`query_per_second`,`query_per_minute`,`query_per_hour`,`query_per_day` 中选填一项 | - | 允许每分钟请求次数 |
| query_per_hour | number | 选填,`query_per_second`,`query_per_minute`,`query_per_hour`,`query_per_day` 中选填一项 | - | 允许每小时请求次数 |
| query_per_day | number | 选填,`query_per_second`,`query_per_minute`,`query_per_hour`,`query_per_day` 中选填一项 | - | 允许每天请求次数 |
# 配置示例
## 识别请求参数 apikey进行区别限流
```yaml
limit_by_param: apikey
limit_keys:
- key: 9a342114-ba8a-11ec-b1bf-00163e1250b5
query_per_second: 10
- key: a6a6d7f2-ba8a-11ec-bec2-00163e1250b5
query_per_minute: 100
```
## 识别请求头 x-ca-key进行区别限流
```yaml
limit_by_header: x-ca-key
limit_keys:
- key: 102234
query_per_second: 10
- key: 308239
query_per_hour: 10
```
## 对特定路由或域名开启
```yaml
# 使用 _rules_ 字段进行细粒度规则配置
_rules_:
# 规则一:按路由名称匹配生效
- _match_route_:
- route-a
- route-b
limit_by_header: x-ca-key
limit_keys:
- key: 102234
query_per_second: 10
# 规则二:按域名匹配生效
- _match_domain_:
- "*.example.com"
- test.com
limit_by_header: x-ca-key
limit_keys:
- key: 102234
query_per_second: 100
```
此例 `_match_route_` 中指定的 `route-a``route-b` 即在创建网关路由时填写的路由名称,当匹配到这两个路由时,将使用此段配置;
此例 `_match_domain_` 中指定的 `*.example.com``test.com` 用于匹配请求的域名,当发现域名匹配时,将使用此段配置;
配置的匹配生效顺序,将按照 `_rules_` 下规则的排列顺序,匹配第一个规则后生效对应配置,后续规则将被忽略。

View File

@@ -0,0 +1 @@
1.0.0

View File

@@ -0,0 +1,86 @@
# 功能说明
`request-block`插件实现了基于 URL、请求头等特征屏蔽 HTTP 请求,可以用于防护部分站点资源不对外部暴露
# 配置字段
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
| -------- | -------- | -------- | -------- | -------- |
| block_urls | array of string | 选填,`block_urls`,`block_headers`,`block_bodys` 中至少必填一项 | - | 配置用于匹配需要屏蔽 URL 的字符串 |
| block_headers | array of string | 选填,`block_urls`,`block_headers`,`block_bodys` 中至少必填一项 | - | 配置用于匹配需要屏蔽请求 Header 的字符串 |
| block_bodys | array of string | 选填,`block_urls`,`block_headers`,`block_bodys` 中至少必填一项 | - | 配置用于匹配需要屏蔽请求 Body 的字符串 |
| blocked_code | number | 选填 | 403 | 配置请求被屏蔽时返回的 HTTP 状态码 |
| blocked_message | string | 选填 | - | 配置请求被屏蔽时返回的 HTTP 应答 Body |
| case_sensitive | bool | 选填 | true | 配置匹配时是否区分大小写,默认区分 |
# 配置示例
## 屏蔽请求 url 路径
```yaml
block_urls:
- swagger.html
- foo=bar
case_sensitive: false
```
根据该配置,下列请求将被禁止访问:
```bash
curl http://example.com?foo=Bar
curl http://exmaple.com/Swagger.html
```
## 屏蔽请求 header
```yaml
block_headers:
- example-key
- example-value
```
根据该配置,下列请求将被禁止访问:
```bash
curl http://example.com -H 'example-key: 123'
curl http://exmaple.com -H 'my-header: example-value'
```
## 屏蔽请求 body
```yaml
block_bodys:
- "hello world"
case_sensitive: false
```
根据该配置,下列请求将被禁止访问:
```bash
curl http://example.com -d 'Hello World'
curl http://exmaple.com -d 'hello world'
```
## 对特定路由或域名开启
```yaml
# 使用 _rules_ 字段进行细粒度规则配置
_rules_:
# 规则一:按路由名称匹配生效
- _match_route_:
- route-a
- route-b
block_bodys:
- "hello world"
# 规则二:按域名匹配生效
- _match_domain_:
- "*.example.com"
- test.com
block_urls:
- "swagger.html"
block_bodys:
- "hello world"
```
此例 `_match_route_` 中指定的 `route-a``route-b` 即在创建网关路由时填写的路由名称,当匹配到这两个路由时,将使用此段配置;
此例 `_match_domain_` 中指定的 `*.example.com``test.com` 用于匹配请求的域名,当发现域名匹配时,将使用此段配置;
配置的匹配生效顺序,将按照 `_rules_` 下规则的排列顺序,匹配第一个规则后生效对应配置,后续规则将被忽略。
# 请求 Body 大小限制
当配置了 `block_bodys` 时,仅支持小于 32 MB 的请求 Body 进行匹配。若请求 Body 大于此限制,并且不存在匹配到的 `block_urls``block_headers` 项时,不会对该请求执行屏蔽操作
当配置了 `block_bodys` 时,若请求 Body 超过全局配置 DownstreamConnectionBufferLimits将返回 `413 Payload Too Large`

View File

@@ -0,0 +1 @@
1.0.0

View File

@@ -0,0 +1,11 @@
# 功能说明
`http2-misdirect`插件用于解决网关开启 HTTP2 时,因为浏览器复用连接导致访问出现 404 等问题。
# 插件原理
HTTP2 协议允许两个不同域名的请求,在域名解析到相同 IP并且使用了相同证书的情况下复用同一条连接。这在一些特殊场景会导致复用连接的请求发送给了错误的 Virtual Host 进行处理,从而导致出现 404 等问题。
本插件基于`HTTP/2 RFC 7540``9.1.1``9.1.2`章节描述,在发现请求 SNI 与当前 Virtual Host 不匹配时,发送 HTTP 421 状态码,强制浏览器新建连接,并根据当前请求域名生成匹配的 SNI从而让网关能正确处理路由。
# 浏览器兼容性
`Safari` 浏览器 `15.1` 版本以下不支持 HTTP 421 状态码,若有此类客户端访问场景,建议对相应域名关闭 HTTP2 的 ALPN

View File

@@ -0,0 +1 @@
1.0.0

View File

@@ -0,0 +1,2 @@
FROM scratch
COPY main.wasm plugin.wasm

View File

@@ -1,48 +1,109 @@
## Intro
[English](./README_EN.md)
This SDK is used to develop the WASM Plugins of Higress.
## Requirements
## 介绍
(need support Go's type parameters)
此 SDK 用于开发 Higress 的 Wasm 插件
Go version: >= 1.18
## 编译环境要求
TinyGo version: >= 0.25.0
(需要支持 go 范型特性)
Go 版本: >= 1.18
TinyGo 版本: >= 0.25.0
## Quick Examples
### wasm plugin config
使用 [request-block](extensions/request-block) 作为例子
```yaml
# this config will take effect globally (all incoming requests are affected)
block_urls:
- "test"
_rules_:
# matching by route name takes effect
- _match_route_:
- route-a
- route-b
block_bodys:
- "hello world"
# matching by domain takes effect
- _match_domain_:
- "*.example.com"
- test.com
block_urls:
- "swagger.html"
block_bodys:
- "hello world"
```
### code
[request-block](example/request-block)
### compile to wasm
### step1. 编译 wasm
```bash
tinygo build -o main.wasm -scheduler=none -target=wasi ./main.go
tinygo build -o main.wasm -scheduler=none -target=wasi ./extensions/request-block/main.go
```
### step2. 构建并推送插件的 docker 镜像
使用这份简单的 [dockerfile](./Dockerfile).
```bash
docker build -t <your_registry_hub>/request-block:1.0.0 .
docker push <your_registry_hub>/request-block:1.0.0
```
### step3. 创建 WasmPlugin 资源
```yaml
apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
name: request-block
namespace: higress-system
spec:
selector:
matchLabels:
higress: higress-system-higress-gateway
pluginConfig:
block_urls:
- "swagger.html"
url: oci://<your_registry_hub>/request-block:1.0.0
```
创建上述资源后如果请求url携带 `swagger.html`, 则这个请求就会被拒绝,例如:
```bash
curl <your_gateway_address>/api/user/swagger.html
```
```text
HTTP/1.1 403 Forbidden
date: Wed, 09 Nov 2022 12:12:32 GMT
server: istio-envoy
content-length: 0
```
如果需要进一步控制插件的执行阶段和顺序
可以阅读此 [文档](https://istio.io/latest/docs/reference/config/proxy_extensions/wasm-plugin/) 了解更多关于 wasmplugin 的配置
## 路由级或域名级生效
```yaml
apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
name: request-block
namespace: higress-system
spec:
selector:
matchLabels:
higress: higress-system-higress-gateway
pluginConfig:
# 跟上面例子一样,这个配置会全局生效,但如果被下面规则匹配到,则会改为执行命中规则的配置
block_urls:
- "swagger.html"
_rules_:
# 路由级生效配置
- _match_route_:
- default/foo
# default 命名空间下名为 foo 的 ingress 会执行下面这个配置
block_bodys:
- "foo"
- _match_route_:
- default/bar
# default 命名空间下名为 bar 的 ingress 会执行下面这个配置
block_bodys:
- "bar"
# 域名级生效配置
- _match_domain_:
- "*.example.com"
# 若请求匹配了上面的域名, 会执行下面这个配置
block_bodys:
- "foo"
- "bar"
url: oci://<your_registry_hub>/request-block:1.0.0
```
所有规则会按上面配置的顺序一次执行匹配,当有一个规则匹配时,就停止匹配,并选择匹配的配置执行插件逻辑

View File

@@ -0,0 +1,103 @@
## Intro
This SDK is used to develop the WASM Plugins of Higress.
## Requirements
(need support Go's type parameters)
Go version: >= 1.18
TinyGo version: >= 0.25.0
## Quick Examples
Use the [request-block](example/request-block) as an example
### step1. compile to wasm
```bash
tinygo build -o main.wasm -scheduler=none -target=wasi ./example/request-block/main.go
```
### step2. build&push docker image
Use this [dockerfile](./Dockerfile).
```bash
docker build -t <your_registry_hub>/request-block:1.0.0 .
docker push <your_registry_hub>/request-block:1.0.0
```
### step3. create WasmPlugin resource
Read this [document](https://istio.io/latest/docs/reference/config/proxy_extensions/wasm-plugin/) to learn more about wasmplugin.
```yaml
apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
name: request-block
namespace: higress-system
spec:
selector:
matchLabels:
higress: higress-system-higress-gateway
pluginConfig:
block_urls:
- "swagger.html"
url: oci://<your_registry_hub>/request-block:1.0.0
```
If the url in request contains the `swagger.html`, the request will be blocked.
```bash
curl <your_gateway_address>/api/user/swagger.html
```
```text
HTTP/1.1 403 Forbidden
date: Wed, 09 Nov 2022 12:12:32 GMT
server: istio-envoy
content-length: 0
```
## route-level & domain-level takes effect
```yaml
apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
name: request-block
namespace: higress-system
spec:
selector:
matchLabels:
higress: higress-system-higress-gateway
pluginConfig:
# this config will take effect globally (all incoming requests not matched by rules below)
block_urls:
- "swagger.html"
_rules_:
# route-level takes effect
- _match_route_:
- default/foo
# the ingress foo in namespace default will use this config
block_bodys:
- "foo"
- _match_route_:
- default/bar
# the ingress bar in namespace default will use this config
block_bodys:
- "bar"
# domain-level takes effect
- _match_domain_:
- "*.example.com"
# if the request's domain matched, this config will be used
block_bodys:
- "foo"
- "bar"
url: oci://<your_registry_hub>/request-block:1.0.0
```
The rules will be matched in the order of configuration. If one match is found, it will stop, and the matching configuration will take effect.

View File

@@ -1,11 +1,11 @@
module github.com/mse-group/wasm-extensions-go/example/hello-world
module github.com/alibaba/higress/plugins/wasm-go/extensions/hello-world
go 1.18
replace github.com/mse-group/wasm-extensions-go => ../..
replace github.com/alibaba/higress/plugins/wasm-go => ../..
require (
github.com/mse-group/wasm-extensions-go v0.0.0
github.com/alibaba/higress/plugins/wasm-go v0.0.0
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c
)

Some files were not shown because too many files have changed in this diff Show More