mirror of
https://github.com/alibaba/higress.git
synced 2026-02-25 21:21:01 +08:00
Compare commits
46 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5140372159 | ||
|
|
f277d4f6ae | ||
|
|
ae20420179 | ||
|
|
a138a037ad | ||
|
|
021387c9d3 | ||
|
|
ea0a694d81 | ||
|
|
8028fe03ca | ||
|
|
a68dde0b91 | ||
|
|
48c3db85c4 | ||
|
|
7e85065832 | ||
|
|
7967f5db70 | ||
|
|
5026973d59 | ||
|
|
7097eef6ba | ||
|
|
fae222806b | ||
|
|
c63cdb62ea | ||
|
|
e43f5d106f | ||
|
|
29c95ea557 | ||
|
|
73d5cc3f1d | ||
|
|
c1ddbcef7c | ||
|
|
dd39c87311 | ||
|
|
612c94dd8a | ||
|
|
e67ed481cf | ||
|
|
ccea33655f | ||
|
|
ad4cfdbd40 | ||
|
|
3598c21da0 | ||
|
|
a624351f84 | ||
|
|
c41264816e | ||
|
|
acd80d2528 | ||
|
|
073c10df77 | ||
|
|
90f89cf588 | ||
|
|
879192cf99 | ||
|
|
d3d000753d | ||
|
|
b8a01113e3 | ||
|
|
0bb9e6dd89 | ||
|
|
ecdd077c72 | ||
|
|
e971faeb0b | ||
|
|
77013e28b6 | ||
|
|
9faa5f37d1 | ||
|
|
665d9fa943 | ||
|
|
a71ecf41d1 | ||
|
|
b825f9176f | ||
|
|
d35d23e2d5 | ||
|
|
6d1e09c146 | ||
|
|
87c39d393f | ||
|
|
c97260c4a9 | ||
|
|
5e509e7032 |
6
.github/workflows/build-and-test-plugin.yaml
vendored
6
.github/workflows/build-and-test-plugin.yaml
vendored
@@ -52,11 +52,9 @@ jobs:
|
||||
uses: actions/cache@v3
|
||||
with:
|
||||
path: |-
|
||||
envoy
|
||||
istio
|
||||
.git/modules
|
||||
key: ${{ runner.os }}-submodules-new-${{ github.run_id }}
|
||||
restore-keys: ${{ runner.os }}-submodules-new
|
||||
key: ${{ runner.os }}-submodules-cache-${{ github.run_id }}
|
||||
restore-keys: ${{ runner.os }}-submodules-cache
|
||||
|
||||
- run: git stash # restore patch
|
||||
|
||||
|
||||
18
.github/workflows/build-and-test.yaml
vendored
18
.github/workflows/build-and-test.yaml
vendored
@@ -36,11 +36,9 @@ jobs:
|
||||
uses: actions/cache@v3
|
||||
with:
|
||||
path: |-
|
||||
envoy
|
||||
istio
|
||||
.git/modules
|
||||
key: ${{ runner.os }}-submodules-new-${{ github.run_id }}
|
||||
restore-keys: ${{ runner.os }}-submodules-new
|
||||
key: ${{ runner.os }}-submodules-cache-${{ github.run_id }}
|
||||
restore-keys: ${{ runner.os }}-submodules-cache
|
||||
|
||||
- run: git stash # restore patch
|
||||
|
||||
@@ -82,11 +80,9 @@ jobs:
|
||||
uses: actions/cache@v3
|
||||
with:
|
||||
path: |-
|
||||
envoy
|
||||
istio
|
||||
.git/modules
|
||||
key: ${{ runner.os }}-submodules-new-${{ github.run_id }}
|
||||
restore-keys: ${{ runner.os }}-submodules-new
|
||||
key: ${{ runner.os }}-submodules-cache-${{ github.run_id }}
|
||||
restore-keys: ${{ runner.os }}-submodules-cache
|
||||
|
||||
- run: git stash # restore patch
|
||||
|
||||
@@ -130,11 +126,9 @@ jobs:
|
||||
uses: actions/cache@v3
|
||||
with:
|
||||
path: |-
|
||||
envoy
|
||||
istio
|
||||
.git/modules
|
||||
key: ${{ runner.os }}-submodules-new-${{ github.run_id }}
|
||||
restore-keys: ${{ runner.os }}-submodules-new
|
||||
key: ${{ runner.os }}-submodules-cache-${{ github.run_id }}
|
||||
restore-keys: ${{ runner.os }}-submodules-cache
|
||||
|
||||
- run: git stash # restore patch
|
||||
|
||||
|
||||
30
.github/workflows/build-image-and-push.yaml
vendored
30
.github/workflows/build-image-and-push.yaml
vendored
@@ -20,6 +20,16 @@ jobs:
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Free Up GitHub Actions Ubuntu Runner Disk Space 🔧
|
||||
uses: jlumbroso/free-disk-space@main
|
||||
with:
|
||||
tool-cache: false
|
||||
android: true
|
||||
dotnet: true
|
||||
haskell: true
|
||||
large-packages: true
|
||||
swap-storage: true
|
||||
|
||||
- name: "Setup Go"
|
||||
uses: actions/setup-go@v3
|
||||
with:
|
||||
@@ -86,6 +96,16 @@ jobs:
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Free Up GitHub Actions Ubuntu Runner Disk Space 🔧
|
||||
uses: jlumbroso/free-disk-space@main
|
||||
with:
|
||||
tool-cache: false
|
||||
android: true
|
||||
dotnet: true
|
||||
haskell: true
|
||||
large-packages: true
|
||||
swap-storage: true
|
||||
|
||||
- name: "Setup Go"
|
||||
uses: actions/setup-go@v3
|
||||
with:
|
||||
@@ -153,6 +173,16 @@ jobs:
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Free Up GitHub Actions Ubuntu Runner Disk Space 🔧
|
||||
uses: jlumbroso/free-disk-space@main
|
||||
with:
|
||||
tool-cache: false
|
||||
android: true
|
||||
dotnet: true
|
||||
haskell: true
|
||||
large-packages: true
|
||||
swap-storage: true
|
||||
|
||||
- name: "Setup Go"
|
||||
uses: actions/setup-go@v3
|
||||
with:
|
||||
|
||||
@@ -26,6 +26,7 @@ header:
|
||||
- 'VERSION'
|
||||
- 'tools/'
|
||||
- 'test/README.md'
|
||||
- 'test/README_CN.md'
|
||||
- 'cmd/hgctl/config/testdata/config'
|
||||
- 'pkg/cmd/hgctl/manifests'
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
/api @johnlanni @CH3CHO
|
||||
/envoy @gengleilei @johnlanni
|
||||
/istio @SpecialYang @johnlanni
|
||||
/pkg @SpecialYang @johnlanni @CH3CHO @Xunzhuo
|
||||
/pkg @SpecialYang @johnlanni @CH3CHO
|
||||
/plugins @johnlanni @WeixinX
|
||||
/registry @NameHaibinZhang @2456868764 @johnlanni
|
||||
/test @Xunzhuo @2456868764 @CH3CHO
|
||||
|
||||
@@ -73,6 +73,7 @@
|
||||
* [分支定义](#分支定义)
|
||||
* [提交规则](#提交规则)
|
||||
* [PR说明](#PR说明)
|
||||
* [开发前准备](#开发前准备)
|
||||
|
||||
### 工作区准备
|
||||
|
||||
@@ -168,6 +169,12 @@ git config --get user.email
|
||||
|
||||
PR 是更改 Higress 项目文件的唯一方法。为了帮助审查人更好地理解你的目的,PR 描述不能太详细。我们鼓励贡献者遵循 [PR 模板](./.github/PULL_REQUEST_TEMPLATE.md) 来完成拉取请求。
|
||||
|
||||
### 开发前准备
|
||||
|
||||
```shell
|
||||
make prebuild && go mod tidy
|
||||
```
|
||||
|
||||
## 测试用例贡献
|
||||
|
||||
任何测试用例都会受到欢迎。目前,Higress 功能测试用例是高优先级的。
|
||||
|
||||
@@ -169,6 +169,12 @@ No matter commit message, or commit content, we do take more emphasis on code re
|
||||
|
||||
PR is the only way to make change to Higress project files. To help reviewers better get your purpose, PR description could not be too detailed. We encourage contributors to follow the [PR template](./.github/PULL_REQUEST_TEMPLATE.md) to finish the pull request.
|
||||
|
||||
### Pre-development preparation
|
||||
|
||||
```shell
|
||||
make prebuild && go mod tidy
|
||||
```
|
||||
|
||||
## Test case contribution
|
||||
|
||||
Any test case would be welcomed. Currently, Higress function test cases are high priority.
|
||||
|
||||
@@ -15,7 +15,7 @@ GO_LDFLAGS += -X $(VERSION_PACKAGE).higressVersion=$(shell cat VERSION) \
|
||||
|
||||
GO ?= go
|
||||
|
||||
export GOPROXY ?= https://proxy.golang.com.cn,direct
|
||||
export GOPROXY ?= https://proxy.golang.org,direct
|
||||
|
||||
TARGET_ARCH ?= amd64
|
||||
|
||||
@@ -138,11 +138,11 @@ export ENVOY_TAR_PATH:=/home/package/envoy.tar.gz
|
||||
|
||||
external/package/envoy-amd64.tar.gz:
|
||||
# cd external/proxy; BUILD_WITH_CONTAINER=1 make test_release
|
||||
cd external/package; wget "https://github.com/alibaba/higress/releases/download/v1.3.2/envoy-amd64.tar.gz"
|
||||
cd external/package; wget -O envoy-amd64.tar.gz "https://github.com/alibaba/higress/releases/download/v1.3.4-rc.1/envoy-symbol-amd64.tar.gz"
|
||||
|
||||
external/package/envoy-arm64.tar.gz:
|
||||
# cd external/proxy; BUILD_WITH_CONTAINER=1 make test_release
|
||||
cd external/package; wget "https://github.com/alibaba/higress/releases/download/v1.3.2/envoy-arm64.tar.gz"
|
||||
cd external/package; wget -O envoy-arm64.tar.gz "https://github.com/alibaba/higress/releases/download/v1.3.4-rc.1/envoy-symbol-arm64.tar.gz"
|
||||
|
||||
build-pilot:
|
||||
cd external/istio; rm -rf out/linux_amd64; GOOS_LOCAL=linux TARGET_OS=linux TARGET_ARCH=amd64 BUILD_WITH_CONTAINER=1 make build-linux
|
||||
@@ -154,13 +154,13 @@ build-pilot-local:
|
||||
build-gateway: prebuild external/package/envoy-amd64.tar.gz external/package/envoy-arm64.tar.gz build-pilot
|
||||
cd external/istio; BUILD_WITH_CONTAINER=1 BUILDX_PLATFORM=true DOCKER_BUILD_VARIANTS=default DOCKER_TARGETS="docker.proxyv2" make docker
|
||||
|
||||
build-gateway-local: prebuild external/package/envoy-amd64.tar.gz external/package/envoy-arm64.tar.gz build-pilot
|
||||
build-gateway-local: prebuild external/package/envoy-amd64.tar.gz external/package/envoy-arm64.tar.gz
|
||||
cd external/istio; rm -rf out/linux_${GOARCH_LOCAL}; GOOS_LOCAL=linux TARGET_OS=linux BUILD_WITH_CONTAINER=1 BUILDX_PLATFORM=false DOCKER_BUILD_VARIANTS=default DOCKER_TARGETS="docker.proxyv2" make docker
|
||||
|
||||
build-istio: prebuild build-pilot
|
||||
cd external/istio; BUILD_WITH_CONTAINER=1 BUILDX_PLATFORM=true DOCKER_BUILD_VARIANTS=default DOCKER_TARGETS="docker.pilot" make docker
|
||||
|
||||
build-istio-local: prebuild build-pilot-local
|
||||
build-istio-local: prebuild
|
||||
cd external/istio; rm -rf out/linux_${GOARCH_LOCAL}; GOOS_LOCAL=linux TARGET_OS=linux BUILD_WITH_CONTAINER=1 BUILDX_PLATFORM=false DOCKER_BUILD_VARIANTS=default DOCKER_TARGETS="docker.pilot" make docker
|
||||
|
||||
build-wasmplugins:
|
||||
@@ -177,13 +177,13 @@ install: pre-install
|
||||
cd helm/higress; helm dependency build
|
||||
helm install higress helm/higress -n higress-system --create-namespace --set 'global.local=true'
|
||||
|
||||
ENVOY_LATEST_IMAGE_TAG ?= sha-53ff28c
|
||||
ISTIO_LATEST_IMAGE_TAG ?= sha-53ff28c
|
||||
ENVOY_LATEST_IMAGE_TAG ?= sha-a68dde0
|
||||
ISTIO_LATEST_IMAGE_TAG ?= sha-a68dde0
|
||||
|
||||
install-dev: pre-install
|
||||
helm install higress helm/core -n higress-system --create-namespace --set 'controller.tag=$(TAG)' --set 'gateway.replicas=1' --set 'pilot.tag=$(ISTIO_LATEST_IMAGE_TAG)' --set 'gateway.tag=$(ENVOY_LATEST_IMAGE_TAG)' --set 'global.local=true'
|
||||
install-dev-wasmplugin: build-wasmplugins pre-install
|
||||
helm install higress helm/core -n higress-system --create-namespace --set 'controller.tag=$(TAG)' --set 'gateway.replicas=1' --set 'pilot.tag=$(ISTIO_LATEST_IMAGE_TAG)' --set 'gateway.tag=$(ENVOY_LATEST_IMAGE_TAG)' --set 'global.local=true' --set 'global.volumeWasmPlugins=true'
|
||||
helm install higress helm/core -n higress-system --create-namespace --set 'controller.tag=$(TAG)' --set 'gateway.replicas=1' --set 'pilot.tag=$(ISTIO_LATEST_IMAGE_TAG)' --set 'gateway.tag=$(ENVOY_LATEST_IMAGE_TAG)' --set 'global.local=true' --set 'global.volumeWasmPlugins=true' --set 'global.onlyPushRouteCluster=false'
|
||||
|
||||
uninstall:
|
||||
helm uninstall higress -n higress-system
|
||||
@@ -233,14 +233,30 @@ include tools/lint.mk
|
||||
.PHONY: gateway-conformance-test
|
||||
gateway-conformance-test:
|
||||
|
||||
# higress-conformance-test-prepare prepares the environment for higress conformance tests.
|
||||
.PHONY: higress-conformance-test-prepare
|
||||
higress-conformance-test-prepare: $(tools/kind) delete-cluster create-cluster docker-build kube-load-image install-dev
|
||||
|
||||
# higress-conformance-test runs ingress api conformance tests.
|
||||
.PHONY: higress-conformance-test
|
||||
higress-conformance-test: $(tools/kind) delete-cluster create-cluster docker-build kube-load-image install-dev run-higress-e2e-test delete-cluster
|
||||
|
||||
# higress-conformance-test-clean cleans the environment for higress conformance tests.
|
||||
.PHONY: higress-conformance-test-clean
|
||||
higress-conformance-test-clean: $(tools/kind) delete-cluster
|
||||
|
||||
# higress-wasmplugin-test-prepare prepares the environment for higress wasmplugin tests.
|
||||
.PHONY: higress-wasmplugin-test-prepare
|
||||
higress-wasmplugin-test-prepare: $(tools/kind) delete-cluster create-cluster docker-build kube-load-image install-dev-wasmplugin
|
||||
|
||||
# higress-wasmplugin-test runs ingress wasmplugin tests.
|
||||
.PHONY: higress-wasmplugin-test
|
||||
higress-wasmplugin-test: $(tools/kind) delete-cluster create-cluster docker-build kube-load-image install-dev-wasmplugin run-higress-e2e-test-wasmplugin delete-cluster
|
||||
|
||||
# higress-wasmplugin-test-clean cleans the environment for higress wasmplugin tests.
|
||||
.PHONY: higress-wasmplugin-test-clean
|
||||
higress-wasmplugin-test-clean: $(tools/kind) delete-cluster
|
||||
|
||||
# create-cluster creates a kube cluster with kind.
|
||||
.PHONY: create-cluster
|
||||
create-cluster: $(tools/kind)
|
||||
@@ -270,6 +286,17 @@ kube-load-image: $(tools/kind) ## Install the Higress image to a kind cluster us
|
||||
tools/hack/kind-load-image.sh docker.io/alihigress/httpbin 1.0.2
|
||||
tools/hack/kind-load-image.sh docker.io/charlie1380/eureka-registry-provider v0.3.0
|
||||
tools/hack/kind-load-image.sh docker.io/bitinit/eureka latest
|
||||
|
||||
# run-higress-e2e-test-setup starts to setup ingress e2e tests.
|
||||
.PHONT: run-higress-e2e-test-setup
|
||||
run-higress-e2e-test-setup:
|
||||
@echo -e "\n\033[36mRunning higress conformance tests...\033[0m"
|
||||
@echo -e "\n\033[36mWaiting higress-controller to be ready...\033[0m\n"
|
||||
kubectl wait --timeout=10m -n higress-system deployment/higress-controller --for=condition=Available
|
||||
@echo -e "\n\033[36mWaiting higress-gateway to be ready...\033[0m\n"
|
||||
kubectl wait --timeout=10m -n higress-system deployment/higress-gateway --for=condition=Available
|
||||
go test -v -tags conformance ./test/e2e/e2e_test.go --ingress-class=higress --debug=true --test-area=setup
|
||||
|
||||
# run-higress-e2e-test starts to run ingress e2e tests.
|
||||
.PHONY: run-higress-e2e-test
|
||||
run-higress-e2e-test:
|
||||
@@ -278,9 +305,39 @@ run-higress-e2e-test:
|
||||
kubectl wait --timeout=10m -n higress-system deployment/higress-controller --for=condition=Available
|
||||
@echo -e "\n\033[36mWaiting higress-gateway to be ready...\033[0m\n"
|
||||
kubectl wait --timeout=10m -n higress-system deployment/higress-gateway --for=condition=Available
|
||||
go test -v -tags conformance ./test/e2e/e2e_test.go --ingress-class=higress --debug=true
|
||||
go test -v -tags conformance ./test/e2e/e2e_test.go --ingress-class=higress --debug=true --test-area=all
|
||||
|
||||
# run-higress-e2e-test starts to run ingress e2e tests.
|
||||
# run-higress-e2e-test-run starts to run ingress e2e conformance tests.
|
||||
.PHONY: run-higress-e2e-test-run
|
||||
run-higress-e2e-test-run:
|
||||
@echo -e "\n\033[36mRunning higress conformance tests...\033[0m"
|
||||
@echo -e "\n\033[36mWaiting higress-controller to be ready...\033[0m\n"
|
||||
kubectl wait --timeout=10m -n higress-system deployment/higress-controller --for=condition=Available
|
||||
@echo -e "\n\033[36mWaiting higress-gateway to be ready...\033[0m\n"
|
||||
kubectl wait --timeout=10m -n higress-system deployment/higress-gateway --for=condition=Available
|
||||
go test -v -tags conformance ./test/e2e/e2e_test.go --ingress-class=higress --debug=true --test-area=run
|
||||
|
||||
# run-higress-e2e-test-clean starts to clean ingress e2e tests.
|
||||
.PHONY: run-higress-e2e-test-clean
|
||||
run-higress-e2e-test-clean:
|
||||
@echo -e "\n\033[36mRunning higress conformance tests...\033[0m"
|
||||
@echo -e "\n\033[36mWaiting higress-controller to be ready...\033[0m\n"
|
||||
kubectl wait --timeout=10m -n higress-system deployment/higress-controller --for=condition=Available
|
||||
@echo -e "\n\033[36mWaiting higress-gateway to be ready...\033[0m\n"
|
||||
kubectl wait --timeout=10m -n higress-system deployment/higress-gateway --for=condition=Available
|
||||
go test -v -tags conformance ./test/e2e/e2e_test.go --ingress-class=higress --debug=true --test-area=clean
|
||||
|
||||
# run-higress-e2e-test-wasmplugin-setup starts to prepare ingress e2e tests.
|
||||
.PHONY: run-higress-e2e-test-wasmplugin-setup
|
||||
run-higress-e2e-test-wasmplugin-setup:
|
||||
@echo -e "\n\033[36mRunning higress conformance tests...\033[0m"
|
||||
@echo -e "\n\033[36mWaiting higress-controller to be ready...\033[0m\n"
|
||||
kubectl wait --timeout=10m -n higress-system deployment/higress-controller --for=condition=Available
|
||||
@echo -e "\n\033[36mWaiting higress-gateway to be ready...\033[0m\n"
|
||||
kubectl wait --timeout=10m -n higress-system deployment/higress-gateway --for=condition=Available
|
||||
go test -v -tags conformance ./test/e2e/e2e_test.go -isWasmPluginTest=true -wasmPluginType=$(PLUGIN_TYPE) -wasmPluginName=$(PLUGIN_NAME) --ingress-class=higress --debug=true --test-area=setup
|
||||
|
||||
# run-higress-e2e-test-wasmplugin starts to run ingress e2e tests.
|
||||
.PHONY: run-higress-e2e-test-wasmplugin
|
||||
run-higress-e2e-test-wasmplugin:
|
||||
@echo -e "\n\033[36mRunning higress conformance tests...\033[0m"
|
||||
@@ -288,4 +345,24 @@ run-higress-e2e-test-wasmplugin:
|
||||
kubectl wait --timeout=10m -n higress-system deployment/higress-controller --for=condition=Available
|
||||
@echo -e "\n\033[36mWaiting higress-gateway to be ready...\033[0m\n"
|
||||
kubectl wait --timeout=10m -n higress-system deployment/higress-gateway --for=condition=Available
|
||||
go test -v -tags conformance ./test/e2e/e2e_test.go -isWasmPluginTest=true -wasmPluginType=$(PLUGIN_TYPE) -wasmPluginName=$(PLUGIN_NAME) --ingress-class=higress --debug=true
|
||||
go test -v -tags conformance ./test/e2e/e2e_test.go -isWasmPluginTest=true -wasmPluginType=$(PLUGIN_TYPE) -wasmPluginName=$(PLUGIN_NAME) --ingress-class=higress --debug=true --test-area=all
|
||||
|
||||
# run-higress-e2e-test-wasmplugin-run starts to run ingress e2e conformance tests.
|
||||
.PHONY: run-higress-e2e-test-wasmplugin-run
|
||||
run-higress-e2e-test-wasmplugin-run:
|
||||
@echo -e "\n\033[36mRunning higress conformance tests...\033[0m"
|
||||
@echo -e "\n\033[36mWaiting higress-controller to be ready...\033[0m\n"
|
||||
kubectl wait --timeout=10m -n higress-system deployment/higress-controller --for=condition=Available
|
||||
@echo -e "\n\033[36mWaiting higress-gateway to be ready...\033[0m\n"
|
||||
kubectl wait --timeout=10m -n higress-system deployment/higress-gateway --for=condition=Available
|
||||
go test -v -tags conformance ./test/e2e/e2e_test.go -isWasmPluginTest=true -wasmPluginType=$(PLUGIN_TYPE) -wasmPluginName=$(PLUGIN_NAME) --ingress-class=higress --debug=true --test-area=run
|
||||
|
||||
# run-higress-e2e-test-wasmplugin-clean starts to clean ingress e2e tests.
|
||||
.PHONY: run-higress-e2e-test-wasmplugin-clean
|
||||
run-higress-e2e-test-wasmplugin-clean:
|
||||
@echo -e "\n\033[36mRunning higress conformance tests...\033[0m"
|
||||
@echo -e "\n\033[36mWaiting higress-controller to be ready...\033[0m\n"
|
||||
kubectl wait --timeout=10m -n higress-system deployment/higress-controller --for=condition=Available
|
||||
@echo -e "\n\033[36mWaiting higress-gateway to be ready...\033[0m\n"
|
||||
kubectl wait --timeout=10m -n higress-system deployment/higress-gateway --for=condition=Available
|
||||
go test -v -tags conformance ./test/e2e/e2e_test.go -isWasmPluginTest=true -wasmPluginType=$(PLUGIN_TYPE) -wasmPluginName=$(PLUGIN_NAME) --ingress-class=higress --debug=true --test-area=clean
|
||||
|
||||
@@ -2,7 +2,11 @@ codecov:
|
||||
require_ci_to_pass: yes
|
||||
coverage:
|
||||
status:
|
||||
patch: no
|
||||
patch:
|
||||
default:
|
||||
target: 50%
|
||||
threshold: 0%
|
||||
if_ci_failed: error # success, failure, error, ignore
|
||||
project:
|
||||
default:
|
||||
target: auto
|
||||
@@ -17,4 +21,4 @@ ignore:
|
||||
comment:
|
||||
layout: "reach,diff,flags,tree"
|
||||
behavior: default
|
||||
require_changes: no
|
||||
require_changes: no
|
||||
49
envoy/1.20/patches/envoy/20240110-fix-srds.patch
Normal file
49
envoy/1.20/patches/envoy/20240110-fix-srds.patch
Normal file
@@ -0,0 +1,49 @@
|
||||
diff -Naur envoy/source/common/router/BUILD envoy-new/source/common/router/BUILD
|
||||
--- envoy/source/common/router/BUILD 2024-01-10 20:10:14.505600746 +0800
|
||||
+++ envoy-new/source/common/router/BUILD 2024-01-10 20:07:25.960379955 +0800
|
||||
@@ -212,6 +212,7 @@
|
||||
"//envoy/router:rds_interface",
|
||||
"//envoy/router:scopes_interface",
|
||||
"//envoy/thread_local:thread_local_interface",
|
||||
+ "//source/common/http:header_utility_lib",
|
||||
"@envoy_api//envoy/config/route/v3:pkg_cc_proto",
|
||||
"@envoy_api//envoy/extensions/filters/network/http_connection_manager/v3:pkg_cc_proto",
|
||||
],
|
||||
diff -Naur envoy/source/common/router/scoped_config_impl.cc envoy-new/source/common/router/scoped_config_impl.cc
|
||||
--- envoy/source/common/router/scoped_config_impl.cc 2024-01-10 20:10:14.529600924 +0800
|
||||
+++ envoy-new/source/common/router/scoped_config_impl.cc 2024-01-10 20:09:50.161422411 +0800
|
||||
@@ -3,6 +3,8 @@
|
||||
#include "envoy/config/route/v3/scoped_route.pb.h"
|
||||
#include "envoy/extensions/filters/network/http_connection_manager/v3/http_connection_manager.pb.h"
|
||||
|
||||
+#include "source/common/http/header_utility.h"
|
||||
+
|
||||
namespace Envoy {
|
||||
namespace Router {
|
||||
|
||||
@@ -74,18 +76,20 @@
|
||||
HostValueExtractorImpl::computeFragment(const Http::HeaderMap& headers,
|
||||
const StreamInfo::StreamInfo&,
|
||||
ReComputeCbPtr& recompute) const {
|
||||
- auto fragment = computeFragment(headers);
|
||||
auto host = static_cast<const Http::RequestHeaderMap&>(headers).getHostValue();
|
||||
+ auto port_start = Http::HeaderUtility::getPortStart(host);
|
||||
+ if (port_start != absl::string_view::npos) {
|
||||
+ host = host.substr(0, port_start);
|
||||
+ }
|
||||
*recompute = [this, host, recompute]() mutable -> std::unique_ptr<ScopeKeyFragmentBase> {
|
||||
return reComputeHelper(std::string(host), recompute, 0);
|
||||
};
|
||||
- return fragment;
|
||||
+ return std::make_unique<StringKeyFragment>(host);
|
||||
}
|
||||
|
||||
std::unique_ptr<ScopeKeyFragmentBase>
|
||||
-HostValueExtractorImpl::computeFragment(const Http::HeaderMap& headers) const {
|
||||
- return std::make_unique<StringKeyFragment>(
|
||||
- static_cast<const Http::RequestHeaderMap&>(headers).getHostValue());
|
||||
+HostValueExtractorImpl::computeFragment(const Http::HeaderMap&) const {
|
||||
+ return nullptr;
|
||||
}
|
||||
|
||||
std::unique_ptr<ScopeKeyFragmentBase>
|
||||
65
envoy/1.20/patches/envoy/20240111-fix-srds-compute.patch
Normal file
65
envoy/1.20/patches/envoy/20240111-fix-srds-compute.patch
Normal file
@@ -0,0 +1,65 @@
|
||||
diff -Naur envoy/source/common/router/scoped_config_impl.cc envoy-new/source/common/router/scoped_config_impl.cc
|
||||
--- envoy/source/common/router/scoped_config_impl.cc 2024-01-11 16:23:55.407881263 +0800
|
||||
+++ envoy-new/source/common/router/scoped_config_impl.cc 2024-01-11 16:23:42.311786814 +0800
|
||||
@@ -53,21 +53,26 @@
|
||||
}
|
||||
|
||||
std::unique_ptr<ScopeKeyFragmentBase>
|
||||
-HostValueExtractorImpl::reComputeHelper(const std::string& host, ReComputeCbPtr& next_recompute,
|
||||
+HostValueExtractorImpl::reComputeHelper(const std::string& host,
|
||||
+ ReComputeCbWeakPtr& weak_next_recompute,
|
||||
uint32_t recompute_seq) const {
|
||||
if (recompute_seq == max_recompute_num_) {
|
||||
ENVOY_LOG_MISC(warn,
|
||||
"recompute host fragment failed, maximum number of recalculations exceeded");
|
||||
return nullptr;
|
||||
}
|
||||
+ auto next_recompute = weak_next_recompute.lock();
|
||||
+ if (!next_recompute) {
|
||||
+ return nullptr;
|
||||
+ }
|
||||
if (host == "*") {
|
||||
*next_recompute = nullptr;
|
||||
return nullptr;
|
||||
}
|
||||
auto masked_host = maskFirstDNSLabel(host);
|
||||
*next_recompute = [this, masked_host, recompute_seq,
|
||||
- next_recompute]() mutable -> std::unique_ptr<ScopeKeyFragmentBase> {
|
||||
- return reComputeHelper(masked_host, next_recompute, recompute_seq + 1);
|
||||
+ weak_next_recompute]() mutable -> std::unique_ptr<ScopeKeyFragmentBase> {
|
||||
+ return reComputeHelper(masked_host, weak_next_recompute, recompute_seq + 1);
|
||||
};
|
||||
return std::make_unique<StringKeyFragment>(masked_host);
|
||||
}
|
||||
@@ -81,8 +86,9 @@
|
||||
if (port_start != absl::string_view::npos) {
|
||||
host = host.substr(0, port_start);
|
||||
}
|
||||
- *recompute = [this, host, recompute]() mutable -> std::unique_ptr<ScopeKeyFragmentBase> {
|
||||
- return reComputeHelper(std::string(host), recompute, 0);
|
||||
+ *recompute = [this, host, weak_recompute = ReComputeCbWeakPtr(recompute)]() mutable
|
||||
+ -> std::unique_ptr<ScopeKeyFragmentBase> {
|
||||
+ return reComputeHelper(std::string(host), weak_recompute, 0);
|
||||
};
|
||||
return std::make_unique<StringKeyFragment>(host);
|
||||
}
|
||||
diff -Naur envoy/source/common/router/scoped_config_impl.h envoy-new/source/common/router/scoped_config_impl.h
|
||||
--- envoy/source/common/router/scoped_config_impl.h 2024-01-11 16:23:55.407881263 +0800
|
||||
+++ envoy-new/source/common/router/scoped_config_impl.h 2024-01-11 16:23:42.311786814 +0800
|
||||
@@ -25,6 +25,7 @@
|
||||
#if defined(ALIMESH)
|
||||
using ReComputeCb = std::function<std::unique_ptr<ScopeKeyFragmentBase>()>;
|
||||
using ReComputeCbPtr = std::shared_ptr<ReComputeCb>;
|
||||
+using ReComputeCbWeakPtr = std::weak_ptr<ReComputeCb>;
|
||||
#endif
|
||||
|
||||
/**
|
||||
@@ -83,7 +84,7 @@
|
||||
|
||||
private:
|
||||
std::unique_ptr<ScopeKeyFragmentBase> reComputeHelper(const std::string& host,
|
||||
- ReComputeCbPtr& next_recompute,
|
||||
+ ReComputeCbWeakPtr& weak_next_recompute,
|
||||
uint32_t recompute_seq) const;
|
||||
|
||||
static constexpr uint32_t DefaultMaxRecomputeNum = 100;
|
||||
1445
envoy/1.20/patches/envoy/20240116-fix-cve.patch
Normal file
1445
envoy/1.20/patches/envoy/20240116-fix-cve.patch
Normal file
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
2
go.mod
2
go.mod
@@ -15,7 +15,7 @@ replace github.com/imdario/mergo => github.com/imdario/mergo v0.3.5
|
||||
|
||||
require (
|
||||
github.com/AlecAivazis/survey/v2 v2.3.6
|
||||
github.com/agiledragon/gomonkey/v2 v2.9.0
|
||||
github.com/agiledragon/gomonkey/v2 v2.11.0
|
||||
github.com/avast/retry-go/v4 v4.3.4
|
||||
github.com/compose-spec/compose-go v1.8.2
|
||||
github.com/docker/cli v20.10.20+incompatible
|
||||
|
||||
4
go.sum
4
go.sum
@@ -160,8 +160,8 @@ github.com/VividCortex/ewma v1.1.1/go.mod h1:2Tkkvm3sRDVXaiyucHiACn4cqf7DpdyLvmx
|
||||
github.com/VividCortex/gohistogram v1.0.0/go.mod h1:Pf5mBqqDxYaXu3hDrrU+w6nw50o/4+TcAqDqk/vUH7g=
|
||||
github.com/afex/hystrix-go v0.0.0-20180502004556-fa1af6a1f4f5/go.mod h1:SkGFH1ia65gfNATL8TAiHDNxPzPdmEL5uirI2Uyuz6c=
|
||||
github.com/agext/levenshtein v1.2.1/go.mod h1:JEDfjyjHDjOF/1e4FlBE/PkbqA9OfWu2ki2W0IB5558=
|
||||
github.com/agiledragon/gomonkey/v2 v2.9.0 h1:PDiKKybR596O6FHW+RVSG0Z7uGCBNbmbUXh3uCNQ7Hc=
|
||||
github.com/agiledragon/gomonkey/v2 v2.9.0/go.mod h1:ap1AmDzcVOAz1YpeJ3TCzIgstoaWLA6jbbgxfB4w2iY=
|
||||
github.com/agiledragon/gomonkey/v2 v2.11.0 h1:5oxSgA+tC1xuGsrIorR+sYiziYltmJyEZ9qA25b6l5U=
|
||||
github.com/agiledragon/gomonkey/v2 v2.11.0/go.mod h1:ap1AmDzcVOAz1YpeJ3TCzIgstoaWLA6jbbgxfB4w2iY=
|
||||
github.com/agl/ed25519 v0.0.0-20170116200512-5312a6153412/go.mod h1:WPjqKcmVOxf0XSf3YxCJs6N6AOSrOx3obionmG7T0y0=
|
||||
github.com/ahmetb/gen-crd-api-reference-docs v0.3.0/go.mod h1:TdjdkYhlOifCQWPs1UdTma97kQQMozf5h26hTuG70u8=
|
||||
github.com/alecthomas/jsonschema v0.0.0-20180308105923-f2c93856175a/go.mod h1:qpebaTNSsyUn5rPSJMsfqEtDw71TTggXM6stUDI16HA=
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
apiVersion: v2
|
||||
appVersion: 1.3.3
|
||||
appVersion: 1.3.4
|
||||
description: Helm chart for deploying higress gateways
|
||||
icon: https://higress.io/img/higress_logo_small.png
|
||||
home: http://higress.io/
|
||||
@@ -10,4 +10,4 @@ name: higress-core
|
||||
sources:
|
||||
- http://github.com/alibaba/higress
|
||||
type: application
|
||||
version: 1.3.3
|
||||
version: 1.3.4
|
||||
|
||||
@@ -154,6 +154,11 @@ spec:
|
||||
type: array
|
||||
httpPath:
|
||||
type: string
|
||||
paramFromEntireBody:
|
||||
properties:
|
||||
paramType:
|
||||
type: string
|
||||
type: object
|
||||
params:
|
||||
items:
|
||||
properties:
|
||||
|
||||
@@ -70,6 +70,10 @@ spec:
|
||||
periodSeconds: 3
|
||||
timeoutSeconds: 5
|
||||
env:
|
||||
- name: DEFAULT_UPSTREAM_CONCURRENCY_THRESHOLD
|
||||
value: "{{ .Values.global.defaultUpstreamConcurrencyThreshold }}"
|
||||
- name: ISTIO_GPRC_MAXRECVMSGSIZE
|
||||
value: "{{ .Values.global.xdsMaxRecvMsgSize }}"
|
||||
- name: ENBALE_SCOPED_RDS
|
||||
value: "{{ .Values.global.enableSRDS }}"
|
||||
- name: ON_DEMAND_RDS
|
||||
|
||||
@@ -175,15 +175,15 @@ spec:
|
||||
protocol: TCP
|
||||
{{- end }}
|
||||
readinessProbe:
|
||||
failureThreshold: 30
|
||||
failureThreshold: {{ .Values.gateway.readinessFailureThreshold }}
|
||||
httpGet:
|
||||
path: /healthz/ready
|
||||
port: 15021
|
||||
scheme: HTTP
|
||||
initialDelaySeconds: 1
|
||||
periodSeconds: 2
|
||||
successThreshold: 1
|
||||
timeoutSeconds: 3
|
||||
initialDelaySeconds: {{ .Values.gateway.readinessInitialDelaySeconds }}
|
||||
periodSeconds: {{ .Values.gateway.readinessPeriodSeconds }}
|
||||
successThreshold: {{ .Values.gateway.readinessSuccessThreshold }}
|
||||
timeoutSeconds: {{ .Values.gateway.readinessTimeoutSeconds }}
|
||||
{{- if not (or .Values.global.local .Values.global.kind) }}
|
||||
resources:
|
||||
{{- toYaml .Values.gateway.resources | nindent 12 }}
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
revision: ""
|
||||
global:
|
||||
enableSRDS: false
|
||||
onDemandRDS: true
|
||||
hostRDSMergeSubset: true
|
||||
xdsMaxRecvMsgSize: 104857600
|
||||
defaultUpstreamConcurrencyThreshold: 10000
|
||||
enableSRDS: true
|
||||
onDemandRDS: false
|
||||
hostRDSMergeSubset: false
|
||||
onlyPushRouteCluster: true
|
||||
# IngressClass filters which ingress resources the higress controller watches.
|
||||
# The default ingress class is higress.
|
||||
@@ -13,7 +15,7 @@ global:
|
||||
# resources in the k8s cluster.
|
||||
ingressClass: "higress"
|
||||
watchNamespace: ""
|
||||
disableAlpnH2: true
|
||||
disableAlpnH2: false
|
||||
enableStatus: true
|
||||
# whether to use autoscaling/v2 template for HPA settings
|
||||
# for internal usage only, not to be configured by users.
|
||||
@@ -151,12 +153,18 @@ global:
|
||||
# The number of successive failed probes before indicating readiness failure.
|
||||
readinessFailureThreshold: 30
|
||||
|
||||
# The number of successive successed probes before indicating readiness success.
|
||||
readinessSuccessThreshold: 30
|
||||
|
||||
# The initial delay for readiness probes in seconds.
|
||||
readinessInitialDelaySeconds: 1
|
||||
|
||||
# The period between readiness probes.
|
||||
readinessPeriodSeconds: 2
|
||||
|
||||
# The readiness timeout seconds
|
||||
readinessTimeoutSeconds: 3
|
||||
|
||||
# Resources for the sidecar.
|
||||
resources:
|
||||
requests:
|
||||
@@ -373,6 +381,21 @@ gateway:
|
||||
replicas: 2
|
||||
image: gateway
|
||||
|
||||
# The number of successive failed probes before indicating readiness failure.
|
||||
readinessFailureThreshold: 30
|
||||
|
||||
# The number of successive successed probes before indicating readiness success.
|
||||
readinessSuccessThreshold: 1
|
||||
|
||||
# The initial delay for readiness probes in seconds.
|
||||
readinessInitialDelaySeconds: 1
|
||||
|
||||
# The period between readiness probes.
|
||||
readinessPeriodSeconds: 2
|
||||
|
||||
# The readiness timeout seconds
|
||||
readinessTimeoutSeconds: 3
|
||||
|
||||
hub: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress
|
||||
tag: ""
|
||||
# revision declares which revision this gateway is a part of
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
dependencies:
|
||||
- name: higress-core
|
||||
repository: file://../core
|
||||
version: 1.3.3
|
||||
version: 1.3.4
|
||||
- name: higress-console
|
||||
repository: https://higress.io/helm-charts/
|
||||
version: 1.3.1
|
||||
digest: sha256:585666df5da403450c5e586a71388bc0d029354b1100b20a50616f56711fa171
|
||||
generated: "2024-01-08T21:40:10.446936+08:00"
|
||||
version: 1.3.2
|
||||
digest: sha256:1a5849d5ead79151e42ce419a96669f6691567e6fbf155643a928e6b62c3daa1
|
||||
generated: "2024-02-20T11:54:00.898489+08:00"
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
apiVersion: v2
|
||||
appVersion: 1.3.3
|
||||
appVersion: 1.3.4
|
||||
description: Helm chart for deploying Higress gateways
|
||||
icon: https://higress.io/img/higress_logo_small.png
|
||||
home: http://higress.io/
|
||||
@@ -12,9 +12,9 @@ sources:
|
||||
dependencies:
|
||||
- name: higress-core
|
||||
repository: "file://../core"
|
||||
version: 1.3.3
|
||||
version: 1.3.4
|
||||
- name: higress-console
|
||||
repository: "https://higress.io/helm-charts/"
|
||||
version: 1.3.1
|
||||
version: 1.3.2
|
||||
type: application
|
||||
version: 1.3.3
|
||||
version: 1.3.4
|
||||
|
||||
373
istio/1.12/patches/istio/20240115-optimize-rds-cache.patch
Normal file
373
istio/1.12/patches/istio/20240115-optimize-rds-cache.patch
Normal file
@@ -0,0 +1,373 @@
|
||||
diff -Naur istio/pilot/pkg/model/push_context.go istio-new/pilot/pkg/model/push_context.go
|
||||
--- istio/pilot/pkg/model/push_context.go 2024-01-15 20:46:45.000000000 +0800
|
||||
+++ istio-new/pilot/pkg/model/push_context.go 2024-01-15 19:20:45.000000000 +0800
|
||||
@@ -96,6 +96,9 @@
|
||||
publicByGateway map[string][]config.Config
|
||||
// root vs namespace/name ->delegate vs virtualservice gvk/namespace/name
|
||||
delegates map[ConfigKey][]ConfigKey
|
||||
+ // Added by ingress
|
||||
+ byHost map[string][]config.Config
|
||||
+ // End added by ingress
|
||||
}
|
||||
|
||||
func newVirtualServiceIndex() virtualServiceIndex {
|
||||
@@ -104,6 +107,9 @@
|
||||
privateByNamespaceAndGateway: map[string]map[string][]config.Config{},
|
||||
exportedToNamespaceByGateway: map[string]map[string][]config.Config{},
|
||||
delegates: map[ConfigKey][]ConfigKey{},
|
||||
+ // Added by ingress
|
||||
+ byHost: map[string][]config.Config{},
|
||||
+ // End added by ingress
|
||||
}
|
||||
}
|
||||
|
||||
@@ -857,6 +863,13 @@
|
||||
return res
|
||||
}
|
||||
|
||||
+// Added by ingress
|
||||
+func (ps *PushContext) VirtualServicesForHost(proxy *Proxy, host string) []config.Config {
|
||||
+ return ps.virtualServiceIndex.byHost[host]
|
||||
+}
|
||||
+
|
||||
+// End added by ingress
|
||||
+
|
||||
// DelegateVirtualServicesConfigKey lists all the delegate virtual services configkeys associated with the provided virtual services
|
||||
func (ps *PushContext) DelegateVirtualServicesConfigKey(vses []config.Config) []ConfigKey {
|
||||
var out []ConfigKey
|
||||
@@ -1468,6 +1481,11 @@
|
||||
for _, virtualService := range vservices {
|
||||
ns := virtualService.Namespace
|
||||
rule := virtualService.Spec.(*networking.VirtualService)
|
||||
+ // Added by ingress
|
||||
+ for _, host := range rule.Hosts {
|
||||
+ ps.virtualServiceIndex.byHost[host] = append(ps.virtualServiceIndex.byHost[host], virtualService)
|
||||
+ }
|
||||
+ // End added by ingress
|
||||
gwNames := getGatewayNames(rule)
|
||||
if len(rule.ExportTo) == 0 {
|
||||
// No exportTo in virtualService. Use the global default
|
||||
diff -Naur istio/pilot/pkg/networking/core/v1alpha3/gateway.go istio-new/pilot/pkg/networking/core/v1alpha3/gateway.go
|
||||
--- istio/pilot/pkg/networking/core/v1alpha3/gateway.go 2024-01-15 20:46:45.000000000 +0800
|
||||
+++ istio-new/pilot/pkg/networking/core/v1alpha3/gateway.go 2024-01-15 20:04:05.000000000 +0800
|
||||
@@ -28,6 +28,7 @@
|
||||
route "github.com/envoyproxy/go-control-plane/envoy/config/route/v3"
|
||||
hcm "github.com/envoyproxy/go-control-plane/envoy/extensions/filters/network/http_connection_manager/v3"
|
||||
tls "github.com/envoyproxy/go-control-plane/envoy/extensions/transport_sockets/tls/v3"
|
||||
+ discovery "github.com/envoyproxy/go-control-plane/envoy/service/discovery/v3"
|
||||
"github.com/hashicorp/go-multierror"
|
||||
|
||||
meshconfig "istio.io/api/mesh/v1alpha1"
|
||||
@@ -35,6 +36,7 @@
|
||||
"istio.io/istio/pilot/pkg/features"
|
||||
"istio.io/istio/pilot/pkg/model"
|
||||
istionetworking "istio.io/istio/pilot/pkg/networking"
|
||||
+ "istio.io/istio/pilot/pkg/networking/core/v1alpha3/envoyfilter"
|
||||
"istio.io/istio/pilot/pkg/networking/core/v1alpha3/extension"
|
||||
"istio.io/istio/pilot/pkg/networking/core/v1alpha3/mseingress"
|
||||
istio_route "istio.io/istio/pilot/pkg/networking/core/v1alpha3/route"
|
||||
@@ -423,8 +425,15 @@
|
||||
gatewayName string
|
||||
}
|
||||
|
||||
-func (configgen *ConfigGeneratorImpl) buildHostRDSConfig(node *model.Proxy, push *model.PushContext,
|
||||
- routeName string) *route.RouteConfiguration {
|
||||
+func (configgen *ConfigGeneratorImpl) buildHostRDSConfig(
|
||||
+ node *model.Proxy,
|
||||
+ req *model.PushRequest,
|
||||
+ routeName string,
|
||||
+ vsCache map[int][]virtualServiceContext,
|
||||
+ efw *model.EnvoyFilterWrapper,
|
||||
+ efKeys []string,
|
||||
+) (*discovery.Resource, bool) {
|
||||
+ push := req.Push
|
||||
var (
|
||||
hostRDSPort string
|
||||
hostRDSHost string
|
||||
@@ -432,7 +441,7 @@
|
||||
portAndHost := strings.SplitN(strings.TrimPrefix(routeName, constants.HigressHostRDSNamePrefix), ".", 2)
|
||||
if len(portAndHost) != 2 {
|
||||
log.Errorf("Invalid route %s when using Higress hostRDS", routeName)
|
||||
- return nil
|
||||
+ return nil, false
|
||||
}
|
||||
hostRDSPort = portAndHost[0]
|
||||
hostRDSHost = portAndHost[1]
|
||||
@@ -441,10 +450,24 @@
|
||||
rdsPort, err := strconv.Atoi(hostRDSPort)
|
||||
if err != nil {
|
||||
log.Errorf("Invalid port %s of route %s when using Higress hostRDS", hostRDSPort, routeName)
|
||||
- return nil
|
||||
+ return nil, false
|
||||
+ }
|
||||
+
|
||||
+ routeCache := &istio_route.Cache{
|
||||
+ RouteName: routeName,
|
||||
+ ProxyVersion: node.Metadata.IstioVersion,
|
||||
+ ListenerPort: rdsPort,
|
||||
+ // Use same host vs to cache, although the cache can be cleared when the port is different, this can be accepted
|
||||
+ VirtualServices: push.VirtualServicesForHost(node, hostRDSHost),
|
||||
+ EnvoyFilterKeys: efKeys,
|
||||
+ }
|
||||
+
|
||||
+ resource, exist := configgen.Cache.Get(routeCache)
|
||||
+ if exist {
|
||||
+ return resource, true
|
||||
}
|
||||
+
|
||||
listenerPort := uint32(rdsPort)
|
||||
- globalHTTPFilters := mseingress.ExtractGlobalHTTPFilters(node, push)
|
||||
|
||||
isH3DiscoveryNeeded := false
|
||||
|
||||
@@ -457,9 +480,9 @@
|
||||
break
|
||||
}
|
||||
}
|
||||
-
|
||||
gatewayRoutes := make(map[string]map[string][]*route.Route)
|
||||
gatewayVirtualServices := make(map[string][]config.Config)
|
||||
+ var listenerVirtualServices []virtualServiceContext
|
||||
var selectedVirtualServices []virtualServiceContext
|
||||
var vHost *route.VirtualHost
|
||||
serverIterator := func(mergedServers map[model.ServerPort]*model.MergedServers) {
|
||||
@@ -478,31 +501,8 @@
|
||||
gatewayVirtualServices[gatewayName] = virtualServices
|
||||
}
|
||||
for _, virtualService := range virtualServices {
|
||||
- hostMatch := false
|
||||
- var selectHost string
|
||||
- virtualServiceHosts := host.NewNames(virtualService.Spec.(*networking.VirtualService).Hosts)
|
||||
- for _, hostname := range virtualServiceHosts {
|
||||
- // exact match
|
||||
- if hostname == host.Name(hostRDSHost) {
|
||||
- hostMatch = true
|
||||
- selectHost = hostRDSHost
|
||||
- break
|
||||
- }
|
||||
- if features.HostRDSMergeSubset {
|
||||
- // subset match
|
||||
- if host.Name(hostRDSHost).SubsetOf(hostname) {
|
||||
- hostMatch = true
|
||||
- selectHost = string(hostname)
|
||||
- }
|
||||
- }
|
||||
- }
|
||||
- if !hostMatch {
|
||||
- continue
|
||||
- }
|
||||
- copiedVS := virtualService.DeepCopy()
|
||||
- copiedVS.Spec.(*networking.VirtualService).Hosts = []string{selectHost}
|
||||
- selectedVirtualServices = append(selectedVirtualServices, virtualServiceContext{
|
||||
- virtualService: copiedVS,
|
||||
+ listenerVirtualServices = append(listenerVirtualServices, virtualServiceContext{
|
||||
+ virtualService: virtualService,
|
||||
server: server,
|
||||
gatewayName: gatewayName,
|
||||
})
|
||||
@@ -510,15 +510,63 @@
|
||||
}
|
||||
}
|
||||
}
|
||||
- serverIterator(merged.MergedServers)
|
||||
- serverIterator(merged.MergedQUICTransportServers)
|
||||
- // Sort by subset
|
||||
- // before: ["*.abc.com", "*.com", "www.abc.com"]
|
||||
- // after: ["www.abc.com", "*.abc.com", "*.com"]
|
||||
- sort.SliceStable(selectedVirtualServices, func(i, j int) bool {
|
||||
- return host.Name(selectedVirtualServices[i].virtualService.Spec.(*networking.VirtualService).Hosts[0]).SubsetOf(
|
||||
- host.Name(selectedVirtualServices[j].virtualService.Spec.(*networking.VirtualService).Hosts[0]))
|
||||
- })
|
||||
+ var vsExists bool
|
||||
+ if listenerVirtualServices, vsExists = vsCache[rdsPort]; !vsExists {
|
||||
+ serverIterator(merged.MergedServers)
|
||||
+ serverIterator(merged.MergedQUICTransportServers)
|
||||
+ vsCache[rdsPort] = listenerVirtualServices
|
||||
+ }
|
||||
+ for _, vsCtx := range listenerVirtualServices {
|
||||
+ virtualService := vsCtx.virtualService
|
||||
+ hostMatch := false
|
||||
+ var selectHost string
|
||||
+ for _, hostname := range virtualService.Spec.(*networking.VirtualService).Hosts {
|
||||
+ // exact match
|
||||
+ if hostname == hostRDSHost {
|
||||
+ hostMatch = true
|
||||
+ selectHost = hostRDSHost
|
||||
+ break
|
||||
+ }
|
||||
+ if features.HostRDSMergeSubset {
|
||||
+ // subset match
|
||||
+ if host.Name(hostRDSHost).SubsetOf(host.Name(hostname)) {
|
||||
+ hostMatch = true
|
||||
+ selectHost = hostname
|
||||
+ }
|
||||
+ }
|
||||
+ }
|
||||
+ if !hostMatch {
|
||||
+ continue
|
||||
+ }
|
||||
+ if len(virtualService.Spec.(*networking.VirtualService).Hosts) > 1 {
|
||||
+ copiedVS := &networking.VirtualService{}
|
||||
+ copiedVS = virtualService.Spec.(*networking.VirtualService)
|
||||
+ copiedVS.Hosts = []string{selectHost}
|
||||
+ selectedVirtualServices = append(selectedVirtualServices, virtualServiceContext{
|
||||
+ virtualService: config.Config{
|
||||
+ Meta: virtualService.Meta,
|
||||
+ Spec: copiedVS,
|
||||
+ Status: virtualService.Status,
|
||||
+ },
|
||||
+ server: vsCtx.server,
|
||||
+ gatewayName: vsCtx.gatewayName,
|
||||
+ })
|
||||
+ } else {
|
||||
+ selectedVirtualServices = append(selectedVirtualServices, vsCtx)
|
||||
+ }
|
||||
+ }
|
||||
+ if features.HostRDSMergeSubset {
|
||||
+ // Sort by subset
|
||||
+ // before: ["*.abc.com", "*.com", "www.abc.com"]
|
||||
+ // after: ["www.abc.com", "*.abc.com", "*.com"]
|
||||
+ sort.SliceStable(selectedVirtualServices, func(i, j int) bool {
|
||||
+ return host.Name(selectedVirtualServices[i].virtualService.Spec.(*networking.VirtualService).Hosts[0]).SubsetOf(
|
||||
+ host.Name(selectedVirtualServices[j].virtualService.Spec.(*networking.VirtualService).Hosts[0]))
|
||||
+ })
|
||||
+ }
|
||||
+
|
||||
+ globalHTTPFilters := mseingress.ExtractGlobalHTTPFilters(node, push)
|
||||
+
|
||||
port := int(listenerPort)
|
||||
for _, ctx := range selectedVirtualServices {
|
||||
virtualService := ctx.virtualService
|
||||
@@ -605,25 +653,42 @@
|
||||
ValidateClusters: proto.BoolFalse,
|
||||
}
|
||||
|
||||
- return routeCfg
|
||||
+ routeCfg = envoyfilter.ApplyRouteConfigurationPatches(networking.EnvoyFilter_GATEWAY, node, efw, routeCfg)
|
||||
+ resource = &discovery.Resource{
|
||||
+ Name: routeName,
|
||||
+ Resource: util.MessageToAny(routeCfg),
|
||||
+ }
|
||||
+
|
||||
+ if features.EnableRDSCaching {
|
||||
+ configgen.Cache.Add(routeCache, req, resource)
|
||||
+ }
|
||||
+
|
||||
+ return resource, false
|
||||
}
|
||||
|
||||
// End added by ingress
|
||||
|
||||
-func (configgen *ConfigGeneratorImpl) buildGatewayHTTPRouteConfig(node *model.Proxy, push *model.PushContext,
|
||||
- routeName string) *route.RouteConfiguration {
|
||||
+// Modifed by ingress
|
||||
+func (configgen *ConfigGeneratorImpl) buildGatewayHTTPRouteConfig(
|
||||
+ node *model.Proxy,
|
||||
+ req *model.PushRequest,
|
||||
+ routeName string,
|
||||
+ vsCache map[int][]virtualServiceContext,
|
||||
+ efw *model.EnvoyFilterWrapper,
|
||||
+ efKeys []string,
|
||||
+) (*discovery.Resource, bool) {
|
||||
if node.MergedGateway == nil {
|
||||
log.Warnf("buildGatewayRoutes: no gateways for router %v", node.ID)
|
||||
- return &route.RouteConfiguration{
|
||||
- Name: routeName,
|
||||
- VirtualHosts: []*route.VirtualHost{},
|
||||
- ValidateClusters: proto.BoolFalse,
|
||||
- }
|
||||
+ return nil, false
|
||||
}
|
||||
-
|
||||
// Added by ingress
|
||||
+ push := req.Push
|
||||
if strings.HasPrefix(routeName, constants.HigressHostRDSNamePrefix) {
|
||||
- return configgen.buildHostRDSConfig(node, push, routeName)
|
||||
+ resource, cacheHit := configgen.buildHostRDSConfig(node, req, routeName, vsCache, efw, efKeys)
|
||||
+ if resource == nil {
|
||||
+ return nil, false
|
||||
+ }
|
||||
+ return resource, cacheHit
|
||||
}
|
||||
// End added by ingress
|
||||
|
||||
@@ -636,7 +701,7 @@
|
||||
|
||||
// This can happen when a gateway has recently been deleted. Envoy will still request route
|
||||
// information due to the draining of listeners, so we should not return an error.
|
||||
- return nil
|
||||
+ return nil, false
|
||||
}
|
||||
|
||||
servers := merged.ServersByRouteName[routeName]
|
||||
@@ -768,9 +833,16 @@
|
||||
ValidateClusters: proto.BoolFalse,
|
||||
}
|
||||
|
||||
- return routeCfg
|
||||
+ routeCfg = envoyfilter.ApplyRouteConfigurationPatches(networking.EnvoyFilter_GATEWAY, node, efw, routeCfg)
|
||||
+ resource := &discovery.Resource{
|
||||
+ Name: routeName,
|
||||
+ Resource: util.MessageToAny(routeCfg),
|
||||
+ }
|
||||
+ return resource, false
|
||||
}
|
||||
|
||||
+// End modified by ingress
|
||||
+
|
||||
// hashRouteList returns a hash of a list of pointers
|
||||
func hashRouteList(r []*route.Route) uint64 {
|
||||
hash := md5.New()
|
||||
diff -Naur istio/pilot/pkg/networking/core/v1alpha3/httproute.go istio-new/pilot/pkg/networking/core/v1alpha3/httproute.go
|
||||
--- istio/pilot/pkg/networking/core/v1alpha3/httproute.go 2024-01-15 20:46:41.000000000 +0800
|
||||
+++ istio-new/pilot/pkg/networking/core/v1alpha3/httproute.go 2024-01-15 10:29:09.000000000 +0800
|
||||
@@ -78,17 +78,30 @@
|
||||
routeConfigurations = append(routeConfigurations, rc)
|
||||
}
|
||||
case model.Router:
|
||||
+ // Modified by ingress
|
||||
+ vsCache := make(map[int][]virtualServiceContext)
|
||||
+ envoyfilterKeys := efw.Keys()
|
||||
for _, routeName := range routeNames {
|
||||
- rc := configgen.buildGatewayHTTPRouteConfig(node, req.Push, routeName)
|
||||
- if rc != nil {
|
||||
- rc = envoyfilter.ApplyRouteConfigurationPatches(networking.EnvoyFilter_GATEWAY, node, efw, rc)
|
||||
- resource := &discovery.Resource{
|
||||
+ rc, cached := configgen.buildGatewayHTTPRouteConfig(node, req, routeName, vsCache, efw, envoyfilterKeys)
|
||||
+ if cached && !features.EnableUnsafeAssertions {
|
||||
+ hit++
|
||||
+ } else {
|
||||
+ miss++
|
||||
+ }
|
||||
+ if rc == nil {
|
||||
+ emptyRoute := &route.RouteConfiguration{
|
||||
+ Name: routeName,
|
||||
+ VirtualHosts: []*route.VirtualHost{},
|
||||
+ ValidateClusters: proto.BoolFalse,
|
||||
+ }
|
||||
+ rc = &discovery.Resource{
|
||||
Name: routeName,
|
||||
- Resource: util.MessageToAny(rc),
|
||||
+ Resource: util.MessageToAny(emptyRoute),
|
||||
}
|
||||
- routeConfigurations = append(routeConfigurations, resource)
|
||||
}
|
||||
+ routeConfigurations = append(routeConfigurations, rc)
|
||||
}
|
||||
+ // End modified by ingress
|
||||
}
|
||||
if !features.EnableRDSCaching {
|
||||
return routeConfigurations, model.DefaultXdsLogDetails
|
||||
diff -Naur istio/pilot/pkg/xds/discovery.go istio-new/pilot/pkg/xds/discovery.go
|
||||
--- istio/pilot/pkg/xds/discovery.go 2024-01-15 20:46:45.000000000 +0800
|
||||
+++ istio-new/pilot/pkg/xds/discovery.go 2024-01-12 19:56:02.000000000 +0800
|
||||
@@ -392,6 +392,9 @@
|
||||
// ConfigUpdate implements ConfigUpdater interface, used to request pushes.
|
||||
// It replaces the 'clear cache' from v1.
|
||||
func (s *DiscoveryServer) ConfigUpdate(req *model.PushRequest) {
|
||||
+ if req.Full {
|
||||
+ log.Infof("full push happen, reason:%v", req.Reason)
|
||||
+ }
|
||||
inboundConfigUpdates.Increment()
|
||||
s.InboundUpdates.Inc()
|
||||
s.pushChannel <- req
|
||||
60
istio/1.12/patches/istio/20240201-optimize-default-arg.patch
Normal file
60
istio/1.12/patches/istio/20240201-optimize-default-arg.patch
Normal file
@@ -0,0 +1,60 @@
|
||||
diff -Naur istio/pilot/cmd/pilot-agent/status/util/stats.go istio-new/pilot/cmd/pilot-agent/status/util/stats.go
|
||||
--- istio/pilot/cmd/pilot-agent/status/util/stats.go 2024-02-01 10:20:13.000000000 +0800
|
||||
+++ istio-new/pilot/cmd/pilot-agent/status/util/stats.go 2024-01-31 22:44:53.000000000 +0800
|
||||
@@ -73,7 +73,7 @@
|
||||
localHostAddr = "localhost"
|
||||
}
|
||||
|
||||
- readinessURL := fmt.Sprintf("http://%s:%d/stats?usedonly&filter=%s", localHostAddr, adminPort, readyStatsRegex)
|
||||
+ readinessURL := fmt.Sprintf("http://%s:%d/stats?usedonly", localHostAddr, adminPort)
|
||||
stats, err := http.DoHTTPGetWithTimeout(readinessURL, readinessTimeout)
|
||||
if err != nil {
|
||||
return nil, false, err
|
||||
@@ -105,7 +105,7 @@
|
||||
localHostAddr = "localhost"
|
||||
}
|
||||
|
||||
- stats, err := http.DoHTTPGet(fmt.Sprintf("http://%s:%d/stats?usedonly&filter=%s", localHostAddr, adminPort, updateStatsRegex))
|
||||
+ stats, err := http.DoHTTPGet(fmt.Sprintf("http://%s:%d/stats?usedonly", localHostAddr, adminPort))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
diff -Naur istio/pilot/pkg/features/pilot.go istio-new/pilot/pkg/features/pilot.go
|
||||
--- istio/pilot/pkg/features/pilot.go 2024-02-01 10:20:17.000000000 +0800
|
||||
+++ istio-new/pilot/pkg/features/pilot.go 2024-02-01 10:16:18.000000000 +0800
|
||||
@@ -575,6 +575,8 @@
|
||||
"If enabled, each host in virtualservice will have an independent RDS, which is used with SRDS").Get()
|
||||
OnDemandRDS = env.RegisterBoolVar("ON_DEMAND_RDS", false,
|
||||
"If enabled, the on demand filter will be added to the HCM filters").Get()
|
||||
+ DefaultUpstreamConcurrencyThreshold = env.RegisterIntVar("DEFAULT_UPSTREAM_CONCURRENCY_THRESHOLD", 1000000,
|
||||
+ "The default threshold of max_requests/max_pending_requests/max_connections of circuit breaker").Get()
|
||||
// End added by ingress
|
||||
)
|
||||
|
||||
diff -Naur istio/pilot/pkg/networking/core/v1alpha3/cluster.go istio-new/pilot/pkg/networking/core/v1alpha3/cluster.go
|
||||
--- istio/pilot/pkg/networking/core/v1alpha3/cluster.go 2024-02-01 10:20:17.000000000 +0800
|
||||
+++ istio-new/pilot/pkg/networking/core/v1alpha3/cluster.go 2024-02-01 10:16:05.000000000 +0800
|
||||
@@ -61,6 +61,7 @@
|
||||
|
||||
// getDefaultCircuitBreakerThresholds returns a copy of the default circuit breaker thresholds for the given traffic direction.
|
||||
func getDefaultCircuitBreakerThresholds() *cluster.CircuitBreakers_Thresholds {
|
||||
+ // Modified by ingress
|
||||
return &cluster.CircuitBreakers_Thresholds{
|
||||
// DefaultMaxRetries specifies the default for the Envoy circuit breaker parameter max_retries. This
|
||||
// defines the maximum number of parallel retries a given Envoy will allow to the upstream cluster. Envoy defaults
|
||||
@@ -68,11 +69,12 @@
|
||||
// where multiple endpoints in a cluster are terminated. In these scenarios the circuit breaker can kick
|
||||
// in before Pilot is able to deliver an updated endpoint list to Envoy, leading to client-facing 503s.
|
||||
MaxRetries: &wrappers.UInt32Value{Value: math.MaxUint32},
|
||||
- MaxRequests: &wrappers.UInt32Value{Value: math.MaxUint32},
|
||||
- MaxConnections: &wrappers.UInt32Value{Value: math.MaxUint32},
|
||||
- MaxPendingRequests: &wrappers.UInt32Value{Value: math.MaxUint32},
|
||||
+ MaxRequests: &wrappers.UInt32Value{Value: uint32(features.DefaultUpstreamConcurrencyThreshold)},
|
||||
+ MaxConnections: &wrappers.UInt32Value{Value: uint32(features.DefaultUpstreamConcurrencyThreshold)},
|
||||
+ MaxPendingRequests: &wrappers.UInt32Value{Value: uint32(features.DefaultUpstreamConcurrencyThreshold)},
|
||||
TrackRemaining: true,
|
||||
}
|
||||
+ // End modified by ingress
|
||||
}
|
||||
|
||||
// BuildClusters returns the list of clusters for the given proxy. This is the CDS output
|
||||
@@ -0,0 +1,88 @@
|
||||
diff -Naur istio/pilot/pkg/networking/core/v1alpha3/gateway.go istio-new/pilot/pkg/networking/core/v1alpha3/gateway.go
|
||||
--- istio/pilot/pkg/networking/core/v1alpha3/gateway.go 2024-02-01 13:53:17.000000000 +0800
|
||||
+++ istio-new/pilot/pkg/networking/core/v1alpha3/gateway.go 2024-02-01 13:52:11.000000000 +0800
|
||||
@@ -501,6 +501,16 @@
|
||||
gatewayVirtualServices[gatewayName] = virtualServices
|
||||
}
|
||||
for _, virtualService := range virtualServices {
|
||||
+ virtualServiceHosts := host.NewNames(virtualService.Spec.(*networking.VirtualService).Hosts)
|
||||
+ serverHosts := host.NamesForNamespace(server.Hosts, virtualService.Namespace)
|
||||
+
|
||||
+ // We have two cases here:
|
||||
+ // 1. virtualService hosts are 1.foo.com, 2.foo.com, 3.foo.com and server hosts are ns/*.foo.com
|
||||
+ // 2. virtualService hosts are *.foo.com, and server hosts are ns/1.foo.com, ns/2.foo.com, ns/3.foo.com
|
||||
+ intersectingHosts := serverHosts.Intersection(virtualServiceHosts)
|
||||
+ if len(intersectingHosts) == 0 {
|
||||
+ continue
|
||||
+ }
|
||||
listenerVirtualServices = append(listenerVirtualServices, virtualServiceContext{
|
||||
virtualService: virtualService,
|
||||
server: server,
|
||||
@@ -615,22 +625,24 @@
|
||||
|
||||
// check all hostname if is not exist with HttpsRedirect set to true
|
||||
// create VirtualHost to redirect
|
||||
- for _, hostname := range server.Hosts {
|
||||
- if !server.GetTls().GetHttpsRedirect() {
|
||||
- continue
|
||||
- }
|
||||
- if vHost != nil && host.Name(hostname) == host.Name(hostRDSHost) {
|
||||
+ if server.GetTls().GetHttpsRedirect() {
|
||||
+ if vHost != nil {
|
||||
vHost.RequireTls = route.VirtualHost_ALL
|
||||
- continue
|
||||
+ } else {
|
||||
+ vHost = &route.VirtualHost{
|
||||
+ Name: util.DomainName(hostRDSHost, port),
|
||||
+ Domains: buildGatewayVirtualHostDomains(hostRDSHost, port),
|
||||
+ IncludeRequestAttemptCount: true,
|
||||
+ RequireTls: route.VirtualHost_ALL,
|
||||
+ }
|
||||
}
|
||||
- vHost = &route.VirtualHost{
|
||||
- Name: util.DomainName(hostname, port),
|
||||
- Domains: buildGatewayVirtualHostDomains(hostname, port),
|
||||
- IncludeRequestAttemptCount: true,
|
||||
- RequireTls: route.VirtualHost_ALL,
|
||||
+ } else if vHost != nil {
|
||||
+ mode := server.GetTls().GetMode()
|
||||
+ if mode == networking.ServerTLSSettings_MUTUAL ||
|
||||
+ mode == networking.ServerTLSSettings_ISTIO_MUTUAL {
|
||||
+ vHost.AllowServerNames = append(vHost.AllowServerNames, server.Hosts...)
|
||||
}
|
||||
}
|
||||
-
|
||||
}
|
||||
var virtualHosts []*route.VirtualHost
|
||||
if vHost == nil {
|
||||
@@ -642,6 +654,30 @@
|
||||
Routes: []*route.Route{},
|
||||
}}
|
||||
} else {
|
||||
+ sort.SliceStable(vHost.AllowServerNames, func(i, j int) bool {
|
||||
+ hostI := vHost.AllowServerNames[i]
|
||||
+ hostJ := vHost.AllowServerNames[j]
|
||||
+ if host.Name(hostI).SubsetOf(host.Name(hostJ)) {
|
||||
+ return true
|
||||
+ }
|
||||
+ return hostI < hostJ
|
||||
+ })
|
||||
+ var uniqueServerNames []string
|
||||
+ hasAllCatch := false
|
||||
+ for i, name := range vHost.AllowServerNames {
|
||||
+ if name == "*" {
|
||||
+ hasAllCatch = true
|
||||
+ break
|
||||
+ }
|
||||
+ if i == 0 || vHost.AllowServerNames[i-1] != name {
|
||||
+ uniqueServerNames = append(uniqueServerNames, name)
|
||||
+ }
|
||||
+ }
|
||||
+ if hasAllCatch {
|
||||
+ vHost.AllowServerNames = nil
|
||||
+ } else {
|
||||
+ vHost.AllowServerNames = uniqueServerNames
|
||||
+ }
|
||||
vHost.Routes = istio_route.CombineVHostRoutes(vHost.Routes)
|
||||
virtualHosts = append(virtualHosts, vHost)
|
||||
}
|
||||
41
istio/1.12/patches/istio/20240202-fix-rds-cache.patch
Normal file
41
istio/1.12/patches/istio/20240202-fix-rds-cache.patch
Normal file
@@ -0,0 +1,41 @@
|
||||
diff -Naur istio/pilot/pkg/xds/discovery.go istio-new/pilot/pkg/xds/discovery.go
|
||||
--- istio/pilot/pkg/xds/discovery.go 2024-02-02 16:26:49.000000000 +0800
|
||||
+++ istio-new/pilot/pkg/xds/discovery.go 2024-02-02 15:38:53.000000000 +0800
|
||||
@@ -18,6 +18,7 @@
|
||||
"context"
|
||||
"fmt"
|
||||
"strconv"
|
||||
+ "strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
@@ -41,6 +42,7 @@
|
||||
"istio.io/istio/pilot/pkg/util/sets"
|
||||
v3 "istio.io/istio/pilot/pkg/xds/v3"
|
||||
"istio.io/istio/pkg/cluster"
|
||||
+ "istio.io/istio/pkg/config/constants"
|
||||
"istio.io/istio/pkg/security"
|
||||
)
|
||||
|
||||
@@ -332,6 +334,21 @@
|
||||
} else {
|
||||
// Otherwise, just clear the updated configs
|
||||
s.Cache.Clear(req.ConfigsUpdated)
|
||||
+ //Added by ingress
|
||||
+ trimKeyMap := make(map[model.ConfigKey]struct{})
|
||||
+ for configKey := range req.ConfigsUpdated {
|
||||
+ if strings.HasPrefix(configKey.Name, constants.IstioIngressGatewayName+"-") {
|
||||
+ trimKeyMap[model.ConfigKey{
|
||||
+ Kind: configKey.Kind,
|
||||
+ Name: strings.TrimPrefix(configKey.Name, constants.IstioIngressGatewayName+"-"),
|
||||
+ Namespace: configKey.Namespace,
|
||||
+ }] = struct{}{}
|
||||
+ }
|
||||
+ }
|
||||
+ if len(trimKeyMap) > 0 {
|
||||
+ s.Cache.Clear(trimKeyMap)
|
||||
+ }
|
||||
+ //End added by ingress
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
diff -Naur istio/pilot/cmd/pilot-agent/status/util/stats.go istio-new/pilot/cmd/pilot-agent/status/util/stats.go
|
||||
--- istio/pilot/cmd/pilot-agent/status/util/stats.go 2024-02-04 18:48:18.000000000 +0800
|
||||
+++ istio-new/pilot/cmd/pilot-agent/status/util/stats.go 2024-02-04 09:35:42.000000000 +0800
|
||||
@@ -37,7 +37,7 @@
|
||||
updateStatsRegex = "^(cluster_manager\\.cds|listener_manager\\.lds)\\.(update_success|update_rejected)$"
|
||||
)
|
||||
|
||||
-var readinessTimeout = time.Second * 3 // Default Readiness timeout. It is set the same in helm charts.
|
||||
+var readinessTimeout = time.Second * 60 // Default Readiness timeout. It is set the same in helm charts.
|
||||
|
||||
type stat struct {
|
||||
name string
|
||||
@@ -105,7 +105,7 @@
|
||||
localHostAddr = "localhost"
|
||||
}
|
||||
|
||||
- stats, err := http.DoHTTPGet(fmt.Sprintf("http://%s:%d/stats?usedonly", localHostAddr, adminPort))
|
||||
+ stats, err := http.DoHTTPGetWithTimeout(fmt.Sprintf("http://%s:%d/stats?usedonly", localHostAddr, adminPort), readinessTimeout)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -584,7 +584,7 @@ func locateChart(cpOpts *action.ChartPathOptions, name string, settings *cli.Env
|
||||
return fileAbsPath, nil
|
||||
}
|
||||
|
||||
func ParseLatestVersion(repoUrl string, version string) (string, error) {
|
||||
func ParseLatestVersion(repoUrl string, version string, devel bool) (string, error) {
|
||||
|
||||
cpOpts := &action.ChartPathOptions{
|
||||
RepoURL: repoUrl,
|
||||
@@ -632,7 +632,16 @@ func ParseLatestVersion(repoUrl string, version string) (string, error) {
|
||||
|
||||
// get higress helm chart latest version
|
||||
if entries, ok := indexFile.Entries[RepoChartIndexYamlHigressIndex]; ok {
|
||||
return entries[0].AppVersion, nil
|
||||
if devel {
|
||||
return entries[0].AppVersion, nil
|
||||
}
|
||||
|
||||
if chatVersion, err := indexFile.Get(RepoChartIndexYamlHigressIndex, ""); err != nil {
|
||||
return "", errors.New("can't find higress latest version")
|
||||
} else {
|
||||
return chatVersion.Version, nil
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
return "", errors.New("can't find higress latest version")
|
||||
|
||||
@@ -52,6 +52,8 @@ type InstallArgs struct {
|
||||
Set []string
|
||||
// ManifestsPath is a path to a ManifestsPath and profiles directory in the local filesystem with a release tgz.
|
||||
ManifestsPath string
|
||||
// Devel if set true when version is latest, it will get latest version, otherwise it will get latest stable version
|
||||
Devel bool
|
||||
}
|
||||
|
||||
func (a *InstallArgs) String() string {
|
||||
@@ -67,6 +69,7 @@ func addInstallFlags(cmd *cobra.Command, args *InstallArgs) {
|
||||
cmd.PersistentFlags().StringSliceVarP(&args.InFilenames, "filename", "f", nil, filenameFlagHelpStr)
|
||||
cmd.PersistentFlags().StringArrayVarP(&args.Set, "set", "s", nil, setFlagHelpStr)
|
||||
cmd.PersistentFlags().StringVarP(&args.ManifestsPath, "manifests", "d", "", manifestsFlagHelpStr)
|
||||
cmd.PersistentFlags().BoolVar(&args.Devel, "devel", false, "use development versions (alpha, beta, and release candidate releases), If version is set, this is ignored")
|
||||
}
|
||||
|
||||
// --manifests is an alias for --set installPackagePath=
|
||||
@@ -141,7 +144,7 @@ func install(writer io.Writer, iArgs *InstallArgs) error {
|
||||
return err
|
||||
}
|
||||
|
||||
err = installManifests(profile, writer)
|
||||
err = installManifests(profile, writer, iArgs.Devel)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to install manifests: %v", err)
|
||||
}
|
||||
@@ -192,8 +195,8 @@ func promptProfileName(writer io.Writer) string {
|
||||
|
||||
}
|
||||
|
||||
func installManifests(profile *helm.Profile, writer io.Writer) error {
|
||||
installer, err := installer.NewInstaller(profile, writer, false)
|
||||
func installManifests(profile *helm.Profile, writer io.Writer, devel bool) error {
|
||||
installer, err := installer.NewInstaller(profile, writer, false, devel, installer.InstallInstallerMode)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -52,6 +52,8 @@ type ComponentOptions struct {
|
||||
Quiet bool
|
||||
// Capabilities
|
||||
Capabilities *chartutil.Capabilities
|
||||
// devel
|
||||
Devel bool
|
||||
}
|
||||
|
||||
type ComponentOption func(*ComponentOptions)
|
||||
@@ -98,6 +100,12 @@ func WithQuiet() ComponentOption {
|
||||
}
|
||||
}
|
||||
|
||||
func WithDevel(devel bool) ComponentOption {
|
||||
return func(opts *ComponentOptions) {
|
||||
opts.Devel = devel
|
||||
}
|
||||
}
|
||||
|
||||
func renderComponentManifest(spec any, renderer helm.Renderer, addOn bool, name ComponentName, namespace string) (string, error) {
|
||||
var valsBytes []byte
|
||||
var valsYaml string
|
||||
|
||||
@@ -52,7 +52,7 @@ func (h *HigressComponent) Run() error {
|
||||
// Parse latest version
|
||||
if h.opts.Version == helm.RepoLatestVersion {
|
||||
|
||||
latestVersion, err := helm.ParseLatestVersion(h.opts.RepoURL, h.opts.Version)
|
||||
latestVersion, err := helm.ParseLatestVersion(h.opts.RepoURL, h.opts.Version, h.opts.Devel)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -28,6 +28,8 @@ import (
|
||||
"k8s.io/client-go/util/homedir"
|
||||
)
|
||||
|
||||
type InstallerMode int32
|
||||
|
||||
const (
|
||||
HgctlHomeDirPath = ".hgctl"
|
||||
StandaloneInstalledPath = "higress-standalone"
|
||||
@@ -37,20 +39,26 @@ const (
|
||||
DefaultIstioNamespace = "istio-system"
|
||||
)
|
||||
|
||||
const (
|
||||
InstallInstallerMode InstallerMode = iota
|
||||
UpgradeInstallerMode
|
||||
UninstallInstallerMode
|
||||
)
|
||||
|
||||
type Installer interface {
|
||||
Install() error
|
||||
UnInstall() error
|
||||
Upgrade() error
|
||||
}
|
||||
|
||||
func NewInstaller(profile *helm.Profile, writer io.Writer, quiet bool) (Installer, error) {
|
||||
func NewInstaller(profile *helm.Profile, writer io.Writer, quiet bool, devel bool, installerMode InstallerMode) (Installer, error) {
|
||||
switch profile.Global.Install {
|
||||
case helm.InstallK8s, helm.InstallLocalK8s:
|
||||
cliClient, err := kubernetes.NewCLIClient(options.DefaultConfigFlags.ToRawKubeConfigLoader())
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to build kubernetes client: %w", err)
|
||||
}
|
||||
installer, err := NewK8sInstaller(profile, cliClient, writer, quiet)
|
||||
installer, err := NewK8sInstaller(profile, cliClient, writer, quiet, devel, installerMode)
|
||||
return installer, err
|
||||
case helm.InstallLocalDocker:
|
||||
installer, err := NewDockerInstaller(profile, writer, quiet)
|
||||
|
||||
@@ -254,7 +254,7 @@ func (o *K8sInstaller) isNamespacedObject(obj *object.K8sObject) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
func NewK8sInstaller(profile *helm.Profile, cli kubernetes.CLIClient, writer io.Writer, quiet bool) (*K8sInstaller, error) {
|
||||
func NewK8sInstaller(profile *helm.Profile, cli kubernetes.CLIClient, writer io.Writer, quiet bool, devel bool, installerMode InstallerMode) (*K8sInstaller, error) {
|
||||
if profile == nil {
|
||||
return nil, errors.New("install profile is empty")
|
||||
}
|
||||
@@ -267,14 +267,20 @@ func NewK8sInstaller(profile *helm.Profile, cli kubernetes.CLIClient, writer io.
|
||||
}
|
||||
fmt.Fprintf(writer, "%s\n", capabilities.KubeVersion.Version)
|
||||
// initialize components
|
||||
higressVersion := profile.Charts.Higress.Version
|
||||
if installerMode == UninstallInstallerMode {
|
||||
// uninstall
|
||||
higressVersion = profile.HigressVersion
|
||||
}
|
||||
components := make(map[ComponentName]Component)
|
||||
opts := []ComponentOption{
|
||||
WithComponentNamespace(profile.Global.Namespace),
|
||||
WithComponentChartPath(profile.InstallPackagePath),
|
||||
WithComponentVersion(profile.Charts.Higress.Version),
|
||||
WithComponentVersion(higressVersion),
|
||||
WithComponentRepoURL(profile.Charts.Higress.Url),
|
||||
WithComponentChartName(profile.Charts.Higress.Name),
|
||||
WithComponentCapabilities(capabilities),
|
||||
WithDevel(devel),
|
||||
}
|
||||
if quiet {
|
||||
opts = append(opts, WithQuiet())
|
||||
|
||||
@@ -37,6 +37,8 @@ type ManifestArgs struct {
|
||||
Set []string
|
||||
// ManifestsPath is a path to a ManifestsPath and profiles directory in the local filesystem with a release tgz.
|
||||
ManifestsPath string
|
||||
// Devel if set true when version is latest, it will get latest version, otherwise it will get latest stable version
|
||||
Devel bool
|
||||
}
|
||||
|
||||
func (a *ManifestArgs) String() string {
|
||||
@@ -70,6 +72,7 @@ func addManifestFlags(cmd *cobra.Command, args *ManifestArgs) {
|
||||
cmd.PersistentFlags().StringSliceVarP(&args.InFilenames, "filename", "f", nil, filenameFlagHelpStr)
|
||||
cmd.PersistentFlags().StringArrayVarP(&args.Set, "set", "s", nil, setFlagHelpStr)
|
||||
cmd.PersistentFlags().StringVarP(&args.ManifestsPath, "manifests", "d", "", manifestsFlagHelpStr)
|
||||
cmd.PersistentFlags().BoolVar(&args.Devel, "devel", false, "use development versions (alpha, beta, and release candidate releases), If version is set, this is ignored")
|
||||
}
|
||||
|
||||
// newManifestGenerateCmd generates a higress install manifest and applies it to a cluster
|
||||
@@ -113,20 +116,20 @@ func generate(writer io.Writer, iArgs *ManifestArgs) error {
|
||||
return err
|
||||
}
|
||||
|
||||
err = genManifests(profile, writer)
|
||||
err = genManifests(profile, writer, iArgs.Devel)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to install manifests: %v", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func genManifests(profile *helm.Profile, writer io.Writer) error {
|
||||
func genManifests(profile *helm.Profile, writer io.Writer, devel bool) error {
|
||||
cliClient, err := kubernetes.NewCLIClient(options.DefaultConfigFlags.ToRawKubeConfigLoader())
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to build kubernetes client: %w", err)
|
||||
}
|
||||
|
||||
op, err := installer.NewK8sInstaller(profile, cliClient, writer, true)
|
||||
op, err := installer.NewK8sInstaller(profile, cliClient, writer, true, devel, installer.InstallInstallerMode)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -126,7 +126,7 @@ func promptUninstall(writer io.Writer) bool {
|
||||
}
|
||||
|
||||
func uninstallManifests(profile *helm.Profile, writer io.Writer, uiArgs *uninstallArgs) error {
|
||||
installer, err := installer.NewInstaller(profile, writer, false)
|
||||
installer, err := installer.NewInstaller(profile, writer, false, false, installer.UninstallInstallerMode)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -37,6 +37,7 @@ func addUpgradeFlags(cmd *cobra.Command, args *upgradeArgs) {
|
||||
cmd.PersistentFlags().StringSliceVarP(&args.InFilenames, "filename", "f", nil, filenameFlagHelpStr)
|
||||
cmd.PersistentFlags().StringArrayVarP(&args.Set, "set", "s", nil, setFlagHelpStr)
|
||||
cmd.PersistentFlags().StringVarP(&args.ManifestsPath, "manifests", "d", "", manifestsFlagHelpStr)
|
||||
cmd.PersistentFlags().BoolVar(&args.Devel, "devel", false, "use development versions (alpha, beta, and release candidate releases), If version is set, this is ignored")
|
||||
}
|
||||
|
||||
// newUpgradeCmd upgrades Istio control plane in-place with eligibility checks.
|
||||
@@ -91,7 +92,7 @@ func upgrade(writer io.Writer, iArgs *InstallArgs) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
err = upgradeManifests(profile, writer)
|
||||
err = upgradeManifests(profile, writer, iArgs.Devel)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -120,8 +121,8 @@ func promptUpgrade(writer io.Writer) bool {
|
||||
}
|
||||
}
|
||||
|
||||
func upgradeManifests(profile *helm.Profile, writer io.Writer) error {
|
||||
installer, err := installer.NewInstaller(profile, writer, false)
|
||||
func upgradeManifests(profile *helm.Profile, writer io.Writer, devel bool) error {
|
||||
installer, err := installer.NewInstaller(profile, writer, false, devel, installer.UpgradeInstallerMode)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -19,6 +19,7 @@ import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
@@ -474,9 +475,6 @@ func (m *IngressConfig) convertVirtualService(configs []common.WrapperConfig) []
|
||||
gateways := []string{m.namespace + "/" +
|
||||
common.CreateConvertedName(m.clusterId, cleanHost),
|
||||
common.CreateConvertedName(constants.IstioIngressGatewayName, cleanHost)}
|
||||
if host != "*" {
|
||||
gateways = append(gateways, m.namespace+"/"+common.CreateConvertedName(m.clusterId, common.CleanHost("*")))
|
||||
}
|
||||
|
||||
wrapperVS, exist := convertOptions.VirtualServices[host]
|
||||
if !exist {
|
||||
@@ -673,6 +671,18 @@ func (m *IngressConfig) convertDestinationRule(configs []common.WrapperConfig) [
|
||||
|
||||
out := make([]config.Config, 0, len(destinationRules))
|
||||
for _, dr := range destinationRules {
|
||||
sort.SliceStable(dr.DestinationRule.TrafficPolicy.PortLevelSettings, func(i, j int) bool {
|
||||
portI := dr.DestinationRule.TrafficPolicy.PortLevelSettings[i].Port
|
||||
portJ := dr.DestinationRule.TrafficPolicy.PortLevelSettings[j].Port
|
||||
if portI == nil && portJ == nil {
|
||||
return true
|
||||
} else if portI == nil {
|
||||
return true
|
||||
} else if portJ == nil {
|
||||
return false
|
||||
}
|
||||
return portI.Number < portJ.Number
|
||||
})
|
||||
drName := util.CreateDestinationRuleName(m.clusterId, dr.ServiceKey.Namespace, dr.ServiceKey.Name)
|
||||
out = append(out, config.Config{
|
||||
Meta: config.Meta{
|
||||
|
||||
@@ -314,9 +314,6 @@ func (m *KIngressConfig) convertVirtualService(configs []common.WrapperConfig) [
|
||||
gateways := []string{m.namespace + "/" +
|
||||
common.CreateConvertedName(m.clusterId, cleanHost),
|
||||
common.CreateConvertedName(constants.IstioIngressGatewayName, cleanHost)}
|
||||
if host != "*" {
|
||||
gateways = append(gateways, m.namespace+"/"+common.CreateConvertedName(m.clusterId, common.CleanHost("*")))
|
||||
}
|
||||
|
||||
wrapperVS, exist := convertOptions.VirtualServices[host]
|
||||
if !exist {
|
||||
|
||||
@@ -83,7 +83,7 @@ func (i *Ingress) NeedRegexMatch(path string) bool {
|
||||
if i.Rewrite == nil {
|
||||
return false
|
||||
}
|
||||
if strings.ContainsAny(path, `\.+*?()|[]{}^$`) {
|
||||
if i.Rewrite.RewriteTarget != "" && strings.ContainsAny(path, `\.+*?()|[]{}^$`) {
|
||||
return true
|
||||
}
|
||||
if strings.ContainsAny(i.Rewrite.RewriteTarget, `$\`) {
|
||||
|
||||
@@ -67,6 +67,15 @@ func TestNeedRegexMatch(t *testing.T) {
|
||||
inputPath: "/.*",
|
||||
expect: true,
|
||||
},
|
||||
{
|
||||
input: &Ingress{
|
||||
Rewrite: &RewriteConfig{
|
||||
UseRegex: false,
|
||||
},
|
||||
},
|
||||
inputPath: "/.",
|
||||
expect: false,
|
||||
},
|
||||
{
|
||||
input: &Ingress{
|
||||
Rewrite: &RewriteConfig{
|
||||
|
||||
@@ -23,13 +23,14 @@ import (
|
||||
)
|
||||
|
||||
const (
|
||||
exact = "exact"
|
||||
regex = "regex"
|
||||
prefix = "prefix"
|
||||
MatchMethod = "match-method"
|
||||
MatchQuery = "match-query"
|
||||
MatchHeader = "match-header"
|
||||
sep = " "
|
||||
exact = "exact"
|
||||
regex = "regex"
|
||||
prefix = "prefix"
|
||||
MatchMethod = "match-method"
|
||||
MatchQuery = "match-query"
|
||||
MatchHeader = "match-header"
|
||||
MatchPseudoHeader = "match-pseudo-header"
|
||||
sep = " "
|
||||
)
|
||||
|
||||
var (
|
||||
@@ -56,6 +57,24 @@ func (m match) Parse(annotations Annotations, config *Ingress, _ *GlobalContext)
|
||||
IngressLog.Errorf("parse headers error %v within ingress %s/%s", err, config.Namespace, config.Name)
|
||||
}
|
||||
|
||||
var pseudoHeaderMatches map[string]map[string]string
|
||||
if pseudoHeaderMatches, err = m.matchByHeaderOrQueryParma(annotations, MatchPseudoHeader, pseudoHeaderMatches); err != nil {
|
||||
IngressLog.Errorf("parse headers error %v within ingress %s/%s", err, config.Namespace, config.Name)
|
||||
}
|
||||
if pseudoHeaderMatches != nil && len(pseudoHeaderMatches) > 0 {
|
||||
if config.Match.Headers == nil {
|
||||
config.Match.Headers = make(map[string]map[string]string)
|
||||
}
|
||||
for typ, mmap := range pseudoHeaderMatches {
|
||||
if config.Match.Headers[typ] == nil {
|
||||
config.Match.Headers[typ] = make(map[string]string)
|
||||
}
|
||||
for k, v := range mmap {
|
||||
config.Match.Headers[typ][":"+k] = v
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if config.Match.QueryParams, err = m.matchByHeaderOrQueryParma(annotations, MatchQuery, config.Match.QueryParams); err != nil {
|
||||
IngressLog.Errorf("parse query params error %v within ingress %s/%s", err, config.Namespace, config.Name)
|
||||
}
|
||||
|
||||
@@ -15,6 +15,7 @@
|
||||
package annotations
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/google/go-cmp/cmp"
|
||||
@@ -112,11 +113,47 @@ func TestMatch_ParseHeaders(t *testing.T) {
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
typ: "exact",
|
||||
key: ":method",
|
||||
value: "GET",
|
||||
expect: map[string]map[string]string{
|
||||
exact: {
|
||||
":method": "GET",
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
typ: "prefix",
|
||||
key: ":path",
|
||||
value: "/foo",
|
||||
expect: map[string]map[string]string{
|
||||
prefix: {
|
||||
":path": "/foo",
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
typ: "regex",
|
||||
key: ":authority",
|
||||
value: "test\\d+\\.com",
|
||||
expect: map[string]map[string]string{
|
||||
regex: {
|
||||
":authority": "test\\d+\\.com",
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range testCases {
|
||||
t.Run("", func(t *testing.T) {
|
||||
key := buildHigressAnnotationKey(tt.typ + "-" + MatchHeader + "-" + tt.key)
|
||||
matchKeyword := MatchHeader
|
||||
headerKey := tt.key
|
||||
if strings.HasPrefix(headerKey, ":") {
|
||||
headerKey = strings.TrimPrefix(headerKey, ":")
|
||||
matchKeyword = MatchPseudoHeader
|
||||
}
|
||||
key := buildHigressAnnotationKey(tt.typ + "-" + matchKeyword + "-" + headerKey)
|
||||
input := Annotations{key: tt.value}
|
||||
config := &Ingress{}
|
||||
_ = parser.Parse(input, config, nil)
|
||||
|
||||
@@ -65,7 +65,12 @@ func (u upstreamTLS) Parse(annotations Annotations, config *Ingress, _ *GlobalCo
|
||||
}
|
||||
|
||||
defer func() {
|
||||
config.UpstreamTLS = upstreamTLSConfig
|
||||
if upstreamTLSConfig.BackendProtocol == defaultBackendProtocol {
|
||||
// no need destination rule when use HTTP protocol
|
||||
config.UpstreamTLS = nil
|
||||
} else {
|
||||
config.UpstreamTLS = upstreamTLSConfig
|
||||
}
|
||||
}()
|
||||
|
||||
if proto, err := annotations.ParseStringASAP(backendProtocol); err == nil {
|
||||
|
||||
@@ -33,6 +33,12 @@ func TestUpstreamTLSParse(t *testing.T) {
|
||||
input: Annotations{},
|
||||
expect: nil,
|
||||
},
|
||||
{
|
||||
input: Annotations{
|
||||
buildNginxAnnotationKey(backendProtocol): "HTTP",
|
||||
},
|
||||
expect: nil,
|
||||
},
|
||||
{
|
||||
input: Annotations{
|
||||
buildNginxAnnotationKey(proxySSLSecret): "",
|
||||
|
||||
@@ -36,7 +36,6 @@ func ValidateBackendResource(resource *v1.TypedLocalObjectReference) bool {
|
||||
if resource == nil || resource.APIGroup == nil ||
|
||||
*resource.APIGroup != netv1.SchemeGroupVersion.Group ||
|
||||
resource.Kind != "McpBridge" || resource.Name != "default" {
|
||||
IngressLog.Warnf("invalid mcpbridge resource: %v", resource)
|
||||
return false
|
||||
}
|
||||
return true
|
||||
|
||||
@@ -37,6 +37,7 @@ type HigressConfig struct {
|
||||
Tracing *Tracing `json:"tracing,omitempty"`
|
||||
Gzip *Gzip `json:"gzip,omitempty"`
|
||||
Downstream *Downstream `json:"downstream,omitempty"`
|
||||
Upstream *Upstream `json:"upstream,omitempty"`
|
||||
DisableXEnvoyHeaders bool `json:"disableXEnvoyHeaders,omitempty"`
|
||||
AddXRealIpHeader bool `json:"addXRealIpHeader,omitempty"`
|
||||
}
|
||||
@@ -47,6 +48,7 @@ func NewDefaultHigressConfig() *HigressConfig {
|
||||
Tracing: NewDefaultTracing(),
|
||||
Gzip: NewDefaultGzip(),
|
||||
Downstream: globalOption.Downstream,
|
||||
Upstream: globalOption.Upstream,
|
||||
DisableXEnvoyHeaders: globalOption.DisableXEnvoyHeaders,
|
||||
AddXRealIpHeader: globalOption.AddXRealIpHeader,
|
||||
}
|
||||
|
||||
@@ -15,7 +15,6 @@
|
||||
package configmap
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"sync/atomic"
|
||||
@@ -38,19 +37,22 @@ const (
|
||||
minInitialConnectionWindowSize = 65535
|
||||
maxInitialConnectionWindowSize = 2147483647
|
||||
|
||||
defaultIdleTimeout = 180
|
||||
defaultMaxRequestHeadersKb = 60
|
||||
defaultConnectionBufferLimits = 32768
|
||||
defaultMaxConcurrentStreams = 100
|
||||
defaultInitialStreamWindowSize = 65535
|
||||
defaultInitialConnectionWindowSize = 1048576
|
||||
defaultAddXRealIpHeader = false
|
||||
defaultDisableXEnvoyHeaders = false
|
||||
defaultIdleTimeout = 180
|
||||
defaultUpStreamIdleTimeout = 10
|
||||
defaultUpStreamConnectionBufferLimits = 10485760
|
||||
defaultMaxRequestHeadersKb = 60
|
||||
defaultConnectionBufferLimits = 32768
|
||||
defaultMaxConcurrentStreams = 100
|
||||
defaultInitialStreamWindowSize = 65535
|
||||
defaultInitialConnectionWindowSize = 1048576
|
||||
defaultAddXRealIpHeader = false
|
||||
defaultDisableXEnvoyHeaders = false
|
||||
)
|
||||
|
||||
// Global configures the behavior of the downstream connection, x-real-ip header and x-envoy headers.
|
||||
type Global struct {
|
||||
Downstream *Downstream `json:"downstream,omitempty"`
|
||||
Upstream *Upstream `json:"upstream,omitempty"`
|
||||
AddXRealIpHeader bool `json:"addXRealIpHeader,omitempty"`
|
||||
DisableXEnvoyHeaders bool `json:"disableXEnvoyHeaders,omitempty"`
|
||||
}
|
||||
@@ -58,7 +60,7 @@ type Global struct {
|
||||
// Downstream configures the behavior of the downstream connection.
|
||||
type Downstream struct {
|
||||
// IdleTimeout limits the time that a connection may be idle and stream idle.
|
||||
IdleTimeout uint32 `json:"idleTimeout,omitempty"`
|
||||
IdleTimeout uint32 `json:"idleTimeout"`
|
||||
// MaxRequestHeadersKb limits the size of request headers allowed.
|
||||
MaxRequestHeadersKb uint32 `json:"maxRequestHeadersKb,omitempty"`
|
||||
// ConnectionBufferLimits configures the buffer size limits for connections.
|
||||
@@ -67,6 +69,14 @@ type Downstream struct {
|
||||
Http2 *Http2 `json:"http2,omitempty"`
|
||||
}
|
||||
|
||||
// Upstream configures the behavior of the upstream connection.
|
||||
type Upstream struct {
|
||||
// IdleTimeout limits the time that a connection may be idle on the upstream.
|
||||
IdleTimeout uint32 `json:"idleTimeout"`
|
||||
// ConnectionBufferLimits configures the buffer size limits for connections.
|
||||
ConnectionBufferLimits uint32 `json:"connectionBufferLimits,omitempty"`
|
||||
}
|
||||
|
||||
// Http2 configures HTTP/2 specific options.
|
||||
type Http2 struct {
|
||||
// MaxConcurrentStreams limits the number of concurrent streams allowed.
|
||||
@@ -125,7 +135,7 @@ func compareGlobal(old *Global, new *Global) (Result, error) {
|
||||
return ResultDelete, nil
|
||||
}
|
||||
|
||||
if new.Downstream == nil && !new.AddXRealIpHeader && !new.DisableXEnvoyHeaders {
|
||||
if new.Downstream == nil && new.Upstream == nil && !new.AddXRealIpHeader && !new.DisableXEnvoyHeaders {
|
||||
return ResultDelete, nil
|
||||
}
|
||||
|
||||
@@ -139,18 +149,30 @@ func compareGlobal(old *Global, new *Global) (Result, error) {
|
||||
// deepCopyGlobal deep copies the global option.
|
||||
func deepCopyGlobal(global *Global) (*Global, error) {
|
||||
newGlobal := NewDefaultGlobalOption()
|
||||
bytes, err := json.Marshal(global)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
if global.Downstream != nil {
|
||||
newGlobal.Downstream.IdleTimeout = global.Downstream.IdleTimeout
|
||||
newGlobal.Downstream.MaxRequestHeadersKb = global.Downstream.MaxRequestHeadersKb
|
||||
newGlobal.Downstream.ConnectionBufferLimits = global.Downstream.ConnectionBufferLimits
|
||||
if global.Downstream.Http2 != nil {
|
||||
newGlobal.Downstream.Http2.MaxConcurrentStreams = global.Downstream.Http2.MaxConcurrentStreams
|
||||
newGlobal.Downstream.Http2.InitialStreamWindowSize = global.Downstream.Http2.InitialStreamWindowSize
|
||||
newGlobal.Downstream.Http2.InitialConnectionWindowSize = global.Downstream.Http2.InitialConnectionWindowSize
|
||||
}
|
||||
}
|
||||
err = json.Unmarshal(bytes, newGlobal)
|
||||
return newGlobal, err
|
||||
if global.Upstream != nil {
|
||||
newGlobal.Upstream.IdleTimeout = global.Upstream.IdleTimeout
|
||||
newGlobal.Upstream.ConnectionBufferLimits = global.Upstream.ConnectionBufferLimits
|
||||
}
|
||||
newGlobal.AddXRealIpHeader = global.AddXRealIpHeader
|
||||
newGlobal.DisableXEnvoyHeaders = global.DisableXEnvoyHeaders
|
||||
return newGlobal, nil
|
||||
}
|
||||
|
||||
// NewDefaultGlobalOption returns a default global config.
|
||||
func NewDefaultGlobalOption() *Global {
|
||||
return &Global{
|
||||
Downstream: NewDefaultDownstream(),
|
||||
Upstream: NewDefaultUpStream(),
|
||||
AddXRealIpHeader: defaultAddXRealIpHeader,
|
||||
DisableXEnvoyHeaders: defaultDisableXEnvoyHeaders,
|
||||
}
|
||||
@@ -166,6 +188,14 @@ func NewDefaultDownstream() *Downstream {
|
||||
}
|
||||
}
|
||||
|
||||
// NewDefaultUpStream returns a default upstream config.
|
||||
func NewDefaultUpStream() *Upstream {
|
||||
return &Upstream{
|
||||
IdleTimeout: defaultUpStreamIdleTimeout,
|
||||
ConnectionBufferLimits: defaultUpStreamConnectionBufferLimits,
|
||||
}
|
||||
}
|
||||
|
||||
// NewDefaultHttp2 returns a default http2 config.
|
||||
func NewDefaultHttp2() *Http2 {
|
||||
return &Http2{
|
||||
@@ -215,12 +245,14 @@ func (g *GlobalOptionController) GetName() string {
|
||||
func (g *GlobalOptionController) AddOrUpdateHigressConfig(name util.ClusterNamespacedName, old *HigressConfig, new *HigressConfig) error {
|
||||
newGlobal := &Global{
|
||||
Downstream: new.Downstream,
|
||||
Upstream: new.Upstream,
|
||||
AddXRealIpHeader: new.AddXRealIpHeader,
|
||||
DisableXEnvoyHeaders: new.DisableXEnvoyHeaders,
|
||||
}
|
||||
|
||||
oldGlobal := &Global{
|
||||
Downstream: old.Downstream,
|
||||
Upstream: old.Upstream,
|
||||
AddXRealIpHeader: old.AddXRealIpHeader,
|
||||
DisableXEnvoyHeaders: old.DisableXEnvoyHeaders,
|
||||
}
|
||||
@@ -264,6 +296,7 @@ func (g *GlobalOptionController) ValidHigressConfig(higressConfig *HigressConfig
|
||||
|
||||
global := &Global{
|
||||
Downstream: higressConfig.Downstream,
|
||||
Upstream: higressConfig.Upstream,
|
||||
AddXRealIpHeader: higressConfig.AddXRealIpHeader,
|
||||
DisableXEnvoyHeaders: higressConfig.DisableXEnvoyHeaders,
|
||||
}
|
||||
@@ -306,6 +339,18 @@ func (g *GlobalOptionController) ConstructEnvoyFilters() ([]*config.Config, erro
|
||||
downstreamConfig := g.generateDownstreamEnvoyFilter(downstreamStruct, bufferLimitStruct, namespace)
|
||||
configPatch = append(configPatch, downstreamConfig...)
|
||||
|
||||
if global.Upstream == nil {
|
||||
return generateEnvoyFilter(namespace, configPatch), nil
|
||||
}
|
||||
|
||||
upstreamStruct := g.constructUpstream(global.Upstream)
|
||||
bufferLimitStruct = g.constructUpstreamBufferLimit(global.Upstream)
|
||||
if len(upstreamStruct) == 0 {
|
||||
return generateEnvoyFilter(namespace, configPatch), nil
|
||||
}
|
||||
upstreamConfig := g.generateUpstreamEnvoyFilter(upstreamStruct, bufferLimitStruct, namespace)
|
||||
configPatch = append(configPatch, upstreamConfig...)
|
||||
|
||||
return generateEnvoyFilter(namespace, configPatch), nil
|
||||
}
|
||||
|
||||
@@ -365,6 +410,32 @@ func (g *GlobalOptionController) generateDownstreamEnvoyFilter(downstreamValueSt
|
||||
return downstreamConfig
|
||||
}
|
||||
|
||||
func (g *GlobalOptionController) generateUpstreamEnvoyFilter(upstreamValueStruct string, bufferLimit string, namespace string) []*networking.EnvoyFilter_EnvoyConfigObjectPatch {
|
||||
upstreamConfig := []*networking.EnvoyFilter_EnvoyConfigObjectPatch{
|
||||
{
|
||||
ApplyTo: networking.EnvoyFilter_CLUSTER,
|
||||
Match: &networking.EnvoyFilter_EnvoyConfigObjectMatch{
|
||||
Context: networking.EnvoyFilter_GATEWAY,
|
||||
},
|
||||
Patch: &networking.EnvoyFilter_Patch{
|
||||
Operation: networking.EnvoyFilter_Patch_MERGE,
|
||||
Value: util.BuildPatchStruct(upstreamValueStruct),
|
||||
},
|
||||
},
|
||||
{
|
||||
ApplyTo: networking.EnvoyFilter_CLUSTER,
|
||||
Match: &networking.EnvoyFilter_EnvoyConfigObjectMatch{
|
||||
Context: networking.EnvoyFilter_GATEWAY,
|
||||
},
|
||||
Patch: &networking.EnvoyFilter_Patch{
|
||||
Operation: networking.EnvoyFilter_Patch_MERGE,
|
||||
Value: util.BuildPatchStruct(bufferLimit),
|
||||
},
|
||||
},
|
||||
}
|
||||
return upstreamConfig
|
||||
}
|
||||
|
||||
// generateAddXRealIpHeaderEnvoyFilter generates the add x-real-ip header envoy filter.
|
||||
func (g *GlobalOptionController) generateAddXRealIpHeaderEnvoyFilter(addXRealIpHeaderStruct string, namespace string) []*networking.EnvoyFilter_EnvoyConfigObjectPatch {
|
||||
addXRealIpHeaderConfig := []*networking.EnvoyFilter_EnvoyConfigObjectPatch{
|
||||
@@ -460,6 +531,32 @@ func (g *GlobalOptionController) constructDownstream(downstream *Downstream) str
|
||||
return downstreamConfig
|
||||
}
|
||||
|
||||
// constructUpstream constructs the upstream config.
|
||||
func (g *GlobalOptionController) constructUpstream(upstream *Upstream) string {
|
||||
upstreamConfig := ""
|
||||
idleTimeout := upstream.IdleTimeout
|
||||
|
||||
upstreamConfig = fmt.Sprintf(`
|
||||
{
|
||||
"common_http_protocol_options": {
|
||||
"idleTimeout": "%ds"
|
||||
}
|
||||
}
|
||||
`, idleTimeout)
|
||||
|
||||
return upstreamConfig
|
||||
}
|
||||
|
||||
// constructUpstreamBufferLimit constructs the upstream buffer limit config.
|
||||
func (g *GlobalOptionController) constructUpstreamBufferLimit(upstream *Upstream) string {
|
||||
upstreamBufferLimitStruct := fmt.Sprintf(`
|
||||
{
|
||||
"per_connection_buffer_limit_bytes": %d
|
||||
}
|
||||
`, upstream.ConnectionBufferLimits)
|
||||
return upstreamBufferLimitStruct
|
||||
}
|
||||
|
||||
// constructAddXRealIpHeader constructs the add x-real-ip header config.
|
||||
func (g *GlobalOptionController) constructAddXRealIpHeader() string {
|
||||
addXRealIpHeaderStruct := fmt.Sprintf(`
|
||||
|
||||
@@ -41,6 +41,7 @@ func Test_validGlobal(t *testing.T) {
|
||||
name: "downstream nil",
|
||||
global: &Global{
|
||||
Downstream: nil,
|
||||
Upstream: NewDefaultUpStream(),
|
||||
AddXRealIpHeader: true,
|
||||
DisableXEnvoyHeaders: true,
|
||||
},
|
||||
@@ -131,35 +132,27 @@ func Test_deepCopyGlobal(t *testing.T) {
|
||||
name: "deep copy 2",
|
||||
global: &Global{
|
||||
Downstream: &Downstream{
|
||||
IdleTimeout: 1,
|
||||
IdleTimeout: 0,
|
||||
MaxRequestHeadersKb: 9600,
|
||||
ConnectionBufferLimits: 4096,
|
||||
Http2: NewDefaultHttp2(),
|
||||
},
|
||||
Upstream: &Upstream{
|
||||
IdleTimeout: 10,
|
||||
},
|
||||
AddXRealIpHeader: true,
|
||||
DisableXEnvoyHeaders: true,
|
||||
},
|
||||
want: &Global{
|
||||
Downstream: &Downstream{
|
||||
IdleTimeout: 1,
|
||||
IdleTimeout: 0,
|
||||
MaxRequestHeadersKb: 9600,
|
||||
ConnectionBufferLimits: 4096,
|
||||
Http2: NewDefaultHttp2(),
|
||||
},
|
||||
AddXRealIpHeader: true,
|
||||
DisableXEnvoyHeaders: true,
|
||||
},
|
||||
wantErr: nil,
|
||||
},
|
||||
{
|
||||
name: "deep copy 3",
|
||||
global: &Global{
|
||||
Downstream: &Downstream{},
|
||||
AddXRealIpHeader: true,
|
||||
DisableXEnvoyHeaders: true,
|
||||
},
|
||||
want: &Global{
|
||||
Downstream: NewDefaultDownstream(),
|
||||
Upstream: &Upstream{
|
||||
IdleTimeout: 10,
|
||||
},
|
||||
AddXRealIpHeader: true,
|
||||
DisableXEnvoyHeaders: true,
|
||||
},
|
||||
@@ -203,7 +196,13 @@ func Test_AddOrUpdateHigressConfig(t *testing.T) {
|
||||
old: NewDefaultHigressConfig(),
|
||||
new: &HigressConfig{
|
||||
Downstream: &Downstream{
|
||||
IdleTimeout: 1,
|
||||
IdleTimeout: 1,
|
||||
MaxRequestHeadersKb: defaultMaxRequestHeadersKb,
|
||||
ConnectionBufferLimits: defaultConnectionBufferLimits,
|
||||
Http2: NewDefaultHttp2(),
|
||||
},
|
||||
Upstream: &Upstream{
|
||||
IdleTimeout: 10,
|
||||
},
|
||||
AddXRealIpHeader: true,
|
||||
DisableXEnvoyHeaders: true,
|
||||
@@ -217,6 +216,9 @@ func Test_AddOrUpdateHigressConfig(t *testing.T) {
|
||||
ConnectionBufferLimits: defaultConnectionBufferLimits,
|
||||
Http2: NewDefaultHttp2(),
|
||||
},
|
||||
Upstream: &Upstream{
|
||||
IdleTimeout: 10,
|
||||
},
|
||||
AddXRealIpHeader: true,
|
||||
DisableXEnvoyHeaders: true,
|
||||
},
|
||||
@@ -225,6 +227,7 @@ func Test_AddOrUpdateHigressConfig(t *testing.T) {
|
||||
name: "delete and push",
|
||||
old: &HigressConfig{
|
||||
Downstream: NewDefaultDownstream(),
|
||||
Upstream: NewDefaultUpStream(),
|
||||
AddXRealIpHeader: defaultAddXRealIpHeader,
|
||||
DisableXEnvoyHeaders: defaultDisableXEnvoyHeaders,
|
||||
},
|
||||
@@ -233,6 +236,7 @@ func Test_AddOrUpdateHigressConfig(t *testing.T) {
|
||||
wantEventPush: "push",
|
||||
wantGlobal: &Global{
|
||||
Downstream: NewDefaultDownstream(),
|
||||
Upstream: NewDefaultUpStream(),
|
||||
AddXRealIpHeader: defaultAddXRealIpHeader,
|
||||
DisableXEnvoyHeaders: defaultDisableXEnvoyHeaders,
|
||||
},
|
||||
|
||||
@@ -1279,8 +1279,10 @@ func createRuleKey(annots map[string]string, hostAndPath string) string {
|
||||
if idx := strings.Index(k, annotations.MatchHeader); idx != -1 {
|
||||
key := k[start:idx] + k[idx+len(annotations.MatchHeader)+1:]
|
||||
headers = append(headers, [2]string{key, val})
|
||||
}
|
||||
if idx := strings.Index(k, annotations.MatchQuery); idx != -1 {
|
||||
} else if idx := strings.Index(k, annotations.MatchPseudoHeader); idx != -1 {
|
||||
key := k[start:idx] + ":" + k[idx+len(annotations.MatchPseudoHeader)+1:]
|
||||
headers = append(headers, [2]string{key, val})
|
||||
} else if idx := strings.Index(k, annotations.MatchQuery); idx != -1 {
|
||||
key := k[start:idx] + k[idx+len(annotations.MatchQuery)+1:]
|
||||
params = append(params, [2]string{key, val})
|
||||
}
|
||||
|
||||
@@ -1302,15 +1302,18 @@ func TestCreateRuleKey(t *testing.T) {
|
||||
}
|
||||
|
||||
annots := annotations.Annotations{
|
||||
buildHigressAnnotationKey(annotations.MatchMethod): "GET PUT",
|
||||
buildHigressAnnotationKey("exact-" + annotations.MatchHeader + "-abc"): "123",
|
||||
buildHigressAnnotationKey("prefix-" + annotations.MatchHeader + "-def"): "456",
|
||||
buildHigressAnnotationKey("exact-" + annotations.MatchQuery + "-region"): "beijing",
|
||||
buildHigressAnnotationKey("prefix-" + annotations.MatchQuery + "-user-id"): "user-",
|
||||
buildHigressAnnotationKey(annotations.MatchMethod): "GET PUT",
|
||||
buildHigressAnnotationKey("exact-" + annotations.MatchHeader + "-abc"): "123",
|
||||
buildHigressAnnotationKey("prefix-" + annotations.MatchHeader + "-def"): "456",
|
||||
buildHigressAnnotationKey("exact-" + annotations.MatchPseudoHeader + "-authority"): "foo.bar.com",
|
||||
buildHigressAnnotationKey("prefix-" + annotations.MatchPseudoHeader + "-scheme"): "htt",
|
||||
buildHigressAnnotationKey("exact-" + annotations.MatchQuery + "-region"): "beijing",
|
||||
buildHigressAnnotationKey("prefix-" + annotations.MatchQuery + "-user-id"): "user-",
|
||||
}
|
||||
expect := "higress.com-prefix-/foo" + sep + //host-pathType-path
|
||||
"GET PUT" + sep + // method
|
||||
"exact-abc\t123" + "\n" + "prefix-def\t456" + sep + // header
|
||||
"exact-:authority\tfoo.bar.com" + "\n" + "exact-abc\t123" + "\n" +
|
||||
"prefix-:scheme\thtt" + "\n" + "prefix-def\t456" + sep + // header
|
||||
"exact-region\tbeijing" + "\n" + "prefix-user-id\tuser-" + sep // params
|
||||
|
||||
key := createRuleKey(annots, wrapperHttpRoute.PathFormat())
|
||||
|
||||
@@ -1226,8 +1226,10 @@ func createRuleKey(annots map[string]string, hostAndPath string) string {
|
||||
if idx := strings.Index(k, annotations.MatchHeader); idx != -1 {
|
||||
key := k[start:idx] + k[idx+len(annotations.MatchHeader)+1:]
|
||||
headers = append(headers, [2]string{key, val})
|
||||
}
|
||||
if idx := strings.Index(k, annotations.MatchQuery); idx != -1 {
|
||||
} else if idx := strings.Index(k, annotations.MatchPseudoHeader); idx != -1 {
|
||||
key := k[start:idx] + ":" + k[idx+len(annotations.MatchPseudoHeader)+1:]
|
||||
headers = append(headers, [2]string{key, val})
|
||||
} else if idx := strings.Index(k, annotations.MatchQuery); idx != -1 {
|
||||
key := k[start:idx] + k[idx+len(annotations.MatchQuery)+1:]
|
||||
params = append(params, [2]string{key, val})
|
||||
}
|
||||
|
||||
@@ -699,8 +699,10 @@ func createRuleKey(annots map[string]string, hostAndPath string) string {
|
||||
if idx := strings.Index(k, annotations.MatchHeader); idx != -1 {
|
||||
key := k[start:idx] + k[idx+len(annotations.MatchHeader)+1:]
|
||||
headers = append(headers, [2]string{key, val})
|
||||
}
|
||||
if idx := strings.Index(k, annotations.MatchQuery); idx != -1 {
|
||||
} else if idx := strings.Index(k, annotations.MatchPseudoHeader); idx != -1 {
|
||||
key := k[start:idx] + ":" + k[idx+len(annotations.MatchPseudoHeader)+1:]
|
||||
headers = append(headers, [2]string{key, val})
|
||||
} else if idx := strings.Index(k, annotations.MatchQuery); idx != -1 {
|
||||
key := k[start:idx] + k[idx+len(annotations.MatchQuery)+1:]
|
||||
params = append(params, [2]string{key, val})
|
||||
}
|
||||
|
||||
@@ -581,15 +581,18 @@ func TestCreateRuleKey(t *testing.T) {
|
||||
}
|
||||
|
||||
annots := annotations.Annotations{
|
||||
buildHigressAnnotationKey(annotations.MatchMethod): "GET PUT",
|
||||
buildHigressAnnotationKey("exact-" + annotations.MatchHeader + "-abc"): "123",
|
||||
buildHigressAnnotationKey("prefix-" + annotations.MatchHeader + "-def"): "456",
|
||||
buildHigressAnnotationKey("exact-" + annotations.MatchQuery + "-region"): "beijing",
|
||||
buildHigressAnnotationKey("prefix-" + annotations.MatchQuery + "-user-id"): "user-",
|
||||
buildHigressAnnotationKey(annotations.MatchMethod): "GET PUT",
|
||||
buildHigressAnnotationKey("exact-" + annotations.MatchHeader + "-abc"): "123",
|
||||
buildHigressAnnotationKey("prefix-" + annotations.MatchHeader + "-def"): "456",
|
||||
buildHigressAnnotationKey("exact-" + annotations.MatchPseudoHeader + "-authority"): "foo.bar.com",
|
||||
buildHigressAnnotationKey("prefix-" + annotations.MatchPseudoHeader + "-scheme"): "htt",
|
||||
buildHigressAnnotationKey("exact-" + annotations.MatchQuery + "-region"): "beijing",
|
||||
buildHigressAnnotationKey("prefix-" + annotations.MatchQuery + "-user-id"): "user-",
|
||||
}
|
||||
expect := "higress.com-prefix-/foo" + sep + //host-pathType-path
|
||||
"GET PUT" + sep + // method
|
||||
"exact-abc\t123" + "\n" + "prefix-def\t456" + sep + // header
|
||||
"exact-:authority\tfoo.bar.com" + "\n" + "exact-abc\t123" + "\n" +
|
||||
"prefix-:scheme\thtt" + "\n" + "prefix-def\t456" + sep + // header
|
||||
"exact-region\tbeijing" + "\n" + "prefix-user-id\tuser-" + sep // params
|
||||
|
||||
key := createRuleKey(annots, wrapperHttpRoute.PathFormat())
|
||||
|
||||
23
plugins/wasm-go/.devcontainer/Dockerfile
Normal file
23
plugins/wasm-go/.devcontainer/Dockerfile
Normal file
@@ -0,0 +1,23 @@
|
||||
FROM higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/gateway:1.3.1
|
||||
|
||||
FROM ubuntu:20.04
|
||||
|
||||
RUN apt -y update \
|
||||
&& apt install -y --no-install-recommends python3-pip net-tools vim wget make curl git 2>&1 \
|
||||
&& apt install -y --reinstall ca-certificates \
|
||||
&& apt-get autoremove -y && apt-get clean \
|
||||
&& rm -rf /tmp/* /var/tmp/* \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
ENV PATH=/opt/tinygo/bin:/opt/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
|
||||
|
||||
RUN wget --no-check-certificate https://github.com/tinygo-org/tinygo/releases/download/v0.29.0/tinygo0.29.0.linux-amd64.tar.gz \
|
||||
&& tar -zvxf tinygo0.29.0.linux-amd64.tar.gz -C /opt \
|
||||
&& rm tinygo0.29.0.linux-amd64.tar.gz
|
||||
|
||||
RUN wget --no-check-certificate https://go.dev/dl/go1.19.linux-amd64.tar.gz \
|
||||
&& tar -zvxf go1.19.linux-amd64.tar.gz -C /opt \
|
||||
&& rm go1.19.linux-amd64.tar.gz \
|
||||
&& go install -v golang.org/x/tools/gopls@latest
|
||||
|
||||
COPY --from=0 /usr/local/bin/envoy /usr/local/bin/envoy
|
||||
21
plugins/wasm-go/.devcontainer/devcontainer.json
Normal file
21
plugins/wasm-go/.devcontainer/devcontainer.json
Normal file
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"name": "Wasm Go Dev",
|
||||
// "dockerFile": "Dockerfile",
|
||||
"image": "liuxr25/wasm-go:tinygo-0.29.0",
|
||||
"runArgs": [
|
||||
"--user=root"
|
||||
],
|
||||
"customizations": {
|
||||
"vscode": {
|
||||
"settings": {
|
||||
"terminal.integrated.shell.linux": "/bin/bash",
|
||||
"python.pythonPath": "/usr/bin/python3"
|
||||
},
|
||||
"extensions": [
|
||||
"ms-python.python",
|
||||
"golang.go"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
76
plugins/wasm-go/.devcontainer/gen_config.py
Normal file
76
plugins/wasm-go/.devcontainer/gen_config.py
Normal file
@@ -0,0 +1,76 @@
|
||||
import json
|
||||
import sys
|
||||
|
||||
|
||||
plugin_name = sys.argv[1]
|
||||
|
||||
with open("extensions/"+plugin_name+"/config.json", "r") as f:
|
||||
plugin_config = json.load(f)
|
||||
|
||||
config = f'''static_resources:
|
||||
listeners:
|
||||
- address:
|
||||
socket_address:
|
||||
address: 0.0.0.0
|
||||
port_value: 8080
|
||||
filter_chains:
|
||||
- filters:
|
||||
- name: envoy.filters.network.http_connection_manager
|
||||
typed_config:
|
||||
'@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
|
||||
codec_type: AUTO
|
||||
stat_prefix: ingress_http
|
||||
route_config:
|
||||
name: test
|
||||
virtual_hosts:
|
||||
- name: direct_response_service
|
||||
domains:
|
||||
- "*"
|
||||
routes:
|
||||
- match:
|
||||
prefix: "/"
|
||||
direct_response:
|
||||
status: 200
|
||||
body:
|
||||
inline_string: "hello world\\n"
|
||||
# - match:
|
||||
# prefix: "/"
|
||||
# route:
|
||||
# cluster: service-backend
|
||||
http_filters:
|
||||
- name: {plugin_name}
|
||||
typed_config:
|
||||
"@type": type.googleapis.com/udpa.type.v1.TypedStruct
|
||||
type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
|
||||
value:
|
||||
config:
|
||||
name: wasmdemo
|
||||
vm_config:
|
||||
runtime: envoy.wasm.runtime.v8
|
||||
code:
|
||||
local:
|
||||
filename: ./extensions/{plugin_name}/main.wasm
|
||||
configuration:
|
||||
"@type": "type.googleapis.com/google.protobuf.StringValue"
|
||||
value: '{json.dumps(plugin_config)}'
|
||||
- name: envoy.filters.http.router
|
||||
typed_config:
|
||||
'@type': type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
|
||||
# clusters:
|
||||
# - name: service-backend
|
||||
# connect_timeout: 600s
|
||||
# type: STATIC
|
||||
# lb_policy: ROUND_ROBIN
|
||||
# load_assignment:
|
||||
# cluster_name: service-backend
|
||||
# endpoints:
|
||||
# - lb_endpoints:
|
||||
# - endpoint:
|
||||
# address:
|
||||
# socket_address:
|
||||
# address: 127.0.0.1
|
||||
# port_value: 8000
|
||||
'''
|
||||
|
||||
with open("extensions/"+plugin_name+"/config.yaml", "w") as f:
|
||||
f.write(config)
|
||||
@@ -60,3 +60,16 @@ builder:
|
||||
.
|
||||
@echo ""
|
||||
@echo "image: ${BUILDER}"
|
||||
|
||||
local-build:
|
||||
tinygo build -scheduler=none -target=wasi -gc=custom -tags='custommalloc nottinygc_finalizer' \
|
||||
-o extensions/${PLUGIN_NAME}/main.wasm \
|
||||
extensions/${PLUGIN_NAME}/main.go
|
||||
@echo ""
|
||||
@echo "wasm: extensions/${PLUGIN_NAME}/main.wasm"
|
||||
|
||||
local-run:
|
||||
python3 .devcontainer/gen_config.py ${PLUGIN_NAME}
|
||||
envoy -c extensions/${PLUGIN_NAME}/config.yaml --concurrency 0 --log-level info --component-log-level wasm:debug
|
||||
|
||||
local-all: local-build local-run
|
||||
58
plugins/wasm-go/extensions/bot-detect/README.md
Normal file
58
plugins/wasm-go/extensions/bot-detect/README.md
Normal file
@@ -0,0 +1,58 @@
|
||||
<p>
|
||||
<a href="README_EN.md"> English </a> | 中文
|
||||
</p>
|
||||
|
||||
# 功能说明
|
||||
`bot-detect`插件可以用于识别并阻止互联网爬虫对站点资源的爬取
|
||||
|
||||
# 配置字段
|
||||
|
||||
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|
||||
| -------- | -------- | -------- | -------- | -------- |
|
||||
| allow | array of string | 选填 | - | 配置匹配 User-Agent 请求头的正则表达式,匹配命中时将允许其访问 |
|
||||
| deny | array of string | 选填 | - | 配置匹配 User-Agent 请求头的正则表达式,匹配命中时将屏蔽请求 |
|
||||
| blocked_code | number | 选填 | 403 | 配置请求被屏蔽时返回的 HTTP 状态码 |
|
||||
| blocked_message | string | 选填 | - | 配置请求被屏蔽时返回的 HTTP 应答 Body |
|
||||
|
||||
`allow` 和 `deny` 字段可以均不配置,则执行默认的爬虫判断逻辑,通过配置 `allow` 字段可以将原本命中默认爬虫判断逻辑的请求放行,通过配置 `deny` 字段可以增加额外的爬虫判断逻辑。
|
||||
|
||||
默认的爬虫判断正则表达式集合如下:
|
||||
|
||||
```bash
|
||||
# Bots General matcher 'name/0.0'
|
||||
(?:\/[A-Za-z0-9\.]+|) {0,5}([A-Za-z0-9 \-_\!\[\]:]{0,50}(?:[Aa]rchiver|[Ii]ndexer|[Ss]craper|[Bb]ot|[Ss]pider|[Cc]rawl[a-z]{0,50}))[/ ](\d+)(?:\.(\d+)(?:\.(\d+)|)|)
|
||||
# Bots General matcher 'name 0.0'
|
||||
(?:\/[A-Za-z0-9\.]+|) {0,5}([A-Za-z0-9 \-_\!\[\]:]{0,50}(?:[Aa]rchiver|[Ii]ndexer|[Ss]craper|[Bb]ot|[Ss]pider|[Cc]rawl[a-z]{0,50})) (\d+)(?:\.(\d+)(?:\.(\d+)|)|)
|
||||
# Bots containing spider|scrape|bot(but not CUBOT)|Crawl
|
||||
((?:[A-z0-9]{1,50}|[A-z\-]{1,50} ?|)(?: the |)(?:[Ss][Pp][Ii][Dd][Ee][Rr]|[Ss]crape|[Cc][Rr][Aa][Ww][Ll])[A-z0-9]{0,50})(?:(?:[ /]| v)(\d+)(?:\.(\d+)|)(?:\.(\d+)|)|)
|
||||
# Bots Pattern '/name-0.0'
|
||||
/((?:Ant-)?Nutch|[A-z]+[Bb]ot|[A-z]+[Ss]pider|Axtaris|fetchurl|Isara|ShopSalad|Tailsweep)[ \-](\d+)(?:\.(\d+)(?:\.(\d+))?)?
|
||||
# Bots Pattern 'name/0.0'
|
||||
\b(008|Altresium|Argus|BaiduMobaider|BoardReader|DNSGroup|DataparkSearch|EDI|Goodzer|Grub|INGRID|Infohelfer|LinkedInBot|LOOQ|Nutch|OgScrper|PathDefender|Peew|PostPost|Steeler|Twitterbot|VSE|WebCrunch|WebZIP|Y!J-BR[A-Z]|YahooSeeker|envolk|sproose|wminer)/(\d+)(?:\.(\d+)|)(?:\.(\d+)|)
|
||||
# More bots
|
||||
(CSimpleSpider|Cityreview Robot|CrawlDaddy|CrawlFire|Finderbots|Index crawler|Job Roboter|KiwiStatus Spider|Lijit Crawler|QuerySeekerSpider|ScollSpider|Trends Crawler|USyd-NLP-Spider|SiteCat Webbot|BotName\/\$BotVersion|123metaspider-Bot|1470\.net crawler|50\.nu|8bo Crawler Bot|Aboundex|Accoona-[A-z]{1,30}-Agent|AdsBot-Google(?:-[a-z]{1,30}|)|altavista|AppEngine-Google|archive.{0,30}\.org_bot|archiver|Ask Jeeves|[Bb]ai[Dd]u[Ss]pider(?:-[A-Za-z]{1,30})(?:-[A-Za-z]{1,30}|)|bingbot|BingPreview|blitzbot|BlogBridge|Bloglovin|BoardReader Blog Indexer|BoardReader Favicon Fetcher|boitho.com-dc|BotSeer|BUbiNG|\b\w{0,30}favicon\w{0,30}\b|\bYeti(?:-[a-z]{1,30}|)|Catchpoint(?: bot|)|[Cc]harlotte|Checklinks|clumboot|Comodo HTTP\(S\) Crawler|Comodo-Webinspector-Crawler|ConveraCrawler|CRAWL-E|CrawlConvera|Daumoa(?:-feedfetcher|)|Feed Seeker Bot|Feedbin|findlinks|Flamingo_SearchEngine|FollowSite Bot|furlbot|Genieo|gigabot|GomezAgent|gonzo1|(?:[a-zA-Z]{1,30}-|)Googlebot(?:-[a-zA-Z]{1,30}|)|Google SketchUp|grub-client|gsa-crawler|heritrix|HiddenMarket|holmes|HooWWWer|htdig|ia_archiver|ICC-Crawler|Icarus6j|ichiro(?:/mobile|)|IconSurf|IlTrovatore(?:-Setaccio|)|InfuzApp|Innovazion Crawler|InternetArchive|IP2[a-z]{1,30}Bot|jbot\b|KaloogaBot|Kraken|Kurzor|larbin|LEIA|LesnikBot|Linguee Bot|LinkAider|LinkedInBot|Lite Bot|Llaut|lycos|Mail\.RU_Bot|masscan|masidani_bot|Mediapartners-Google|Microsoft .{0,30} Bot|mogimogi|mozDex|MJ12bot|msnbot(?:-media {0,2}|)|msrbot|Mtps Feed Aggregation System|netresearch|Netvibes|NewsGator[^/]{0,30}|^NING|Nutch[^/]{0,30}|Nymesis|ObjectsSearch|OgScrper|Orbiter|OOZBOT|PagePeeker|PagesInventory|PaxleFramework|Peeplo Screenshot Bot|PlantyNet_WebRobot|Pompos|Qwantify|Read%20Later|Reaper|RedCarpet|Retreiver|Riddler|Rival IQ|scooter|Scrapy|Scrubby|searchsight|seekbot|semanticdiscovery|SemrushBot|Simpy|SimplePie|SEOstats|SimpleRSS|SiteCon|Slackbot-LinkExpanding|Slack-ImgProxy|Slurp|snappy|Speedy Spider|Squrl Java|Stringer|TheUsefulbot|ThumbShotsBot|Thumbshots\.ru|Tiny Tiny RSS|Twitterbot|WhatsApp|URL2PNG|Vagabondo|VoilaBot|^vortex|Votay bot|^voyager|WASALive.Bot|Web-sniffer|WebThumb|WeSEE:[A-z]{1,30}|WhatWeb|WIRE|WordPress|Wotbox|www\.almaden\.ibm\.com|Xenu(?:.s|) Link Sleuth|Xerka [A-z]{1,30}Bot|yacy(?:bot|)|YahooSeeker|Yahoo! Slurp|Yandex\w{1,30}|YodaoBot(?:-[A-z]{1,30}|)|YottaaMonitor|Yowedo|^Zao|^Zao-Crawler|ZeBot_www\.ze\.bz|ZooShot|ZyBorg)(?:[ /]v?(\d+)(?:\.(\d+)(?:\.(\d+)|)|)|)
|
||||
```
|
||||
|
||||
# 配置示例
|
||||
|
||||
## 放行原本命中爬虫规则的请求
|
||||
```yaml
|
||||
allow:
|
||||
- ".*Go-http-client.*"
|
||||
```
|
||||
|
||||
若不作该配置,默认的 Golang 网络库请求会被视做爬虫,被禁止访问
|
||||
|
||||
|
||||
## 增加爬虫判断
|
||||
```yaml
|
||||
deny:
|
||||
- "spd-tools.*"
|
||||
```
|
||||
|
||||
根据该配置,下列请求将被禁止访问:
|
||||
|
||||
```bash
|
||||
curl http://example.com -H 'User-Agent: spd-tools/1.1'
|
||||
curl http://exmaple.com -H 'User-Agent: spd-tools'
|
||||
```
|
||||
58
plugins/wasm-go/extensions/bot-detect/README_EN.md
Normal file
58
plugins/wasm-go/extensions/bot-detect/README_EN.md
Normal file
@@ -0,0 +1,58 @@
|
||||
<p>
|
||||
English | <a href="README.md">中文</a>
|
||||
</p>
|
||||
|
||||
# Description
|
||||
`bot-detect` plugin can be used to identify and prevent web crawlers from crawling websites.
|
||||
|
||||
# Configuration Fields
|
||||
|
||||
| Name | Type | Requirement | Default Value | Description |
|
||||
| -------- | -------- | -------- | -------- | -------- |
|
||||
| allow | array of string | Optional | - | A regular expression to match the User-Agent request header and will allow access if the match hits |
|
||||
| deny | array of string | Optional | - | A regular expression to match the User-Agent request header and will block the request if the match hits |
|
||||
| blocked_code | number | Optional | 403 | The HTTP status code returned when a request is blocked |
|
||||
| blocked_message | string | Optional | - | The HTTP response Body returned when a request is blocked |
|
||||
|
||||
If field `allow` and field `deny` are not configured at the same time, the default logic to identify crawlers will be executed. By configuring the `allow` field, requests that would otherwise hit the default logic can be allowed. The judgement can be extended by configuring the `deny` field
|
||||
|
||||
The default set of crawler judgment regular expressions is as follows:
|
||||
|
||||
```bash
|
||||
# Bots General matcher 'name/0.0'
|
||||
(?:\/[A-Za-z0-9\.]+|) {0,5}([A-Za-z0-9 \-_\!\[\]:]{0,50}(?:[Aa]rchiver|[Ii]ndexer|[Ss]craper|[Bb]ot|[Ss]pider|[Cc]rawl[a-z]{0,50}))[/ ](\d+)(?:\.(\d+)(?:\.(\d+)|)|)
|
||||
# Bots General matcher 'name 0.0'
|
||||
(?:\/[A-Za-z0-9\.]+|) {0,5}([A-Za-z0-9 \-_\!\[\]:]{0,50}(?:[Aa]rchiver|[Ii]ndexer|[Ss]craper|[Bb]ot|[Ss]pider|[Cc]rawl[a-z]{0,50})) (\d+)(?:\.(\d+)(?:\.(\d+)|)|)
|
||||
# Bots containing spider|scrape|bot(but not CUBOT)|Crawl
|
||||
((?:[A-z0-9]{1,50}|[A-z\-]{1,50} ?|)(?: the |)(?:[Ss][Pp][Ii][Dd][Ee][Rr]|[Ss]crape|[Cc][Rr][Aa][Ww][Ll])[A-z0-9]{0,50})(?:(?:[ /]| v)(\d+)(?:\.(\d+)|)(?:\.(\d+)|)|)
|
||||
# Bots Pattern '/name-0.0'
|
||||
/((?:Ant-)?Nutch|[A-z]+[Bb]ot|[A-z]+[Ss]pider|Axtaris|fetchurl|Isara|ShopSalad|Tailsweep)[ \-](\d+)(?:\.(\d+)(?:\.(\d+))?)?
|
||||
# Bots Pattern 'name/0.0'
|
||||
\b(008|Altresium|Argus|BaiduMobaider|BoardReader|DNSGroup|DataparkSearch|EDI|Goodzer|Grub|INGRID|Infohelfer|LinkedInBot|LOOQ|Nutch|OgScrper|PathDefender|Peew|PostPost|Steeler|Twitterbot|VSE|WebCrunch|WebZIP|Y!J-BR[A-Z]|YahooSeeker|envolk|sproose|wminer)/(\d+)(?:\.(\d+)|)(?:\.(\d+)|)
|
||||
# More bots
|
||||
(CSimpleSpider|Cityreview Robot|CrawlDaddy|CrawlFire|Finderbots|Index crawler|Job Roboter|KiwiStatus Spider|Lijit Crawler|QuerySeekerSpider|ScollSpider|Trends Crawler|USyd-NLP-Spider|SiteCat Webbot|BotName\/\$BotVersion|123metaspider-Bot|1470\.net crawler|50\.nu|8bo Crawler Bot|Aboundex|Accoona-[A-z]{1,30}-Agent|AdsBot-Google(?:-[a-z]{1,30}|)|altavista|AppEngine-Google|archive.{0,30}\.org_bot|archiver|Ask Jeeves|[Bb]ai[Dd]u[Ss]pider(?:-[A-Za-z]{1,30})(?:-[A-Za-z]{1,30}|)|bingbot|BingPreview|blitzbot|BlogBridge|Bloglovin|BoardReader Blog Indexer|BoardReader Favicon Fetcher|boitho.com-dc|BotSeer|BUbiNG|\b\w{0,30}favicon\w{0,30}\b|\bYeti(?:-[a-z]{1,30}|)|Catchpoint(?: bot|)|[Cc]harlotte|Checklinks|clumboot|Comodo HTTP\(S\) Crawler|Comodo-Webinspector-Crawler|ConveraCrawler|CRAWL-E|CrawlConvera|Daumoa(?:-feedfetcher|)|Feed Seeker Bot|Feedbin|findlinks|Flamingo_SearchEngine|FollowSite Bot|furlbot|Genieo|gigabot|GomezAgent|gonzo1|(?:[a-zA-Z]{1,30}-|)Googlebot(?:-[a-zA-Z]{1,30}|)|Google SketchUp|grub-client|gsa-crawler|heritrix|HiddenMarket|holmes|HooWWWer|htdig|ia_archiver|ICC-Crawler|Icarus6j|ichiro(?:/mobile|)|IconSurf|IlTrovatore(?:-Setaccio|)|InfuzApp|Innovazion Crawler|InternetArchive|IP2[a-z]{1,30}Bot|jbot\b|KaloogaBot|Kraken|Kurzor|larbin|LEIA|LesnikBot|Linguee Bot|LinkAider|LinkedInBot|Lite Bot|Llaut|lycos|Mail\.RU_Bot|masscan|masidani_bot|Mediapartners-Google|Microsoft .{0,30} Bot|mogimogi|mozDex|MJ12bot|msnbot(?:-media {0,2}|)|msrbot|Mtps Feed Aggregation System|netresearch|Netvibes|NewsGator[^/]{0,30}|^NING|Nutch[^/]{0,30}|Nymesis|ObjectsSearch|OgScrper|Orbiter|OOZBOT|PagePeeker|PagesInventory|PaxleFramework|Peeplo Screenshot Bot|PlantyNet_WebRobot|Pompos|Qwantify|Read%20Later|Reaper|RedCarpet|Retreiver|Riddler|Rival IQ|scooter|Scrapy|Scrubby|searchsight|seekbot|semanticdiscovery|SemrushBot|Simpy|SimplePie|SEOstats|SimpleRSS|SiteCon|Slackbot-LinkExpanding|Slack-ImgProxy|Slurp|snappy|Speedy Spider|Squrl Java|Stringer|TheUsefulbot|ThumbShotsBot|Thumbshots\.ru|Tiny Tiny RSS|Twitterbot|WhatsApp|URL2PNG|Vagabondo|VoilaBot|^vortex|Votay bot|^voyager|WASALive.Bot|Web-sniffer|WebThumb|WeSEE:[A-z]{1,30}|WhatWeb|WIRE|WordPress|Wotbox|www\.almaden\.ibm\.com|Xenu(?:.s|) Link Sleuth|Xerka [A-z]{1,30}Bot|yacy(?:bot|)|YahooSeeker|Yahoo! Slurp|Yandex\w{1,30}|YodaoBot(?:-[A-z]{1,30}|)|YottaaMonitor|Yowedo|^Zao|^Zao-Crawler|ZeBot_www\.ze\.bz|ZooShot|ZyBorg)(?:[ /]v?(\d+)(?:\.(\d+)(?:\.(\d+)|)|)|)
|
||||
```
|
||||
|
||||
# Configuration Samples
|
||||
|
||||
## Release Requests that would otherwise Hit the Crawler Rules
|
||||
```yaml
|
||||
allow:
|
||||
- ".*Go-http-client.*"
|
||||
```
|
||||
|
||||
Without this configuration, the default Golang web library request will be treated as a crawler and access will be denied.
|
||||
|
||||
|
||||
## Add Crawler Judgement
|
||||
```yaml
|
||||
deny:
|
||||
- "spd-tools.*"
|
||||
```
|
||||
|
||||
According to this configuration, the following requests will be denied:
|
||||
|
||||
```bash
|
||||
curl http://example.com -H 'User-Agent: spd-tools/1.1'
|
||||
curl http://exmaple.com -H 'User-Agent: spd-tools'
|
||||
```
|
||||
1
plugins/wasm-go/extensions/bot-detect/VERSION
Normal file
1
plugins/wasm-go/extensions/bot-detect/VERSION
Normal file
@@ -0,0 +1 @@
|
||||
1.0.0
|
||||
30
plugins/wasm-go/extensions/bot-detect/botdetect.yaml
Normal file
30
plugins/wasm-go/extensions/bot-detect/botdetect.yaml
Normal file
@@ -0,0 +1,30 @@
|
||||
apiVersion: extensions.higress.io/v1alpha1
|
||||
kind: WasmPlugin
|
||||
metadata:
|
||||
annotations:
|
||||
higress.io/wasm-plugin-description: 用于识别并阻止互联网爬虫对站点资源的爬取
|
||||
higress.io/wasm-plugin-title: Bot Detect
|
||||
creationTimestamp: '2024-01-03T10:34:36Z'
|
||||
generation: 2
|
||||
labels:
|
||||
higress.io/resource-definer: higress
|
||||
higress.io/wasm-plugin-built-in: 'true'
|
||||
higress.io/wasm-plugin-category: custom
|
||||
higress.io/wasm-plugin-name: bot-detect
|
||||
higress.io/wasm-plugin-version: 1.0.0
|
||||
name: bot-detect
|
||||
namespace: higress-system
|
||||
spec:
|
||||
defaultConfigDisable: true
|
||||
matchRules:
|
||||
- config:
|
||||
blocked_code: 401
|
||||
blocked_message: a bot
|
||||
deny:
|
||||
- Chrome
|
||||
configDisable: false
|
||||
ingress:
|
||||
- test
|
||||
phase: AUTHN
|
||||
priority: 310
|
||||
url: oci://higress-registry.cn-hangzhou.cr.aliyuncs.com/20240103/bot-detect:1.0.0
|
||||
@@ -0,0 +1,68 @@
|
||||
/*
|
||||
* Copyright (c) 2022 Alibaba Group Holding Ltd.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package config
|
||||
|
||||
import (
|
||||
regexp "github.com/wasilibs/go-re2"
|
||||
)
|
||||
|
||||
var DefaultBotRegex = []*regexp.Regexp{
|
||||
regexp.MustCompile(`(\/[A-Za-z0-9\.]+|) {0,5}([A-Za-z0-9 \-_\!\[\]:]{0,50}([Aa]rchiver|[Ii]ndexer|[Ss]craper|[Bb]ot|[Ss]pider|[Cc]rawl[a-z]{0,50}))[/ ](\d+)(\.(\d+)(\.(\d+)|)|)`),
|
||||
regexp.MustCompile(`((\/[A-Za-z0-9\.]+|) {0,5}([A-Za-z0-9 \-_\!\[\]:]{0,50}([Aa]rchiver|[Ii]ndexer|[Ss]craper|[Bb]ot|[Ss]pider|[Cc]rawl[a-z]{0,50})) (\d+)(\.(\d+)(\.(\d+)|)|))`),
|
||||
regexp.MustCompile(`((([A-z0-9]{1,50}|[A-z\-]{1,} ?|)( the |)(?:[Ss][Pp][Ii][Dd][Ee][Rr]|[Ss]crape|[Cc][Rr][Aa][Ww][Ll])[A-z0-9]{0,50})(([ /]| v)(\d+)(\.(\d+)|)(\.(\d+)|)|))`),
|
||||
regexp.MustCompile(`((Ant-)?Nutch|[A-z]+[Bb]ot|[A-z]+[Ss]pider|Axtaris|fetchurl|Isara|ShopSalad|Tailsweep)[ \-](\d+)(\.(\d+)(\.(\d+))?)?`),
|
||||
regexp.MustCompile(`\b(008|Altresium|Argus|BaiduMobaider|BoardReader|DNSGroup|DataparkSearch|EDI|Goodzer|Grub|INGRID|Infohelfer|LinkedInBot|LOOQ|Nutch|OgScrper|PathDefender|Peew|PostPost|Steeler|Twitterbot|VSE|WebCrunch|WebZIP|Y!J-BR[A-Z]|YahooSeeker|envolk|sproose|wminer)/(\d+)(?:\.(\d+)|)(?:\.(\d+)|)`),
|
||||
regexp.MustCompile(`((CSimpleSpider|Cityreview Robot|CrawlDaddy|CrawlFire|Finderbots|Index crawler|Job Roboter|KiwiStatus Spider|Lijit Crawler|QuerySeekerSpider|ScollSpider|Trends Crawler|USyd-NLP-Spider|SiteCat Webbot|BotName\/\$BotVersion|123metaspider-Bot|1470\.net crawler|50\.nu|8bo Crawler Bot|Aboundex|Accoona-[A-z]{1,30}-Agent|AdsBot-Google(?:-[a-z]{1,30}|)|altavista|AppEngine-Google|archive.{0,30}\.org_bot|archiver|Ask Jeeves|[Bb]ai[Dd]u[Ss]pider(?:-[A-Za-z]{1,30})(?:-[A-Za-z]{1,30}|)|bingbot|BingPreview|blitzbot|BlogBridge|Bloglovin|BoardReader Blog Indexer|BoardReader Favicon Fetcher|boitho.com-dc|BotSeer|BUbiNG|\b\w{0,30}favicon\w{0,30}\b|\bYeti(?:-[a-z]{1,30}|)|Catchpoint(?: bot|)|[Cc]harlotte|Checklinks|clumboot|Comodo HTTP\(S\) Crawler|Comodo-Webinspector-Crawler|ConveraCrawler|CRAWL-E|CrawlConvera|Daumoa(?:-feedfetcher|)|Feed Seeker Bot|Feedbin|findlinks|Flamingo_SearchEngine|FollowSite Bot|furlbot|Genieo|gigabot|GomezAgent|gonzo1|(?:[a-zA-Z]{1,30}-|)Googlebot(?:-[a-zA-Z]{1,30}|)|Google SketchUp|grub-client|gsa-crawler|heritrix|HiddenMarket|holmes|HooWWWer|htdig|ia_archiver|ICC-Crawler|Icarus6j|ichiro(?:/mobile|)|IconSurf|IlTrovatore(?:-Setaccio|)|InfuzApp|Innovazion Crawler|InternetArchive|IP2[a-z]{1,30}Bot|jbot\b|KaloogaBot|Kraken|Kurzor|larbin|LEIA|LesnikBot|Linguee Bot|LinkAider|LinkedInBot|Lite Bot|Llaut|lycos|Mail\.RU_Bot|masscan|masidani_bot|Mediapartners-Google|Microsoft .{0,30} Bot|mogimogi|mozDex|MJ12bot|msnbot(?:-media {0,2}|)|msrbot|Mtps Feed Aggregation System|netresearch|Netvibes|NewsGator[^/]{0,30}|^NING|Nutch[^/]{0,30}|Nymesis|ObjectsSearch|OgScrper|Orbiter|OOZBOT|PagePeeker|PagesInventory|PaxleFramework|Peeplo Screenshot Bot|PlantyNet_WebRobot|Pompos|Qwantify|Read%20Later|Reaper|RedCarpet|Retreiver|Riddler|Rival IQ|scooter|Scrapy|Scrubby|searchsight|seekbot|semanticdiscovery|SemrushBot|Simpy|SimplePie|SEOstats|SimpleRSS|SiteCon|Slackbot-LinkExpanding|Slack-ImgProxy|Slurp|snappy|Speedy Spider|Squrl Java|Stringer|TheUsefulbot|ThumbShotsBot|Thumbshots\.ru|Tiny Tiny RSS|Twitterbot|WhatsApp|URL2PNG|Vagabondo|VoilaBot|^vortex|Votay bot|^voyager|WASALive.Bot|Web-sniffer|WebThumb|WeSEE:[A-z]{1,30}|WhatWeb|WIRE|WordPress|Wotbox|www\.almaden\.ibm\.com|Xenu(?:.s|) Link Sleuth|Xerka [A-z]{1,30}Bot|yacy(?:bot|)|YahooSeeker|Yahoo! Slurp|Yandex\w{1,30}|YodaoBot(?:-[A-z]{1,30}|)|YottaaMonitor|Yowedo|^Zao|^Zao-Crawler|ZeBot_www\.ze\.bz|ZooShot|ZyBorg)(?:[ /]v?(\d+)(?:\.(\d+)(?:\.(\d+)|)|)|))`),
|
||||
}
|
||||
|
||||
type BotDetectConfig struct {
|
||||
BlockedCode uint32 `json:"blocked_code"`
|
||||
BlockedMessage string `json:"blocked_message"`
|
||||
Allow []*regexp.Regexp `json:"allow"`
|
||||
Deny []*regexp.Regexp `json:"deny"`
|
||||
}
|
||||
|
||||
func (bdc *BotDetectConfig) FillDefaultValue() {
|
||||
if bdc.BlockedCode == 0 {
|
||||
bdc.BlockedCode = 403
|
||||
}
|
||||
if bdc.BlockedMessage == "" {
|
||||
bdc.BlockedMessage = "Invalid User-Agent"
|
||||
}
|
||||
}
|
||||
|
||||
func (bdc *BotDetectConfig) Process(ua string) (bool, string) {
|
||||
if ua == "" {
|
||||
return false, "can not be empty"
|
||||
}
|
||||
for _, allowRule := range bdc.Allow {
|
||||
if allowRule.MatchString(ua) {
|
||||
return true, ""
|
||||
}
|
||||
}
|
||||
for _, denyRule := range bdc.Deny {
|
||||
if denyRule.MatchString(ua) {
|
||||
return false, denyRule.String()
|
||||
}
|
||||
}
|
||||
for _, defaultRule := range DefaultBotRegex {
|
||||
if defaultRule.MatchString(ua) {
|
||||
return false, defaultRule.String()
|
||||
}
|
||||
}
|
||||
return true, ""
|
||||
}
|
||||
@@ -0,0 +1,138 @@
|
||||
/*
|
||||
* Copyright (c) 2022 Alibaba Group Holding Ltd.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package config
|
||||
|
||||
import (
|
||||
"github.com/stretchr/testify/assert"
|
||||
regexp "github.com/wasilibs/go-re2"
|
||||
"log"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func toRegexMatch(regexs []string) []*regexp.Regexp {
|
||||
re := make([]*regexp.Regexp, 0)
|
||||
for _, regex := range regexs {
|
||||
c, err := regexp.Compile(regex)
|
||||
if err != nil {
|
||||
log.Default().Fatal(err.Error())
|
||||
}
|
||||
re = append(re, c)
|
||||
}
|
||||
return re
|
||||
}
|
||||
|
||||
func TestBotDetectConfig_ProcessTest(t *testing.T) {
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
ua string
|
||||
allow []string
|
||||
deny []string
|
||||
blockCode uint32
|
||||
blockMessage string
|
||||
want bool
|
||||
}{
|
||||
{
|
||||
"test empty bot detect",
|
||||
"",
|
||||
[]string{},
|
||||
[]string{},
|
||||
401,
|
||||
"bot has been blocked",
|
||||
false,
|
||||
},
|
||||
{
|
||||
"test default bot detect",
|
||||
"Ant-Tailsweep-1",
|
||||
[]string{},
|
||||
[]string{},
|
||||
401,
|
||||
"bot has been blocked",
|
||||
false,
|
||||
},
|
||||
{
|
||||
"test default bot detect",
|
||||
"indexer/1.2",
|
||||
[]string{},
|
||||
[]string{},
|
||||
401,
|
||||
"bot has been blocked",
|
||||
false,
|
||||
},
|
||||
{
|
||||
"test default bot detect",
|
||||
"indexer/1.1.0",
|
||||
[]string{},
|
||||
[]string{},
|
||||
401,
|
||||
"bot has been blocked",
|
||||
false,
|
||||
},
|
||||
{
|
||||
"test default bot detect",
|
||||
"YottaaMonitor",
|
||||
[]string{},
|
||||
[]string{},
|
||||
401,
|
||||
"bot has been blocked",
|
||||
false,
|
||||
},
|
||||
{
|
||||
"test allow bot detect",
|
||||
"BaiduMobaider",
|
||||
[]string{"BaiduMobaider"},
|
||||
[]string{},
|
||||
401,
|
||||
"bot has been blocked",
|
||||
true,
|
||||
},
|
||||
{
|
||||
"test deny bot detect",
|
||||
"Chrome",
|
||||
[]string{},
|
||||
[]string{"Chrome"},
|
||||
401,
|
||||
"bot has been blocked",
|
||||
false,
|
||||
},
|
||||
{
|
||||
"test allow and deny bot detect",
|
||||
"SameBotDetect",
|
||||
[]string{"SameBotDetect"},
|
||||
[]string{"SameBotDetect"},
|
||||
401,
|
||||
"bot has been blocked",
|
||||
true,
|
||||
},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
bdc := BotDetectConfig{
|
||||
BlockedCode: test.blockCode,
|
||||
BlockedMessage: test.blockMessage,
|
||||
Allow: toRegexMatch(test.allow),
|
||||
Deny: toRegexMatch(test.deny),
|
||||
}
|
||||
actual, _ := bdc.Process(test.ua)
|
||||
assert.Equal(t, test.want, actual, "")
|
||||
})
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
23
plugins/wasm-go/extensions/bot-detect/go.mod
Normal file
23
plugins/wasm-go/extensions/bot-detect/go.mod
Normal file
@@ -0,0 +1,23 @@
|
||||
module bot-detect
|
||||
|
||||
go 1.19
|
||||
|
||||
require (
|
||||
github.com/alibaba/higress/plugins/wasm-go v1.3.2
|
||||
github.com/stretchr/testify v1.8.0
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c
|
||||
github.com/tidwall/gjson v1.14.3
|
||||
github.com/wasilibs/go-re2 v1.4.1
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/davecgh/go-spew v1.1.1 // indirect
|
||||
github.com/google/uuid v1.3.0 // indirect
|
||||
github.com/higress-group/nottinygc v0.0.0-20231101025119-e93c4c2f8520 // indirect
|
||||
github.com/magefile/mage v1.15.0 // indirect
|
||||
github.com/pmezard/go-difflib v1.0.0 // indirect
|
||||
github.com/tetratelabs/wazero v1.6.0 // indirect
|
||||
github.com/tidwall/match v1.1.1 // indirect
|
||||
github.com/tidwall/pretty v1.2.0 // indirect
|
||||
gopkg.in/yaml.v3 v3.0.1 // indirect
|
||||
)
|
||||
36
plugins/wasm-go/extensions/bot-detect/go.sum
Normal file
36
plugins/wasm-go/extensions/bot-detect/go.sum
Normal file
@@ -0,0 +1,36 @@
|
||||
github.com/alibaba/higress/plugins/wasm-go v1.3.2 h1:OKFo9zK7PFxvtSq9TmT8TwI6xqmNq5LZXfDBqPLOgkw=
|
||||
github.com/alibaba/higress/plugins/wasm-go v1.3.2/go.mod h1:WZ/68vwe8qWhusa6C4/gMwUqas0jvHWSOa1bp8iK8F4=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
|
||||
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/higress-group/nottinygc v0.0.0-20231101025119-e93c4c2f8520 h1:IHDghbGQ2DTIXHBHxWfqCYQW1fKjyJ/I7W1pMyUDeEA=
|
||||
github.com/higress-group/nottinygc v0.0.0-20231101025119-e93c4c2f8520/go.mod h1:Nz8ORLaFiLWotg6GeKlJMhv8cci8mM43uEnLA5t8iew=
|
||||
github.com/magefile/mage v1.15.0 h1:BvGheCMAsG3bWUDbZ8AyXXpCNwU9u5CB6sM+HNb9HYg=
|
||||
github.com/magefile/mage v1.15.0/go.mod h1:z5UZb/iS3GoOSn0JgWuiw7dxlurVYTu+/jHXqQg881A=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
|
||||
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk=
|
||||
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c h1:OCUFXVGixHLfNjg6/QYEhv+jHJ5mRGhpEUVFv9eWPJE=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c/go.mod h1:5t/pWFNJ9eMyu/K/Z+OeGhDJ9sN9eCo8fc2pyM/Qjg4=
|
||||
github.com/tetratelabs/wazero v1.6.0 h1:z0H1iikCdP8t+q341xqepY4EWvHEw8Es7tlqiVzlP3g=
|
||||
github.com/tetratelabs/wazero v1.6.0/go.mod h1:0U0G41+ochRKoPKCJlh0jMg1CHkyfK8kDqiirMmKY8A=
|
||||
github.com/tidwall/gjson v1.14.3 h1:9jvXn7olKEHU1S9vwoMGliaT8jq1vJ7IH/n9zD9Dnlw=
|
||||
github.com/tidwall/gjson v1.14.3/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
|
||||
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
|
||||
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
|
||||
github.com/tidwall/pretty v1.2.0 h1:RWIZEg2iJ8/g6fDDYzMpobmaoGh5OLl4AXtGUGPcqCs=
|
||||
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
|
||||
github.com/wasilibs/go-re2 v1.4.1 h1:E5+9O1M8UoGeqLB2A9omeoaWImqpuYDs9cKwvTJq/Oo=
|
||||
github.com/wasilibs/go-re2 v1.4.1/go.mod h1:ynB8eCwd9JsqUnsk8WlPDk6cEeme8BguZmnqOSURE4Y=
|
||||
github.com/wasilibs/nottinygc v0.4.0 h1:h1TJMihMC4neN6Zq+WKpLxgd9xCFMw7O9ETLwY2exJQ=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
104
plugins/wasm-go/extensions/bot-detect/main.go
Normal file
104
plugins/wasm-go/extensions/bot-detect/main.go
Normal file
@@ -0,0 +1,104 @@
|
||||
/*
|
||||
* Copyright (c) 2022 Alibaba Group Holding Ltd.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"bot-detect/config"
|
||||
"github.com/alibaba/higress/plugins/wasm-go/pkg/wrapper"
|
||||
"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm"
|
||||
"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm/types"
|
||||
"github.com/tidwall/gjson"
|
||||
regexp "github.com/wasilibs/go-re2"
|
||||
)
|
||||
|
||||
func main() {
|
||||
wrapper.SetCtx(
|
||||
"bot-detect",
|
||||
wrapper.ParseConfigBy(parseConfig),
|
||||
wrapper.ProcessRequestHeadersBy(onHttpRequestHeaders),
|
||||
)
|
||||
}
|
||||
|
||||
func parseConfig(json gjson.Result, botDetectConfig *config.BotDetectConfig, log wrapper.Log) error {
|
||||
log.Debug("parseConfig()")
|
||||
|
||||
if json.Get("blocked_code").Exists() {
|
||||
botDetectConfig.BlockedCode = uint32(int(json.Get("blocked_code").Int()))
|
||||
}
|
||||
|
||||
if json.Get("blocked_message").Exists() {
|
||||
botDetectConfig.BlockedMessage = json.Get("blocked_message").String()
|
||||
}
|
||||
|
||||
allowRules := make([]gjson.Result, 0)
|
||||
denyRules := make([]gjson.Result, 0)
|
||||
|
||||
allowRulesValue := json.Get("allow")
|
||||
if allowRulesValue.Exists() && allowRulesValue.IsArray() {
|
||||
allowRules = json.Get("allow").Array()
|
||||
}
|
||||
|
||||
denyRulesValue := json.Get("deny")
|
||||
if denyRulesValue.Exists() && denyRulesValue.IsArray() {
|
||||
denyRules = json.Get("deny").Array()
|
||||
}
|
||||
|
||||
for _, allowRule := range allowRules {
|
||||
c, err := regexp.Compile(allowRule.String())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
botDetectConfig.Allow = append(botDetectConfig.Allow, c)
|
||||
}
|
||||
|
||||
for _, denyRule := range denyRules {
|
||||
c, err := regexp.Compile(denyRule.String())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
botDetectConfig.Deny = append(botDetectConfig.Deny, c)
|
||||
}
|
||||
|
||||
// Fill default values
|
||||
botDetectConfig.FillDefaultValue()
|
||||
log.Debugf("botDetectConfig:%+v", botDetectConfig)
|
||||
return nil
|
||||
|
||||
}
|
||||
|
||||
func onHttpRequestHeaders(ctx wrapper.HttpContext, botDetectConfig config.BotDetectConfig, log wrapper.Log) types.Action {
|
||||
log.Debug("onHttpRequestHeaders()")
|
||||
//// Get user-agent header
|
||||
ua, err := proxywasm.GetHttpRequestHeader("user-agent")
|
||||
if err != nil {
|
||||
log.Warnf("failed to get user-agent: %v", err)
|
||||
return types.ActionPause
|
||||
}
|
||||
host := ctx.Host()
|
||||
scheme := ctx.Scheme()
|
||||
path := ctx.Path()
|
||||
method := ctx.Method()
|
||||
|
||||
if ok, rule := botDetectConfig.Process(ua); !ok {
|
||||
proxywasm.SendHttpResponse(botDetectConfig.BlockedCode, nil, []byte(botDetectConfig.BlockedMessage), -1)
|
||||
log.Debugf("scheme:%s, host:%s, method:%s, path:%s user-agent:%s has been blocked by rule:%s", scheme, host, method, path, ua, rule)
|
||||
return types.ActionPause
|
||||
}
|
||||
|
||||
log.Debugf("scheme:%s, host:%s, method:%s, path:%s user-agent:%s has been passed", scheme, host, method, path, ua)
|
||||
return types.ActionContinue
|
||||
}
|
||||
84
plugins/wasm-go/extensions/custom-response/README.md
Normal file
84
plugins/wasm-go/extensions/custom-response/README.md
Normal file
@@ -0,0 +1,84 @@
|
||||
<p>
|
||||
<a href="README_EN.md"> English </a> | 中文
|
||||
</p>
|
||||
|
||||
# 功能说明
|
||||
`custom-response`插件支持配置自定义的响应,包括自定义 HTTP 应答状态码、HTTP 应答头,以及 HTTP 应答 Body。可以用于 Mock 响应,也可以用于判断特定状态码后给出自定义应答,例如在触发网关限流策略时实现自定义响应。
|
||||
|
||||
# 配置字段
|
||||
|
||||
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|
||||
| -------- | -------- | -------- | -------- | -------- |
|
||||
| status_code | number | 选填 | 200 | 自定义 HTTP 应答状态码 |
|
||||
| headers | array of string | 选填 | - | 自定义 HTTP 应答头,key 和 value 用`=`分隔 |
|
||||
| body | string | 选填 | - | 自定义 HTTP 应答 Body |
|
||||
| enable_on_status | array of number | 选填 | - | 匹配原始状态码,生成自定义响应,不填写时,不判断原始状态码 |
|
||||
|
||||
# 配置示例
|
||||
|
||||
## Mock 应答场景
|
||||
|
||||
```yaml
|
||||
status_code: 200
|
||||
headers:
|
||||
- Content-Type=application/json
|
||||
- Hello=World
|
||||
body: "{\"hello\":\"world\"}"
|
||||
|
||||
```
|
||||
|
||||
根据该配置,请求将返回自定义应答如下:
|
||||
|
||||
```text
|
||||
HTTP/1.1 200 OK
|
||||
Content-Type: application/json
|
||||
Hello: World
|
||||
Content-Length: 17
|
||||
|
||||
{"hello":"world"}
|
||||
```
|
||||
|
||||
## 触发限流时自定义响应
|
||||
|
||||
```yaml
|
||||
enable_on_status:
|
||||
- 429
|
||||
status_code: 302
|
||||
headers:
|
||||
- Location=https://example.com
|
||||
```
|
||||
|
||||
触发网关限流时一般会返回 `429` 状态码,这时请求将返回自定义应答如下:
|
||||
|
||||
```text
|
||||
HTTP/1.1 302 Found
|
||||
Location: https://example.com
|
||||
```
|
||||
|
||||
从而实现基于浏览器 302 重定向机制,将限流后的用户引导到其他页面,比如可以是一个 CDN 上的静态页面。
|
||||
|
||||
如果希望触发限流时,正常返回其他应答,参考 Mock 应答场景配置相应的字段即可。
|
||||
|
||||
## 对特定路由或域名开启
|
||||
```yaml
|
||||
# 使用 matchRules 字段进行细粒度规则配置
|
||||
matchRules:
|
||||
# 规则一:按 Ingress 名称匹配生效
|
||||
- ingress:
|
||||
- default/foo
|
||||
- default/bar
|
||||
body: "{\"hello\":\"world\"}"
|
||||
# 规则二:按域名匹配生效
|
||||
- domain:
|
||||
- "*.example.com"
|
||||
- test.com
|
||||
enable_on_status:
|
||||
- 429
|
||||
status_code: 200
|
||||
headers:
|
||||
- Content-Type=application/json
|
||||
body: "{\"errmsg\": \"rate limited\"}"
|
||||
```
|
||||
此例 `ingress` 中指定的 `default/foo` 和 `default/bar` 对应 default 命名空间下名为 foo 和 bar 的 Ingress,当匹配到这两个 Ingress 时,将使用此段配置;
|
||||
此例 `domain` 中指定的 `*.example.com` 和 `test.com` 用于匹配请求的域名,当发现域名匹配时,将使用此段配置;
|
||||
配置的匹配生效顺序,将按照 `matchRules` 下规则的排列顺序,匹配第一个规则后生效对应配置,后续规则将被忽略。
|
||||
84
plugins/wasm-go/extensions/custom-response/README_EN.md
Normal file
84
plugins/wasm-go/extensions/custom-response/README_EN.md
Normal file
@@ -0,0 +1,84 @@
|
||||
<p>
|
||||
English | <a href="README.md">中文</a>
|
||||
</p>
|
||||
|
||||
# Description
|
||||
`custom-response` plugin implements a function of sending custom responses, including custom HTTP response status codes, HTTP response headers and HTTP response body, which can be used in the scenarios of response mocking and sending a custom response for specific status codes, such as customizing the response for rate-limited requests.
|
||||
|
||||
# Configuration Fields
|
||||
|
||||
| Name | Type | Requirement | Default Value | Description |
|
||||
| -------- | -------- | -------- | -------- | -------- |
|
||||
| status_code | number | Optional | 200 | Custom HTTP response status code |
|
||||
| headers | array of string | Optional | - | Custom HTTP response header. Key and value shall be separated using `=`. |
|
||||
| body | string | Optional | - | Custom HTTP response body |
|
||||
| enable_on_status | array of number | Optional | - | The original response status code to match. Generate the custom response only the actual status code matches the configuration. Ignore the status code match if left unconfigured. |
|
||||
|
||||
# Configuration Samples
|
||||
|
||||
## Mock Responses
|
||||
|
||||
```yaml
|
||||
status_code: 200
|
||||
headers:
|
||||
- Content-Type=application/json
|
||||
- Hello=World
|
||||
body: "{\"hello\":\"world\"}"
|
||||
|
||||
```
|
||||
|
||||
According to the configuration above, all the requests will get the following custom response:
|
||||
|
||||
```text
|
||||
HTTP/1.1 200 OK
|
||||
Content-Type: application/json
|
||||
Hello: World
|
||||
Content-Length: 17
|
||||
|
||||
{"hello":"world"}
|
||||
```
|
||||
|
||||
## Send a Custom Response when Rate-Limited
|
||||
|
||||
```yaml
|
||||
enable_on_status:
|
||||
- 429
|
||||
status_code: 302
|
||||
headers:
|
||||
- Location=https://example.com
|
||||
```
|
||||
|
||||
When rate-limited, normally gateway will return a status code of `429` . Now, rate-limited requests will get the following custom response:
|
||||
|
||||
```text
|
||||
HTTP/1.1 302 Found
|
||||
Location: https://example.com
|
||||
```
|
||||
|
||||
So based on the 302 redirecting mechanism provided by browsers, this can redirect rate-limited users to other pages, for example, a static page hosted on CDN.
|
||||
|
||||
If you'd like to send other responses when rate-limited, please add other fields into the configuration, referring to the Mock Responses scenario.
|
||||
|
||||
## Only Enabled for Specific Routes or Domains
|
||||
```yaml
|
||||
# Use matchRules field for fine-grained rule configurations
|
||||
matchRules:
|
||||
# Rule 1: Match by Ingress name
|
||||
- ingress:
|
||||
- default/foo
|
||||
- default/bar
|
||||
body: "{\"hello\":\"world\"}"
|
||||
# Rule 2: Match by domain
|
||||
- domain:
|
||||
- "*.example.com"
|
||||
- test.com
|
||||
enable_on_status:
|
||||
- 429
|
||||
status_code: 200
|
||||
headers:
|
||||
- Content-Type=application/json
|
||||
body: "{\"errmsg\": \"rate limited\"}"
|
||||
```
|
||||
In the rule sample of `ingress`, `default/foo` and `default/bar` are the Ingresses named foo and bar in the default namespace. When the current Ingress names matches the configuration, the rule following shall be applied.
|
||||
In the rule sample of `domain`, `*.example.com` and `test.com` are the domain names used for request matching. When the current domain name matches the configuration, the rule following shall be applied.
|
||||
All rules shall be checked following the order of items in the `matchRules` field, The first matched rule will be applied. All remained will be ignored.
|
||||
71
plugins/wasm-go/extensions/custom-response/envoy.yaml
Normal file
71
plugins/wasm-go/extensions/custom-response/envoy.yaml
Normal file
@@ -0,0 +1,71 @@
|
||||
admin:
|
||||
address:
|
||||
socket_address:
|
||||
protocol: TCP
|
||||
address: 0.0.0.0
|
||||
port_value: 9901
|
||||
static_resources:
|
||||
listeners:
|
||||
- name: listener_0
|
||||
address:
|
||||
socket_address:
|
||||
protocol: TCP
|
||||
address: 0.0.0.0
|
||||
port_value: 10000
|
||||
filter_chains:
|
||||
- filters:
|
||||
- name: envoy.filters.network.http_connection_manager
|
||||
typed_config:
|
||||
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
|
||||
scheme_header_transformation:
|
||||
scheme_to_overwrite: https
|
||||
stat_prefix: ingress_http
|
||||
route_config:
|
||||
name: local_route
|
||||
virtual_hosts:
|
||||
- name: local_service
|
||||
domains: ["*"]
|
||||
routes:
|
||||
- match:
|
||||
prefix: "/"
|
||||
route:
|
||||
cluster: httpbin
|
||||
http_filters:
|
||||
- name: wasmdemo
|
||||
typed_config:
|
||||
"@type": type.googleapis.com/udpa.type.v1.TypedStruct
|
||||
type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
|
||||
value:
|
||||
config:
|
||||
name: wasmdemo
|
||||
vm_config:
|
||||
runtime: envoy.wasm.runtime.v8
|
||||
code:
|
||||
local:
|
||||
filename: /etc/envoy/main.wasm
|
||||
configuration:
|
||||
"@type": "type.googleapis.com/google.protobuf.StringValue"
|
||||
value: |
|
||||
{
|
||||
"headers": ["key1=value1", "key2=value2"],
|
||||
"status_code": 200,
|
||||
"enable_on_status": [200, 201],
|
||||
"body": "{\"hello\":\"world\"}"
|
||||
}
|
||||
- name: envoy.filters.http.router
|
||||
clusters:
|
||||
- name: httpbin
|
||||
connect_timeout: 30s
|
||||
type: LOGICAL_DNS
|
||||
# Comment out the following line to test on v6 networks
|
||||
dns_lookup_family: V4_ONLY
|
||||
lb_policy: ROUND_ROBIN
|
||||
load_assignment:
|
||||
cluster_name: httpbin
|
||||
endpoints:
|
||||
- lb_endpoints:
|
||||
- endpoint:
|
||||
address:
|
||||
socket_address:
|
||||
address: httpbin
|
||||
port_value: 80
|
||||
20
plugins/wasm-go/extensions/custom-response/go.mod
Normal file
20
plugins/wasm-go/extensions/custom-response/go.mod
Normal file
@@ -0,0 +1,20 @@
|
||||
module github.com/alibaba/higress/plugins/wasm-go/extensions/basic-auth
|
||||
|
||||
go 1.19
|
||||
|
||||
replace github.com/alibaba/higress/plugins/wasm-go => ../..
|
||||
|
||||
require (
|
||||
github.com/alibaba/higress/plugins/wasm-go v0.0.0
|
||||
github.com/pkg/errors v0.9.1
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0
|
||||
github.com/tidwall/gjson v1.14.3
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/google/uuid v1.3.0 // indirect
|
||||
github.com/higress-group/nottinygc v0.0.0-20231101025119-e93c4c2f8520 // indirect
|
||||
github.com/magefile/mage v1.14.0 // indirect
|
||||
github.com/tidwall/match v1.1.1 // indirect
|
||||
github.com/tidwall/pretty v1.2.0 // indirect
|
||||
)
|
||||
15
plugins/wasm-go/extensions/custom-response/go.sum
Normal file
15
plugins/wasm-go/extensions/custom-response/go.sum
Normal file
@@ -0,0 +1,15 @@
|
||||
github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
|
||||
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/higress-group/nottinygc v0.0.0-20231101025119-e93c4c2f8520 h1:IHDghbGQ2DTIXHBHxWfqCYQW1fKjyJ/I7W1pMyUDeEA=
|
||||
github.com/higress-group/nottinygc v0.0.0-20231101025119-e93c4c2f8520/go.mod h1:Nz8ORLaFiLWotg6GeKlJMhv8cci8mM43uEnLA5t8iew=
|
||||
github.com/magefile/mage v1.14.0 h1:6QDX3g6z1YvJ4olPhT1wksUcSa/V0a1B+pJb73fBjyo=
|
||||
github.com/magefile/mage v1.14.0/go.mod h1:z5UZb/iS3GoOSn0JgWuiw7dxlurVYTu+/jHXqQg881A=
|
||||
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0 h1:kS7BvMKN+FiptV4pfwiNX8e3q14evxAWkhYbxt8EI1M=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0/go.mod h1:qkW5MBz2jch2u8bS59wws65WC+Gtx3x0aPUX5JL7CXI=
|
||||
github.com/tidwall/gjson v1.14.3 h1:9jvXn7olKEHU1S9vwoMGliaT8jq1vJ7IH/n9zD9Dnlw=
|
||||
github.com/tidwall/gjson v1.14.3/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
|
||||
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
|
||||
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
|
||||
github.com/tidwall/pretty v1.2.0 h1:RWIZEg2iJ8/g6fDDYzMpobmaoGh5OLl4AXtGUGPcqCs=
|
||||
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
|
||||
135
plugins/wasm-go/extensions/custom-response/main.go
Normal file
135
plugins/wasm-go/extensions/custom-response/main.go
Normal file
@@ -0,0 +1,135 @@
|
||||
// Copyright (c) 2022 Alibaba Group Holding Ltd.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/alibaba/higress/plugins/wasm-go/pkg/wrapper"
|
||||
"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm"
|
||||
"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm/types"
|
||||
"github.com/tidwall/gjson"
|
||||
)
|
||||
|
||||
func main() {
|
||||
wrapper.SetCtx(
|
||||
"custom-response",
|
||||
wrapper.ParseConfigBy(parseConfig),
|
||||
wrapper.ProcessRequestHeadersBy(onHttpRequestHeaders),
|
||||
wrapper.ProcessResponseHeadersBy(onHttpResponseHeaders),
|
||||
)
|
||||
}
|
||||
|
||||
type CustomResponseConfig struct {
|
||||
statusCode uint32
|
||||
headers [][2]string
|
||||
body string
|
||||
enableOnStatus []uint32
|
||||
contentType string
|
||||
}
|
||||
|
||||
func parseConfig(gjson gjson.Result, config *CustomResponseConfig, log wrapper.Log) error {
|
||||
headersArray := gjson.Get("headers").Array()
|
||||
config.headers = make([][2]string, 0, len(headersArray))
|
||||
for _, v := range headersArray {
|
||||
kv := strings.SplitN(v.String(), "=", 2)
|
||||
if len(kv) == 2 {
|
||||
key := strings.TrimSpace(kv[0])
|
||||
value := strings.TrimSpace(kv[1])
|
||||
if strings.EqualFold(key, "content-type") {
|
||||
config.contentType = value
|
||||
} else if strings.EqualFold(key, "content-length") {
|
||||
continue
|
||||
} else {
|
||||
config.headers = append(config.headers, [2]string{key, value})
|
||||
}
|
||||
} else {
|
||||
return fmt.Errorf("invalid header pair format: %s", v.String())
|
||||
}
|
||||
}
|
||||
|
||||
config.body = gjson.Get("body").String()
|
||||
if config.contentType == "" && config.body != "" {
|
||||
if json.Valid([]byte(config.body)) {
|
||||
config.contentType = "application/json; charset=utf-8"
|
||||
} else {
|
||||
config.contentType = "text/plain; charset=utf-8"
|
||||
}
|
||||
}
|
||||
config.headers = append(config.headers, [2]string{"content-type", config.contentType})
|
||||
|
||||
config.statusCode = 200
|
||||
if gjson.Get("status_code").Exists() {
|
||||
statusCode := gjson.Get("status_code")
|
||||
parsedStatusCode, err := strconv.Atoi(statusCode.String())
|
||||
if err != nil {
|
||||
return fmt.Errorf("invalid status code value: %s", statusCode.String())
|
||||
}
|
||||
config.statusCode = uint32(parsedStatusCode)
|
||||
}
|
||||
|
||||
enableOnStatusArray := gjson.Get("enable_on_status").Array()
|
||||
config.enableOnStatus = make([]uint32, 0, len(enableOnStatusArray))
|
||||
for _, v := range enableOnStatusArray {
|
||||
parsedEnableOnStatus, err := strconv.Atoi(v.String())
|
||||
if err != nil {
|
||||
return fmt.Errorf("invalid enable_on_status value: %s", v.String())
|
||||
}
|
||||
config.enableOnStatus = append(config.enableOnStatus, uint32(parsedEnableOnStatus))
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func onHttpRequestHeaders(ctx wrapper.HttpContext, config CustomResponseConfig, log wrapper.Log) types.Action {
|
||||
if len(config.enableOnStatus) != 0 {
|
||||
return types.ActionContinue
|
||||
}
|
||||
err := proxywasm.SendHttpResponse(config.statusCode, config.headers, []byte(config.body), -1)
|
||||
if err != nil {
|
||||
log.Errorf("send http response failed: %v", err)
|
||||
}
|
||||
|
||||
return types.ActionPause
|
||||
}
|
||||
|
||||
func onHttpResponseHeaders(ctx wrapper.HttpContext, config CustomResponseConfig, log wrapper.Log) types.Action {
|
||||
// enableOnStatus is not empty, compare the status code.
|
||||
// if match the status code, mock the response.
|
||||
statusCodeStr, err := proxywasm.GetHttpResponseHeader(":status")
|
||||
if err != nil {
|
||||
log.Errorf("get http response status code failed: %v", err)
|
||||
return types.ActionContinue
|
||||
}
|
||||
statusCode, err := strconv.ParseUint(statusCodeStr, 10, 32)
|
||||
if err != nil {
|
||||
log.Errorf("parse http response status code failed: %v", err)
|
||||
return types.ActionContinue
|
||||
}
|
||||
|
||||
for _, v := range config.enableOnStatus {
|
||||
if uint32(statusCode) == v {
|
||||
err = proxywasm.SendHttpResponse(config.statusCode, config.headers, []byte(config.body), -1)
|
||||
if err != nil {
|
||||
log.Errorf("send http response failed: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return types.ActionContinue
|
||||
}
|
||||
@@ -5,7 +5,7 @@ go 1.18
|
||||
replace github.com/alibaba/higress/plugins/wasm-go => ../..
|
||||
|
||||
require (
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0
|
||||
github.com/tidwall/gjson v1.14.3
|
||||
)
|
||||
|
||||
|
||||
@@ -7,8 +7,8 @@ github.com/magefile/mage v1.14.0 h1:6QDX3g6z1YvJ4olPhT1wksUcSa/V0a1B+pJb73fBjyo=
|
||||
github.com/magefile/mage v1.14.0/go.mod h1:z5UZb/iS3GoOSn0JgWuiw7dxlurVYTu+/jHXqQg881A=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c h1:OCUFXVGixHLfNjg6/QYEhv+jHJ5mRGhpEUVFv9eWPJE=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c/go.mod h1:5t/pWFNJ9eMyu/K/Z+OeGhDJ9sN9eCo8fc2pyM/Qjg4=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0 h1:kS7BvMKN+FiptV4pfwiNX8e3q14evxAWkhYbxt8EI1M=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0/go.mod h1:qkW5MBz2jch2u8bS59wws65WC+Gtx3x0aPUX5JL7CXI=
|
||||
github.com/tidwall/gjson v1.14.3 h1:9jvXn7olKEHU1S9vwoMGliaT8jq1vJ7IH/n9zD9Dnlw=
|
||||
github.com/tidwall/gjson v1.14.3/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
|
||||
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
|
||||
|
||||
@@ -6,7 +6,7 @@ replace github.com/alibaba/higress/plugins/wasm-go => ../..
|
||||
|
||||
require (
|
||||
github.com/alibaba/higress/plugins/wasm-go v0.0.0
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0
|
||||
)
|
||||
|
||||
require (
|
||||
|
||||
@@ -7,8 +7,8 @@ github.com/magefile/mage v1.14.0 h1:6QDX3g6z1YvJ4olPhT1wksUcSa/V0a1B+pJb73fBjyo=
|
||||
github.com/magefile/mage v1.14.0/go.mod h1:z5UZb/iS3GoOSn0JgWuiw7dxlurVYTu+/jHXqQg881A=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c h1:OCUFXVGixHLfNjg6/QYEhv+jHJ5mRGhpEUVFv9eWPJE=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c/go.mod h1:5t/pWFNJ9eMyu/K/Z+OeGhDJ9sN9eCo8fc2pyM/Qjg4=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0 h1:kS7BvMKN+FiptV4pfwiNX8e3q14evxAWkhYbxt8EI1M=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0/go.mod h1:qkW5MBz2jch2u8bS59wws65WC+Gtx3x0aPUX5JL7CXI=
|
||||
github.com/tidwall/gjson v1.14.3 h1:9jvXn7olKEHU1S9vwoMGliaT8jq1vJ7IH/n9zD9Dnlw=
|
||||
github.com/tidwall/gjson v1.14.3/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
|
||||
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
|
||||
|
||||
@@ -6,7 +6,7 @@ replace github.com/alibaba/higress/plugins/wasm-go => ../..
|
||||
|
||||
require (
|
||||
github.com/alibaba/higress/plugins/wasm-go v0.0.0
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0
|
||||
github.com/tidwall/gjson v1.14.3
|
||||
)
|
||||
|
||||
|
||||
@@ -7,8 +7,8 @@ github.com/magefile/mage v1.14.0 h1:6QDX3g6z1YvJ4olPhT1wksUcSa/V0a1B+pJb73fBjyo=
|
||||
github.com/magefile/mage v1.14.0/go.mod h1:z5UZb/iS3GoOSn0JgWuiw7dxlurVYTu+/jHXqQg881A=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c h1:OCUFXVGixHLfNjg6/QYEhv+jHJ5mRGhpEUVFv9eWPJE=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c/go.mod h1:5t/pWFNJ9eMyu/K/Z+OeGhDJ9sN9eCo8fc2pyM/Qjg4=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0 h1:kS7BvMKN+FiptV4pfwiNX8e3q14evxAWkhYbxt8EI1M=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0/go.mod h1:qkW5MBz2jch2u8bS59wws65WC+Gtx3x0aPUX5JL7CXI=
|
||||
github.com/tidwall/gjson v1.14.3 h1:9jvXn7olKEHU1S9vwoMGliaT8jq1vJ7IH/n9zD9Dnlw=
|
||||
github.com/tidwall/gjson v1.14.3/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
|
||||
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
|
||||
|
||||
@@ -5,7 +5,7 @@ go 1.19
|
||||
require (
|
||||
github.com/alibaba/higress/plugins/wasm-go v0.0.0-20230807053545-d307d0e755f1
|
||||
github.com/go-jose/go-jose/v3 v3.0.0
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0
|
||||
github.com/tidwall/gjson v1.14.3
|
||||
golang.org/x/oauth2 v0.11.0
|
||||
)
|
||||
|
||||
@@ -20,8 +20,8 @@ github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZN
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c h1:OCUFXVGixHLfNjg6/QYEhv+jHJ5mRGhpEUVFv9eWPJE=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c/go.mod h1:5t/pWFNJ9eMyu/K/Z+OeGhDJ9sN9eCo8fc2pyM/Qjg4=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0 h1:kS7BvMKN+FiptV4pfwiNX8e3q14evxAWkhYbxt8EI1M=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0/go.mod h1:qkW5MBz2jch2u8bS59wws65WC+Gtx3x0aPUX5JL7CXI=
|
||||
github.com/tidwall/gjson v1.14.3 h1:9jvXn7olKEHU1S9vwoMGliaT8jq1vJ7IH/n9zD9Dnlw=
|
||||
github.com/tidwall/gjson v1.14.3/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
|
||||
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
|
||||
|
||||
101
plugins/wasm-go/extensions/opa/README.md
Normal file
101
plugins/wasm-go/extensions/opa/README.md
Normal file
@@ -0,0 +1,101 @@
|
||||
# 功能说明
|
||||
|
||||
该插件实现了 `OPA` 策略控制
|
||||
|
||||
# 该教程使用k8s,[k8s配置文件](../../../../test/e2e/conformance/tests/go-wasm-opa.yaml)
|
||||
|
||||
支持client `k8s,nacos,ip,route` 策略去访问
|
||||
|
||||
## 配置字段
|
||||
|
||||
| 字段 | 数据类型 | 填写要求 | 默认值 | 描述 |
|
||||
|---------------|--------|------|-----|--------------------------------------|
|
||||
| policy | string | 必填 | - | opa 策略 |
|
||||
| timeout | string | 必填 | - | 访问超时时间设置 |
|
||||
| serviceSource | string | 必填 | - | k8s,nacos,ip,route |
|
||||
| host | string | 非必填 | - | 服务主机(serviceSource为`ip`必填) |
|
||||
| serviceName | string | 非必填 | - | 服务名称(serviceSource为`k8s,nacos,ip`必填) |
|
||||
| servicePort | string | 非必填 | - | 服务端口(serviceSource为`k8s,nacos,ip`必填) |
|
||||
| namespace | string | 非必填 | - | 服务端口(serviceSource为`k8s,nacos`必填) |
|
||||
|
||||
这是一个用于OPA认证配置的表格,确保在提供所有必要的信息时遵循上述指导。
|
||||
|
||||
## 配置示例
|
||||
|
||||
```yaml
|
||||
serviceSource: k8s
|
||||
serviceName: opa
|
||||
servicePort: 8181
|
||||
namespace: higress-backend
|
||||
policy: example1
|
||||
timeout: 5s
|
||||
```
|
||||
|
||||
# 在宿主机上执行OPA的流程
|
||||
|
||||
## 启动opa服务
|
||||
|
||||
```shell
|
||||
docker run -d --name opa -p 8181:8181 openpolicyagent/opa:0.35.0 run -s
|
||||
```
|
||||
|
||||
## 创建opa策略
|
||||
|
||||
```shell
|
||||
curl -X PUT '127.0.0.1:8181/v1/policies/example1' \
|
||||
-H 'Content-Type: text/plain' \
|
||||
-d 'package example1
|
||||
|
||||
import input.request
|
||||
|
||||
default allow = false
|
||||
|
||||
allow {
|
||||
# HTTP method must GET
|
||||
request.method == "GET"
|
||||
}'
|
||||
```
|
||||
|
||||
## 查询策略
|
||||
|
||||
```shell
|
||||
curl -X POST '127.0.0.1:8181/v1/data/example1/allow' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"input":{"request":{"method":"GET"}}}'
|
||||
```
|
||||
|
||||
# 测试插件
|
||||
|
||||
## 打包 WASM 插件
|
||||
|
||||
> 在 `wasm-go` 目录下把Dockerfile文件改成`PLUGIN_NAME=opa`,然后执行以下命令
|
||||
|
||||
```shell
|
||||
docker build -t build-wasm-opa --build-arg GOPROXY=https://goproxy.cn,direct --platform=linux/amd64 .
|
||||
```
|
||||
|
||||
## 拷贝插件
|
||||
|
||||
> 在当前的目录执行以下命令,将插件拷贝当前的目录
|
||||
|
||||
```shell
|
||||
docker cp wasm-opa:/plugin.wasm .
|
||||
```
|
||||
|
||||
## 运行插件
|
||||
|
||||
> 运行前修改envoy.yaml 这两个字段 `OPA_SERVER` `OPA_PORT` 替换宿主机上的IP和端口
|
||||
|
||||
```shell
|
||||
docker compose up
|
||||
```
|
||||
|
||||
## 使用curl测试插件
|
||||
|
||||
```shell
|
||||
curl http://127.0.0.1:10000/get -X GET -v
|
||||
```
|
||||
|
||||
```shell
|
||||
curl http://127.0.0.1:10000/get -X POST -v
|
||||
```
|
||||
1
plugins/wasm-go/extensions/opa/VERSION
Normal file
1
plugins/wasm-go/extensions/opa/VERSION
Normal file
@@ -0,0 +1 @@
|
||||
1.0.0
|
||||
82
plugins/wasm-go/extensions/opa/config.go
Normal file
82
plugins/wasm-go/extensions/opa/config.go
Normal file
@@ -0,0 +1,82 @@
|
||||
// Copyright (c) 2022 Alibaba Group Holding Ltd.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"strings"
|
||||
|
||||
"github.com/alibaba/higress/plugins/wasm-go/pkg/wrapper"
|
||||
"github.com/tidwall/gjson"
|
||||
)
|
||||
|
||||
type OpaConfig struct {
|
||||
policy string
|
||||
timeout uint32
|
||||
|
||||
client wrapper.HttpClient
|
||||
}
|
||||
|
||||
func Client(json gjson.Result) (wrapper.HttpClient, error) {
|
||||
serviceSource := strings.TrimSpace(json.Get("serviceSource").String())
|
||||
serviceName := strings.TrimSpace(json.Get("serviceName").String())
|
||||
servicePort := json.Get("servicePort").Int()
|
||||
|
||||
host := strings.TrimSpace(json.Get("host").String())
|
||||
if host == "" {
|
||||
if serviceName == "" || servicePort == 0 {
|
||||
return nil, errors.New("invalid service config")
|
||||
}
|
||||
}
|
||||
|
||||
var namespace string
|
||||
if serviceSource == "k8s" || serviceSource == "nacos" {
|
||||
if namespace = strings.TrimSpace(json.Get("namespace").String()); namespace == "" {
|
||||
return nil, errors.New("namespace not allow empty")
|
||||
}
|
||||
}
|
||||
|
||||
switch serviceSource {
|
||||
case "k8s":
|
||||
return wrapper.NewClusterClient(wrapper.K8sCluster{
|
||||
ServiceName: serviceName,
|
||||
Namespace: namespace,
|
||||
Port: servicePort,
|
||||
}), nil
|
||||
case "nacos":
|
||||
return wrapper.NewClusterClient(wrapper.NacosCluster{
|
||||
ServiceName: serviceName,
|
||||
NamespaceID: namespace,
|
||||
Port: servicePort,
|
||||
}), nil
|
||||
case "ip":
|
||||
return wrapper.NewClusterClient(wrapper.StaticIpCluster{
|
||||
ServiceName: serviceName,
|
||||
Host: host,
|
||||
Port: servicePort,
|
||||
}), nil
|
||||
case "dns":
|
||||
return wrapper.NewClusterClient(wrapper.DnsCluster{
|
||||
ServiceName: serviceName,
|
||||
Port: servicePort,
|
||||
Domain: json.Get("domain").String(),
|
||||
}), nil
|
||||
case "route":
|
||||
return wrapper.NewClusterClient(wrapper.RouteCluster{
|
||||
Host: host,
|
||||
}), nil
|
||||
}
|
||||
return nil, errors.New("unknown service source: " + serviceSource)
|
||||
}
|
||||
50
plugins/wasm-go/extensions/opa/config_test.go
Normal file
50
plugins/wasm-go/extensions/opa/config_test.go
Normal file
@@ -0,0 +1,50 @@
|
||||
// Copyright (c) 2022 Alibaba Group Holding Ltd.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/alibaba/higress/plugins/wasm-go/pkg/wrapper"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/tidwall/gjson"
|
||||
)
|
||||
|
||||
func TestConfig(t *testing.T) {
|
||||
json := gjson.Result{Type: gjson.JSON, Raw: `{"serviceSource": "k8s","serviceName": "opa","servicePort": 8181,"namespace": "example1","policy": "example1","timeout": "5s"}`}
|
||||
config := &OpaConfig{}
|
||||
assert.NoError(t, parseConfig(json, config, wrapper.Log{}))
|
||||
assert.Equal(t, config.policy, "example1")
|
||||
assert.Equal(t, config.timeout, uint32(5000))
|
||||
assert.NotNil(t, config.client)
|
||||
|
||||
type tt struct {
|
||||
raw string
|
||||
result bool
|
||||
}
|
||||
|
||||
tests := []tt{
|
||||
{raw: `{}`, result: false},
|
||||
{raw: `{"policy": "example1","timeout": "5s"}`, result: false},
|
||||
{raw: `{"serviceSource": "route","host": "example.com","policy": "example1","timeout": "5s"}`, result: true},
|
||||
{raw: `{"serviceSource": "nacos","serviceName": "opa","servicePort": 8181,"policy": "example1","timeout": "5s"}`, result: false},
|
||||
{raw: `{"serviceSource": "nacos","serviceName": "opa","servicePort": 8181,"namespace": "example1","policy": "example1","timeout": "5s"}`, result: true},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
json = gjson.Result{Type: gjson.JSON, Raw: test.raw}
|
||||
assert.Equal(t, parseConfig(json, config, wrapper.Log{}) == nil, test.result)
|
||||
}
|
||||
}
|
||||
16
plugins/wasm-go/extensions/opa/docker-compose.yaml
Normal file
16
plugins/wasm-go/extensions/opa/docker-compose.yaml
Normal file
@@ -0,0 +1,16 @@
|
||||
version: '3.7'
|
||||
services:
|
||||
envoy:
|
||||
image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/gateway:1.3.1
|
||||
entrypoint: /usr/local/bin/envoy
|
||||
command: -c /etc/envoy/envoy.yaml --component-log-level wasm:debug
|
||||
networks:
|
||||
- wasmtest
|
||||
ports:
|
||||
- "10000:10000"
|
||||
volumes:
|
||||
- ./envoy.yaml:/etc/envoy/envoy.yaml
|
||||
- ./plugin.wasm:/etc/envoy/plugin.wasm
|
||||
|
||||
networks:
|
||||
wasmtest: { }
|
||||
69
plugins/wasm-go/extensions/opa/envoy.yaml
Normal file
69
plugins/wasm-go/extensions/opa/envoy.yaml
Normal file
@@ -0,0 +1,69 @@
|
||||
admin:
|
||||
address:
|
||||
socket_address:
|
||||
protocol: TCP
|
||||
address: 0.0.0.0
|
||||
port_value: 9901
|
||||
static_resources:
|
||||
listeners:
|
||||
- name: listener_0
|
||||
address:
|
||||
socket_address:
|
||||
protocol: TCP
|
||||
address: 0.0.0.0
|
||||
port_value: 10000
|
||||
filter_chains:
|
||||
- filters:
|
||||
- name: envoy.filters.network.http_connection_manager
|
||||
typed_config:
|
||||
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
|
||||
stat_prefix: ingress_http
|
||||
route_config:
|
||||
name: local_route
|
||||
virtual_hosts:
|
||||
- name: local_service
|
||||
domains: [ "*" ]
|
||||
routes:
|
||||
- match:
|
||||
prefix: "/"
|
||||
route:
|
||||
cluster: opa-server
|
||||
http_filters:
|
||||
- name: wasmdemo
|
||||
typed_config:
|
||||
"@type": type.googleapis.com/udpa.type.v1.TypedStruct
|
||||
type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
|
||||
value:
|
||||
config:
|
||||
name: wasmdemo
|
||||
vm_config:
|
||||
runtime: envoy.wasm.runtime.v8
|
||||
code:
|
||||
local:
|
||||
filename: /etc/envoy/plugin.wasm
|
||||
configuration:
|
||||
"@type": "type.googleapis.com/google.protobuf.StringValue"
|
||||
value: |
|
||||
{
|
||||
"serviceSource": "route",
|
||||
"host": "OPA_SERVER:OPA_PORT",
|
||||
"policy": "example1",
|
||||
"timeout": "5s"
|
||||
}
|
||||
- name: envoy.filters.http.router
|
||||
clusters:
|
||||
- name: opa-server
|
||||
connect_timeout: 0.5s
|
||||
type: STRICT_DNS
|
||||
lb_policy: ROUND_ROBIN
|
||||
dns_refresh_rate: 5s
|
||||
dns_lookup_family: V4_ONLY
|
||||
load_assignment:
|
||||
cluster_name: opa-server
|
||||
endpoints:
|
||||
- lb_endpoints:
|
||||
- endpoint:
|
||||
address:
|
||||
socket_address:
|
||||
address: OPA_SERVER # opa server Host IP
|
||||
port_value: OPA_PORT # opa server Host PORT
|
||||
23
plugins/wasm-go/extensions/opa/go.mod
Normal file
23
plugins/wasm-go/extensions/opa/go.mod
Normal file
@@ -0,0 +1,23 @@
|
||||
module github.com/alibaba/higress/plugins/wasm-go/extensions/opa
|
||||
|
||||
go 1.19
|
||||
|
||||
require (
|
||||
github.com/alibaba/higress/plugins/wasm-go v0.0.0
|
||||
github.com/stretchr/testify v1.8.4
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0
|
||||
github.com/tidwall/gjson v1.14.4
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/davecgh/go-spew v1.1.1 // indirect
|
||||
github.com/google/uuid v1.5.0 // indirect
|
||||
github.com/higress-group/nottinygc v0.0.0-20231101025119-e93c4c2f8520 // indirect
|
||||
github.com/magefile/mage v1.14.0 // indirect
|
||||
github.com/pmezard/go-difflib v1.0.0 // indirect
|
||||
github.com/tidwall/match v1.1.1 // indirect
|
||||
github.com/tidwall/pretty v1.2.0 // indirect
|
||||
gopkg.in/yaml.v3 v3.0.1 // indirect
|
||||
)
|
||||
|
||||
replace github.com/alibaba/higress/plugins/wasm-go => ../..
|
||||
24
plugins/wasm-go/extensions/opa/go.sum
Normal file
24
plugins/wasm-go/extensions/opa/go.sum
Normal file
@@ -0,0 +1,24 @@
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/google/uuid v1.5.0 h1:1p67kYwdtXjb0gL0BPiP1Av9wiZPo5A8z2cWkTZ+eyU=
|
||||
github.com/google/uuid v1.5.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/higress-group/nottinygc v0.0.0-20231101025119-e93c4c2f8520 h1:IHDghbGQ2DTIXHBHxWfqCYQW1fKjyJ/I7W1pMyUDeEA=
|
||||
github.com/higress-group/nottinygc v0.0.0-20231101025119-e93c4c2f8520/go.mod h1:Nz8ORLaFiLWotg6GeKlJMhv8cci8mM43uEnLA5t8iew=
|
||||
github.com/magefile/mage v1.14.0 h1:6QDX3g6z1YvJ4olPhT1wksUcSa/V0a1B+pJb73fBjyo=
|
||||
github.com/magefile/mage v1.14.0/go.mod h1:z5UZb/iS3GoOSn0JgWuiw7dxlurVYTu+/jHXqQg881A=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
|
||||
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0 h1:kS7BvMKN+FiptV4pfwiNX8e3q14evxAWkhYbxt8EI1M=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0/go.mod h1:qkW5MBz2jch2u8bS59wws65WC+Gtx3x0aPUX5JL7CXI=
|
||||
github.com/tidwall/gjson v1.14.4 h1:uo0p8EbA09J7RQaflQ1aBRffTR7xedD2bcIVSYxLnkM=
|
||||
github.com/tidwall/gjson v1.14.4/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
|
||||
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
|
||||
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
|
||||
github.com/tidwall/pretty v1.2.0 h1:RWIZEg2iJ8/g6fDDYzMpobmaoGh5OLl4AXtGUGPcqCs=
|
||||
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
134
plugins/wasm-go/extensions/opa/main.go
Normal file
134
plugins/wasm-go/extensions/opa/main.go
Normal file
@@ -0,0 +1,134 @@
|
||||
// Copyright (c) 2022 Alibaba Group Holding Ltd.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/alibaba/higress/plugins/wasm-go/pkg/wrapper"
|
||||
"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm"
|
||||
"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm/types"
|
||||
"github.com/tidwall/gjson"
|
||||
)
|
||||
|
||||
func main() {
|
||||
wrapper.SetCtx(
|
||||
"opa",
|
||||
wrapper.ParseConfigBy(parseConfig),
|
||||
wrapper.ProcessRequestHeadersBy(onHttpRequestHeaders),
|
||||
wrapper.ProcessRequestBodyBy(onHttpRequestBody),
|
||||
)
|
||||
}
|
||||
|
||||
type Metadata struct {
|
||||
Input map[string]interface{} `json:"input"`
|
||||
}
|
||||
|
||||
func parseConfig(json gjson.Result, config *OpaConfig, log wrapper.Log) error {
|
||||
policy := json.Get("policy").String()
|
||||
if strings.TrimSpace(policy) == "" {
|
||||
return errors.New("policy not allow empty")
|
||||
}
|
||||
|
||||
timeout := json.Get("timeout").String()
|
||||
if strings.TrimSpace(timeout) == "" {
|
||||
return errors.New("timeout not allow empty")
|
||||
}
|
||||
|
||||
duration, err := time.ParseDuration(timeout)
|
||||
if err != nil {
|
||||
return errors.New("timeout parse fail: " + err.Error())
|
||||
}
|
||||
|
||||
var uint32Duration uint32
|
||||
|
||||
if duration.Milliseconds() > int64(^uint32(0)) {
|
||||
} else {
|
||||
uint32Duration = uint32(duration.Milliseconds())
|
||||
}
|
||||
config.timeout = uint32Duration
|
||||
|
||||
client, err := Client(json)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
config.client = client
|
||||
config.policy = policy
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func onHttpRequestHeaders(ctx wrapper.HttpContext, config OpaConfig, log wrapper.Log) types.Action {
|
||||
return opaCall(ctx, config, nil, log)
|
||||
}
|
||||
|
||||
func onHttpRequestBody(ctx wrapper.HttpContext, config OpaConfig, body []byte, log wrapper.Log) types.Action {
|
||||
return opaCall(ctx, config, body, log)
|
||||
}
|
||||
|
||||
func opaCall(ctx wrapper.HttpContext, config OpaConfig, body []byte, log wrapper.Log) types.Action {
|
||||
request := make(map[string]interface{}, 6)
|
||||
headers, _ := proxywasm.GetHttpRequestHeaders()
|
||||
|
||||
request["method"] = ctx.Method()
|
||||
request["scheme"] = ctx.Scheme()
|
||||
request["path"] = ctx.Path()
|
||||
request["headers"] = headers
|
||||
if len(body) != 0 {
|
||||
request["body"] = body
|
||||
}
|
||||
parse, _ := url.Parse(ctx.Path())
|
||||
query, _ := url.ParseQuery(parse.RawQuery)
|
||||
request["query"] = query
|
||||
|
||||
data, _ := json.Marshal(Metadata{Input: map[string]interface{}{"request": request}})
|
||||
if err := config.client.Post(fmt.Sprintf("/v1/data/%s/allow", config.policy),
|
||||
[][2]string{{"Content-Type", "application/json"}},
|
||||
data, rspCall, config.timeout); err != nil {
|
||||
log.Errorf("client opa fail %v", err)
|
||||
return types.ActionPause
|
||||
}
|
||||
return types.ActionPause
|
||||
}
|
||||
|
||||
func rspCall(statusCode int, _ http.Header, responseBody []byte) {
|
||||
if statusCode != http.StatusOK {
|
||||
proxywasm.SendHttpResponse(uint32(statusCode), nil, []byte("opa state not is 200"), -1)
|
||||
return
|
||||
}
|
||||
var rsp map[string]interface{}
|
||||
if err := json.Unmarshal(responseBody, &rsp); err != nil {
|
||||
proxywasm.SendHttpResponse(http.StatusInternalServerError, nil, []byte(fmt.Sprintf("opa parse rsp fail %+v", err)), -1)
|
||||
return
|
||||
}
|
||||
|
||||
result, ok := rsp["result"].(bool)
|
||||
if !ok {
|
||||
proxywasm.SendHttpResponse(http.StatusInternalServerError, nil, []byte("rsp type conversion fail"), -1)
|
||||
return
|
||||
}
|
||||
|
||||
if !result {
|
||||
proxywasm.SendHttpResponse(http.StatusUnauthorized, nil, []byte("opa server not allowed"), -1)
|
||||
return
|
||||
}
|
||||
proxywasm.ResumeHttpRequest()
|
||||
}
|
||||
@@ -6,7 +6,7 @@ replace github.com/alibaba/higress/plugins/wasm-go => ../..
|
||||
|
||||
require (
|
||||
github.com/alibaba/higress/plugins/wasm-go v0.0.0
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0
|
||||
github.com/tidwall/gjson v1.14.3
|
||||
)
|
||||
|
||||
|
||||
@@ -7,8 +7,8 @@ github.com/magefile/mage v1.14.0 h1:6QDX3g6z1YvJ4olPhT1wksUcSa/V0a1B+pJb73fBjyo=
|
||||
github.com/magefile/mage v1.14.0/go.mod h1:z5UZb/iS3GoOSn0JgWuiw7dxlurVYTu+/jHXqQg881A=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c h1:OCUFXVGixHLfNjg6/QYEhv+jHJ5mRGhpEUVFv9eWPJE=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c/go.mod h1:5t/pWFNJ9eMyu/K/Z+OeGhDJ9sN9eCo8fc2pyM/Qjg4=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0 h1:kS7BvMKN+FiptV4pfwiNX8e3q14evxAWkhYbxt8EI1M=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0/go.mod h1:qkW5MBz2jch2u8bS59wws65WC+Gtx3x0aPUX5JL7CXI=
|
||||
github.com/tidwall/gjson v1.14.3 h1:9jvXn7olKEHU1S9vwoMGliaT8jq1vJ7IH/n9zD9Dnlw=
|
||||
github.com/tidwall/gjson v1.14.3/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
|
||||
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
|
||||
|
||||
146
plugins/wasm-go/extensions/request-validation/README.md
Normal file
146
plugins/wasm-go/extensions/request-validation/README.md
Normal file
@@ -0,0 +1,146 @@
|
||||
# 功能说明
|
||||
`request-validation`插件用于提前验证向上游服务转发的请求。该插件使用`JSON Schema`机制进行数据验证,可以验证请求的body及header数据。
|
||||
|
||||
# 配置字段
|
||||
|
||||
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|
||||
| -------- | -------- | -------- |-----| -------- |
|
||||
|header_schema|object|选填| - |配置用于验证请求header的JSON Schema|
|
||||
|body_schema|object|选填| - |配置用于验证请求body的JSON Schema|
|
||||
|rejected_code|number|选填| 403 |配置请求被拒绝时返回的HTTP状态码|
|
||||
|rejected_msg|string|选填| - |配置请求被拒绝时返回的HTTP应答Body|
|
||||
|enable_swagger|bool|选填| false |配置是否开启swagger文档验证|
|
||||
|enable_oas3|bool|选填| false |配置是否开启OAS3文档验证|
|
||||
|
||||
**校验规则对header和body是一样的,下面以body为例说明**
|
||||
|
||||
# 配置示例
|
||||
|
||||
## 枚举(Enum)验证
|
||||
```yaml
|
||||
body_schema:
|
||||
type: object
|
||||
required:
|
||||
- enum_payload
|
||||
properties:
|
||||
enum_payload:
|
||||
type: string
|
||||
enum:
|
||||
- "enum_string_1"
|
||||
- "enum_string_2"
|
||||
default: "enum_string_1"
|
||||
```
|
||||
|
||||
## 布尔(Boolean)验证
|
||||
```yaml
|
||||
body_schema:
|
||||
type: object
|
||||
required:
|
||||
- boolean_payload
|
||||
properties:
|
||||
boolean_payload:
|
||||
type: boolean
|
||||
default: true
|
||||
```
|
||||
|
||||
## 数字范围(Number or Integer)验证
|
||||
```yaml
|
||||
body_schema:
|
||||
type: object
|
||||
required:
|
||||
- integer_payload
|
||||
properties:
|
||||
integer_payload:
|
||||
type: integer
|
||||
minimum: 1
|
||||
maximum: 10
|
||||
```
|
||||
|
||||
## 字符串长度(String)验证
|
||||
```yaml
|
||||
body_schema:
|
||||
type: object
|
||||
required:
|
||||
- string_payload
|
||||
properties:
|
||||
string_payload:
|
||||
type: string
|
||||
minLength: 1
|
||||
maxLength: 10
|
||||
```
|
||||
|
||||
## 正则表达式(Regex)验证
|
||||
```yaml
|
||||
body_schema:
|
||||
type: object
|
||||
required:
|
||||
- regex_payload
|
||||
properties:
|
||||
regex_payload:
|
||||
type: string
|
||||
minLength: 1
|
||||
maxLength: 10
|
||||
pattern: "^[a-zA-Z0-9_]+$"
|
||||
```
|
||||
|
||||
## 数组(Array)验证
|
||||
```yaml
|
||||
body_schema:
|
||||
type: object
|
||||
required:
|
||||
- array_payload
|
||||
properties:
|
||||
array_payload:
|
||||
type: array
|
||||
minItems: 1
|
||||
items:
|
||||
type: integer
|
||||
minimum: 1
|
||||
maximum: 10
|
||||
uniqueItems: true
|
||||
default: [1, 2, 3]
|
||||
```
|
||||
|
||||
## 多字段组合(Combined)验证
|
||||
```yaml
|
||||
body_schema:
|
||||
type: object
|
||||
required:
|
||||
- boolean_payload
|
||||
- array_payload
|
||||
- regex_payload
|
||||
properties:
|
||||
boolean_payload:
|
||||
type: boolean
|
||||
array_payload:
|
||||
type: array
|
||||
minItems: 1
|
||||
items:
|
||||
type: integer
|
||||
minimum: 1
|
||||
maximum: 10
|
||||
uniqueItems: true
|
||||
default: [1, 2, 3]
|
||||
regex_payload:
|
||||
type: string
|
||||
minLength: 1
|
||||
maxLength: 10
|
||||
pattern: "^[a-zA-Z0-9_]+$"
|
||||
```
|
||||
|
||||
## 自定义拒绝信息
|
||||
```yaml
|
||||
body_schema:
|
||||
type: object
|
||||
required:
|
||||
- boolean_payload
|
||||
properties:
|
||||
boolean_payload:
|
||||
type: boolean
|
||||
rejected_code: 403
|
||||
rejected_msg: "请求被拒绝"
|
||||
```
|
||||
|
||||
# 本地调试
|
||||
|
||||
参考[使用 GO 语言开发 WASM 插件](https://higress.io/zh-cn/docs/user/wasm-go#%E4%B8%89%E6%9C%AC%E5%9C%B0%E8%B0%83%E8%AF%95)
|
||||
1
plugins/wasm-go/extensions/request-validation/VERSION
Normal file
1
plugins/wasm-go/extensions/request-validation/VERSION
Normal file
@@ -0,0 +1 @@
|
||||
1.0.0
|
||||
18
plugins/wasm-go/extensions/request-validation/go.mod
Normal file
18
plugins/wasm-go/extensions/request-validation/go.mod
Normal file
@@ -0,0 +1,18 @@
|
||||
module github.com/alibaba/higress/plugins/wasm-go/extensions/request-validation
|
||||
|
||||
go 1.19
|
||||
|
||||
require (
|
||||
github.com/alibaba/higress/plugins/wasm-go v1.3.1
|
||||
github.com/santhosh-tekuri/jsonschema v1.2.4
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0
|
||||
github.com/tidwall/gjson v1.17.0
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/google/uuid v1.3.0 // indirect
|
||||
github.com/higress-group/nottinygc v0.0.0-20231101025119-e93c4c2f8520 // indirect
|
||||
github.com/magefile/mage v1.14.0 // indirect
|
||||
github.com/tidwall/match v1.1.1 // indirect
|
||||
github.com/tidwall/pretty v1.2.0 // indirect
|
||||
)
|
||||
22
plugins/wasm-go/extensions/request-validation/go.sum
Normal file
22
plugins/wasm-go/extensions/request-validation/go.sum
Normal file
@@ -0,0 +1,22 @@
|
||||
github.com/alibaba/higress/plugins/wasm-go v1.3.1 h1:d+t4W2NyqmqUz6DPZENflODfkLgdVlTfyso+nq0fSkg=
|
||||
github.com/alibaba/higress/plugins/wasm-go v1.3.1/go.mod h1:WZ/68vwe8qWhusa6C4/gMwUqas0jvHWSOa1bp8iK8F4=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
|
||||
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/higress-group/nottinygc v0.0.0-20231101025119-e93c4c2f8520 h1:IHDghbGQ2DTIXHBHxWfqCYQW1fKjyJ/I7W1pMyUDeEA=
|
||||
github.com/higress-group/nottinygc v0.0.0-20231101025119-e93c4c2f8520/go.mod h1:Nz8ORLaFiLWotg6GeKlJMhv8cci8mM43uEnLA5t8iew=
|
||||
github.com/magefile/mage v1.14.0 h1:6QDX3g6z1YvJ4olPhT1wksUcSa/V0a1B+pJb73fBjyo=
|
||||
github.com/magefile/mage v1.14.0/go.mod h1:z5UZb/iS3GoOSn0JgWuiw7dxlurVYTu+/jHXqQg881A=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/santhosh-tekuri/jsonschema v1.2.4 h1:hNhW8e7t+H1vgY+1QeEQpveR6D4+OwKPXCfD2aieJis=
|
||||
github.com/santhosh-tekuri/jsonschema v1.2.4/go.mod h1:TEAUOeZSmIxTTuHatJzrvARHiuO9LYd+cIxzgEHCQI4=
|
||||
github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0 h1:kS7BvMKN+FiptV4pfwiNX8e3q14evxAWkhYbxt8EI1M=
|
||||
github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0/go.mod h1:qkW5MBz2jch2u8bS59wws65WC+Gtx3x0aPUX5JL7CXI=
|
||||
github.com/tidwall/gjson v1.17.0 h1:/Jocvlh98kcTfpN2+JzGQWQcqrPQwDrVEMApx/M5ZwM=
|
||||
github.com/tidwall/gjson v1.17.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
|
||||
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
|
||||
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
|
||||
github.com/tidwall/pretty v1.2.0 h1:RWIZEg2iJ8/g6fDDYzMpobmaoGh5OLl4AXtGUGPcqCs=
|
||||
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
199
plugins/wasm-go/extensions/request-validation/main.go
Normal file
199
plugins/wasm-go/extensions/request-validation/main.go
Normal file
@@ -0,0 +1,199 @@
|
||||
// Copyright (c) 2022 Alibaba Group Holding Ltd.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/alibaba/higress/plugins/wasm-go/pkg/wrapper"
|
||||
"github.com/santhosh-tekuri/jsonschema"
|
||||
"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm"
|
||||
"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm/types"
|
||||
"github.com/tidwall/gjson"
|
||||
)
|
||||
|
||||
const (
|
||||
defaultHeaderSchema = "header"
|
||||
defaultBodySchema = "body"
|
||||
defaultRejectedCode = 403
|
||||
)
|
||||
|
||||
func main() {
|
||||
wrapper.SetCtx(
|
||||
"request-validation",
|
||||
wrapper.ProcessRequestHeadersBy(onHttpRequestHeaders),
|
||||
wrapper.ProcessRequestBodyBy(onHttpRequestBody),
|
||||
wrapper.ParseConfigBy(parseConfig),
|
||||
)
|
||||
}
|
||||
|
||||
// Config is the config for request validation.
|
||||
type Config struct {
|
||||
// compiler is the compiler for json schema.
|
||||
compiler *jsonschema.Compiler
|
||||
// rejectedCode is the code for rejected request.
|
||||
rejectedCode uint32
|
||||
// rejectedMsg is the message for rejected request.
|
||||
rejectedMsg string
|
||||
// draft is the draft version of json schema.
|
||||
draft *jsonschema.Draft
|
||||
// enableBodySchema is the flag for enable body schema.
|
||||
enableBodySchema bool
|
||||
// enableHeaderSchema is the flag for enable header schema.
|
||||
enableHeaderSchema bool
|
||||
}
|
||||
|
||||
func parseConfig(result gjson.Result, config *Config, log wrapper.Log) error {
|
||||
headerSchema := result.Get("header_schema").String()
|
||||
bodySchema := result.Get("body_schema").String()
|
||||
enableSwagger := result.Get("enable_swagger").Bool()
|
||||
enableOas3 := result.Get("enable_oas3").Bool()
|
||||
code := result.Get("rejected_code").Int()
|
||||
msg := result.Get("rejected_msg").String()
|
||||
|
||||
// set config default value
|
||||
config.enableBodySchema = false
|
||||
config.enableHeaderSchema = false
|
||||
|
||||
// check enable_swagger and enable_oas3
|
||||
if enableSwagger && enableOas3 {
|
||||
return fmt.Errorf("enable_swagger and enable_oas3 can not be true at the same time")
|
||||
}
|
||||
|
||||
// set draft version
|
||||
if enableSwagger {
|
||||
config.draft = jsonschema.Draft4
|
||||
}
|
||||
if enableOas3 {
|
||||
config.draft = jsonschema.Draft7
|
||||
}
|
||||
if !enableSwagger && !enableOas3 {
|
||||
config.draft = jsonschema.Draft7
|
||||
}
|
||||
|
||||
// create compiler
|
||||
compiler := jsonschema.NewCompiler()
|
||||
compiler.Draft = config.draft
|
||||
config.compiler = compiler
|
||||
|
||||
// add header schema to compiler
|
||||
if headerSchema != "" {
|
||||
err := config.compiler.AddResource(defaultHeaderSchema, strings.NewReader(headerSchema))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
config.enableHeaderSchema = true
|
||||
}
|
||||
|
||||
// add body schema to compiler
|
||||
if bodySchema != "" {
|
||||
err := config.compiler.AddResource(defaultBodySchema, strings.NewReader(bodySchema))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
config.enableBodySchema = true
|
||||
}
|
||||
|
||||
// check rejected_code is valid
|
||||
if code != 0 && code > 100 && code < 600 {
|
||||
config.rejectedCode = uint32(code)
|
||||
} else {
|
||||
config.rejectedCode = defaultRejectedCode
|
||||
}
|
||||
config.rejectedMsg = msg
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func onHttpRequestHeaders(ctx wrapper.HttpContext, config Config, log wrapper.Log) types.Action {
|
||||
if !config.enableHeaderSchema {
|
||||
return types.ActionContinue
|
||||
}
|
||||
|
||||
// get headers
|
||||
headers, err := proxywasm.GetHttpRequestHeaders()
|
||||
if err != nil {
|
||||
log.Errorf("get request headers failed: %v", err)
|
||||
return types.ActionContinue
|
||||
}
|
||||
|
||||
// covert to schema
|
||||
schema := make(map[string]interface{})
|
||||
for _, header := range headers {
|
||||
schema[header[0]] = header[1]
|
||||
}
|
||||
|
||||
// convert to json string
|
||||
schemaBytes, err := json.Marshal(schema)
|
||||
if err != nil {
|
||||
log.Errorf("marshal schema failed: %v", err)
|
||||
return types.ActionContinue
|
||||
}
|
||||
|
||||
// validate
|
||||
document := strings.NewReader(string(schemaBytes))
|
||||
compile, err := config.compiler.Compile(defaultHeaderSchema)
|
||||
if err != nil {
|
||||
log.Errorf("compile schema failed: %v", err)
|
||||
return types.ActionContinue
|
||||
}
|
||||
err = compile.Validate(document)
|
||||
if err != nil {
|
||||
log.Errorf("validate request headers failed: %v", err)
|
||||
proxywasm.SendHttpResponse(config.rejectedCode, nil, []byte(config.rejectedMsg), -1)
|
||||
return types.ActionPause
|
||||
}
|
||||
|
||||
return types.ActionContinue
|
||||
}
|
||||
|
||||
func onHttpRequestBody(ctx wrapper.HttpContext, config Config, body []byte, log wrapper.Log) types.Action {
|
||||
if !config.enableBodySchema {
|
||||
return types.ActionContinue
|
||||
}
|
||||
|
||||
// covert to schema
|
||||
schema := make(map[string]interface{})
|
||||
err := json.Unmarshal(body, &schema)
|
||||
if err != nil {
|
||||
log.Errorf("unmarshal body failed: %v", err)
|
||||
return types.ActionContinue
|
||||
}
|
||||
|
||||
// convert to json string
|
||||
schemaBytes, err := json.Marshal(schema)
|
||||
if err != nil {
|
||||
log.Errorf("marshal schema failed: %v", err)
|
||||
return types.ActionContinue
|
||||
}
|
||||
|
||||
// validate
|
||||
document := strings.NewReader(string(schemaBytes))
|
||||
compile, err := config.compiler.Compile(defaultBodySchema)
|
||||
if err != nil {
|
||||
log.Errorf("compile schema failed: %v", err)
|
||||
return types.ActionContinue
|
||||
}
|
||||
err = compile.Validate(document)
|
||||
if err != nil {
|
||||
log.Errorf("validate request body failed: %v", err)
|
||||
proxywasm.SendHttpResponse(config.rejectedCode, nil, []byte(config.rejectedMsg), -1)
|
||||
return types.ActionPause
|
||||
}
|
||||
|
||||
return types.ActionContinue
|
||||
}
|
||||
@@ -5,15 +5,16 @@
|
||||
# 配置字段
|
||||
|
||||
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|
||||
| -------- | -------- | -------- | -------- | -------- |
|
||||
| type | string | 必填,可选值为 `request`, `response` | - | 指定转换器类型 |
|
||||
| rules | array of object | 选填 | - | 指定转换操作类型以及请求/响应头、请求查询参数、请求/响应体参数的转换规则 |
|
||||
| :----: | :----: | :----: | :----: | -------- |
|
||||
| reqRules | string | 选填,reqRules和respRules至少填一个 | - | 请求转换器配置,指定转换操作类型以及请求头、请求查询参数、请求体的转换规则 |
|
||||
| respRules | string | 选填,reqRules和respRules至少填一个 | - | 响应转换器配置,指定转换操作类型以及响应头、响应体的转换规则 |
|
||||
|
||||
`rules`中每一项的配置字段说明如下:
|
||||
`reqRules`和`respRules`中每一项的配置字段说明如下:
|
||||
|
||||
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|
||||
| -------- | -------- | -------- | -------- | -------- |
|
||||
| :----: | :----: | :----: | :----: | -------- |
|
||||
| operate | string | 必填,可选值为 `remove`, `rename`, `replace`, `add`, `append`, `map`, `dedupe` | - | 指定转换操作类型,支持的操作类型有删除 (remove)、重命名 (rename)、更新 (replace)、添加 (add)、追加 (append)、映射 (map)、去重 (dedupe),当存在多项不同类型的转换规则时,按照上述操作类型顺序依次执行 |
|
||||
| mapSource | string | 选填,可选值为`headers`, `querys`,`body` | - | 仅在operate为`map`时有效。指定映射来源,若不填该字段,则默认映射来源为自身 |
|
||||
| headers | array of object | 选填 | - | 指定请求/响应头转换规则 |
|
||||
| querys | array of object | 选填 | - | 指定请求查询参数转换规则 |
|
||||
| body | array of object | 选填 | - | 指定请求/响应体参数转换规则,请求体转换允许 content-type 为 `application/json`, `application/x-www-form-urlencoded`, `multipart/form-data`;响应体转换仅允许 content-type 为 `application/json` |
|
||||
@@ -21,9 +22,20 @@
|
||||
`headers`, `querys`, `body`中每一项的配置字段说明如下:
|
||||
|
||||
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|
||||
| -------- | -------- | -------- | -------- |---------------------------------------------------|
|
||||
| key | string | 选填 | - | 指定键,详见[转换操作类型](#转换操作类型) |
|
||||
| value | string | 选填 | - | 指定值,详见[转换操作类型](#转换操作类型) |
|
||||
| :----: | :----: | :----: | -------- |---------------------------------------------------|
|
||||
| key | string | 选填 | - | 在operate为`remove`时使用,用法详见[转换操作类型](#转换操作类型) |
|
||||
| oldKey | string | 选填 | - |在operate为`rename`时使用,用法详见[转换操作类型](#转换操作类型) |
|
||||
| newKey | string | 选填 | - | 在operate为`rename`时使用,用法详见[转换操作类型](#转换操作类型) |
|
||||
| key | string | 选填 | - | 在operate为`replace`时使用,用法详见[转换操作类型](#转换操作类型) |
|
||||
| newValue | string | 选填 | - | 在operate为`replace`时使用,用法详见[转换操作类型](#转换操作类型) |
|
||||
| key | string | 选填 | - | 在operate为`add`时使用,用法详见[转换操作类型](#转换操作类型) |
|
||||
| value | string | 选填 | - | 在operate为`add`时使用,用法详见[转换操作类型](#转换操作类型) |
|
||||
| key | string | 选填 | - | 在operate为`append`时使用,用法详见[转换操作类型](#转换操作类型) |
|
||||
| appendValue | string | 选填 | - | 在operate为`append`时使用,用法详见[转换操作类型](#转换操作类型) |
|
||||
| fromKey | string | 选填 | - | 在operate为`map`时使用,用法详见[转换操作类型](#转换操作类型) |
|
||||
| toKey | string | 选填 | - | 在operate为`map`时使用,用法详见[转换操作类型](#转换操作类型) |
|
||||
| key | string | 选填 | - | 在operate为`dedupe`时使用,用法详见[转换操作类型](#转换操作类型) |
|
||||
| strategy | string | 选填 | - | 在operate为`dedupe`时使用,用法详见[转换操作类型](#转换操作类型) |
|
||||
| value_type | string | 选填,可选值为 `object`, `boolean`, `number`, `string` | string | 当`content-type: application/json`时,该字段指定请求/响应体参数的值类型 |
|
||||
| host_pattern | string | 选填 | - | 指定请求主机名匹配规则,当转换操作类型为 `replace`, `add`, `append` 时有效 |
|
||||
| path_pattern | string | 选填 | - | 指定请求路径匹配规则,当转换操作类型为 `replace`, `add`, `append` 时有效 |
|
||||
@@ -32,8 +44,8 @@
|
||||
|
||||
* `request transformer` 支持以下转换对象:请求头部、请求查询参数、请求体(application/json, application/x-www-form-urlencoded, multipart/form-data)
|
||||
* `response transformer` 支持以下转换对象:响应头部、响应体(application/json)
|
||||
|
||||
* 转换操作类型的执行顺序:remove → rename → replace → add → append → map → dedupe
|
||||
* 插件支持双向转换能力,即单个插件能够完成对请求和响应都做转换
|
||||
* 转换操作类型的执行顺序,为配置文件中编写的顺序,如:remove → rename → replace → add → append → map → dedupe或者dedupe → map → append → add → replace → rename → remove等
|
||||
* 当转换对象为 headers 时,` key` 不区分大小写;当为 headers 且为 `rename`, `map` 操作时,`value` 也不区分大小写(因为此时该字段具有 key 含义);而 querys 和 body 的 `key`, `value` 字段均区分大小写
|
||||
* `value_type` 仅对 content-type 为 application/json 的请求/响应体有效
|
||||
* `host_pattern` 和 `path_pathern` 支持 [RE2 语法](https://pkg.go.dev/regexp/syntax),仅对 `replace`, `add`, `append` 操作有效,且在一项转换规则中两者只能选填其一,若均填写,则 `host_pattern` 生效,而 `path_pattern` 失效
|
||||
@@ -43,7 +55,7 @@
|
||||
# 转换操作类型
|
||||
|
||||
| 操作类型 | key 字段含义 | value 字段含义 | 描述 |
|
||||
| ------------- | ----------------- |-----| ------------------------------------------------------------ |
|
||||
| :----: | :----: | :----: | ------------------------------------------------------------ |
|
||||
| 删除 remove | 目标 key |无需设置| 若存在指定的 `key`,则删除;否则无操作 |
|
||||
| 重命名 rename | 目标 oldKey |新的 key 名称 newKey| 若存在指定的 `oldKey:value`,则将其键名重命名为 `newKey`,得到 `newKey:value`;否则无操作 |
|
||||
| 更新 replace | 目标 key |新的 value 值 newValue| 若存在指定的 `key:value`,则将其 value 更新为 `newValue`,得到 `key:newValue`;否则无操作 |
|
||||
@@ -62,19 +74,18 @@
|
||||
### 转换请求头部
|
||||
|
||||
```yaml
|
||||
type: request
|
||||
rules:
|
||||
reqRules:
|
||||
- operate: remove
|
||||
headers:
|
||||
- key: X-remove
|
||||
- operate: rename
|
||||
headers:
|
||||
- key: X-not-renamed
|
||||
value: X-renamed
|
||||
- oldKey: X-not-renamed
|
||||
newKey: X-renamed
|
||||
- operate: replace
|
||||
headers:
|
||||
- key: X-replace
|
||||
value: replaced
|
||||
newValue: replaced
|
||||
- operate: add
|
||||
headers:
|
||||
- key: X-add-append
|
||||
@@ -83,20 +94,20 @@ rules:
|
||||
- operate: append
|
||||
headers:
|
||||
- key: X-add-append
|
||||
value: path-$1
|
||||
appendValue: path-$1
|
||||
path_pattern: ^.*?\/(\w+)[\?]{0,1}.*$
|
||||
- operate: map
|
||||
headers:
|
||||
- key: X-add-append
|
||||
value: X-map
|
||||
- fromKey: X-add-append
|
||||
toKey: X-map
|
||||
- operate: dedupe
|
||||
headers:
|
||||
- key: X-dedupe-first
|
||||
value: RETAIN_FIRST
|
||||
strategy: RETAIN_FIRST
|
||||
- key: X-dedupe-last
|
||||
value: RETAIN_LAST
|
||||
strategy: RETAIN_LAST
|
||||
- key: X-dedupe-unique
|
||||
value: RETAIN_UNIQUE
|
||||
strategy: RETAIN_UNIQUE
|
||||
```
|
||||
|
||||
发送请求
|
||||
@@ -131,19 +142,18 @@ $ curl -v console.higress.io/get -H 'host: foo.bar.com' \
|
||||
### 转换请求查询参数
|
||||
|
||||
```yaml
|
||||
type: request
|
||||
rules:
|
||||
reqRules:
|
||||
- operate: remove
|
||||
querys:
|
||||
- key: k1
|
||||
- operate: rename
|
||||
querys:
|
||||
- key: k2
|
||||
value: k2-new
|
||||
- oldKey: k2
|
||||
newKey: k2-new
|
||||
- operate: replace
|
||||
querys:
|
||||
- key: k2-new
|
||||
value: v2-new
|
||||
newValue: v2-new
|
||||
- operate: add
|
||||
querys:
|
||||
- key: k3
|
||||
@@ -152,15 +162,15 @@ rules:
|
||||
- operate: append
|
||||
querys:
|
||||
- key: k3
|
||||
value: v32
|
||||
appendValue: v32
|
||||
- operate: map
|
||||
querys:
|
||||
- key: k3
|
||||
value: k4
|
||||
- fromKey: k3
|
||||
toKey: k4
|
||||
- operate: dedupe
|
||||
querys:
|
||||
- key: k4
|
||||
value: RETAIN_FIRST
|
||||
strategy: RETAIN_FIRST
|
||||
```
|
||||
|
||||
发送请求
|
||||
@@ -186,19 +196,18 @@ $ curl -v "console.higress.io/get?k1=v11&k1=v12&k2=v2"
|
||||
### 转换请求体
|
||||
|
||||
```yaml
|
||||
type: request
|
||||
rules:
|
||||
reqRules:
|
||||
- operate: remove
|
||||
body:
|
||||
- key: a1
|
||||
- operate: rename
|
||||
body:
|
||||
- key: a2
|
||||
value: a2-new
|
||||
- oldKey: a2
|
||||
newKey: a2-new
|
||||
- operate: replace
|
||||
body:
|
||||
- key: a3
|
||||
value: t3-new
|
||||
newValue: t3-new
|
||||
value_type: string
|
||||
- operate: add
|
||||
body:
|
||||
@@ -208,17 +217,17 @@ rules:
|
||||
- operate: append
|
||||
body:
|
||||
- key: a1-new
|
||||
value: t1-$1-append
|
||||
appendValue: t1-$1-append
|
||||
value_type: string
|
||||
host_pattern: ^(.*)\.com$
|
||||
- operate: map
|
||||
body:
|
||||
- key: a1-new
|
||||
value: a4
|
||||
- fromKey: a1-new
|
||||
toKey: a4
|
||||
- operate: dedupe
|
||||
body:
|
||||
- key: a4
|
||||
value: RETAIN_FIRST
|
||||
strategy: RETAIN_FIRST
|
||||
```
|
||||
|
||||
发送请求:
|
||||
@@ -313,8 +322,7 @@ $ curl -v -X POST console.higress.io/post -H 'host: foo.bar.com' \
|
||||
1.通常情况下,指定的 key 中含有 `.` 表示嵌套含义,如下:
|
||||
|
||||
```yaml
|
||||
type: response
|
||||
rules:
|
||||
respRules:
|
||||
- operate: add
|
||||
body:
|
||||
- key: foo.bar
|
||||
@@ -339,8 +347,7 @@ $ curl -v console.higress.io/get
|
||||
> 当使用双引号括住字符串时使用 `\\.` 进行转义
|
||||
|
||||
```yaml
|
||||
type: response
|
||||
rules:
|
||||
respRules:
|
||||
- operate: add
|
||||
body:
|
||||
- key: foo\.bar
|
||||
@@ -378,8 +385,7 @@ $ curl -v console.higress.io/get
|
||||
1.移除 `user` 第一个元素:
|
||||
|
||||
```yaml
|
||||
type: request
|
||||
rules:
|
||||
reqRules:
|
||||
- operate: remove
|
||||
body:
|
||||
- key: users.0
|
||||
@@ -409,12 +415,11 @@ $ curl -v -X POST console.higress.io/post \
|
||||
2.将 `users` 第一个元素的 key 为 `123` 重命名为 `msg`:
|
||||
|
||||
```yaml
|
||||
type: request
|
||||
rules:
|
||||
reqRules:
|
||||
- operate: rename
|
||||
body:
|
||||
- key: users.0.123
|
||||
value: users.0.first
|
||||
- oldKey: users.0.123
|
||||
newKey: users.0.first
|
||||
```
|
||||
|
||||
```bash
|
||||
@@ -466,12 +471,11 @@ $ curl -v -X POST console.higress.io/post \
|
||||
```
|
||||
|
||||
```yaml
|
||||
type: request
|
||||
rules:
|
||||
reqRules:
|
||||
- operate: replace
|
||||
body:
|
||||
- key: users.#.age
|
||||
value: 20
|
||||
newValue: 20
|
||||
```
|
||||
|
||||
```bash
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user