From 3e7c559997e7ba2b4d7420022923f8fa62ac3361 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E6=BE=84=E6=BD=AD?= Date: Sat, 31 Jan 2026 13:42:44 +0800 Subject: [PATCH] feat(skill): improve nginx-to-higress-migration with critical warnings and guides (#3417) --- .../nginx-to-higress-migration/SKILL.md | 173 ++++++++++++++++++ .../references/annotation-mapping.md | 34 ++++ 2 files changed, 207 insertions(+) diff --git a/.claude/skills/nginx-to-higress-migration/SKILL.md b/.claude/skills/nginx-to-higress-migration/SKILL.md index f151f3760..737cf15b1 100644 --- a/.claude/skills/nginx-to-higress-migration/SKILL.md +++ b/.claude/skills/nginx-to-higress-migration/SKILL.md @@ -7,6 +7,22 @@ description: "Migrate from ingress-nginx to Higress in Kubernetes environments. Automate migration from ingress-nginx to Higress in Kubernetes environments. +## ⚠️ Critical Limitation: Snippet Annotations NOT Supported + +> **Before you begin:** Higress does **NOT** support the following nginx annotations: +> - `nginx.ingress.kubernetes.io/server-snippet` +> - `nginx.ingress.kubernetes.io/configuration-snippet` +> - `nginx.ingress.kubernetes.io/http-snippet` +> +> These annotations will be **silently ignored**, causing functionality loss! +> +> **Pre-migration check (REQUIRED):** +> ```bash +> kubectl get ingress -A -o yaml | grep -E "snippet" | wc -l +> ``` +> If count > 0, you MUST plan WASM plugin replacements before migration. +> See [Phase 6](#phase-6-use-built-in-plugins-or-create-custom-wasm-plugin-if-needed) for alternatives. + ## Prerequisites - kubectl configured with cluster access @@ -14,6 +30,41 @@ Automate migration from ingress-nginx to Higress in Kubernetes environments. - Go 1.24+ (for WASM plugin compilation) - Docker (for plugin image push) +## Pre-Migration Checklist + +### Before Starting + +- [ ] Backup all Ingress resources + ```bash + kubectl get ingress -A -o yaml > ingress-backup.yaml + ``` +- [ ] Identify snippet usage (see warning above) +- [ ] List all nginx annotations in use + ```bash + kubectl get ingress -A -o yaml | grep "nginx.ingress.kubernetes.io" | sort | uniq -c + ``` +- [ ] Verify Higress compatibility for each annotation (see [annotation-mapping.md](references/annotation-mapping.md)) +- [ ] Plan WASM plugins for unsupported features +- [ ] Prepare test environment (Kind/Minikube for testing recommended) + +### During Migration + +- [ ] Install Higress in parallel with nginx +- [ ] Verify all pods running in higress-system namespace +- [ ] Run test script against Higress gateway +- [ ] Compare responses between nginx and Higress +- [ ] Deploy any required WASM plugins +- [ ] Configure monitoring/alerting + +### After Migration + +- [ ] All routes verified working +- [ ] Custom functionality (snippet replacements) tested +- [ ] Monitoring dashboards configured +- [ ] Team trained on Higress operations +- [ ] Documentation updated +- [ ] Rollback procedure tested + ## Migration Workflow ### Phase 1: Discovery @@ -108,6 +159,47 @@ helm upgrade higress higress/higress -n higress-system \ --set global.enableStatus=true ``` +#### Kind/Local Environment Setup + +In Kind or local Kubernetes clusters, the LoadBalancer service will stay in `PENDING` state. Use one of these methods: + +**Option 1: Port Forward (Recommended for testing)** +```bash +# Forward Higress gateway to local port +kubectl port-forward -n higress-system svc/higress-gateway 8080:80 8443:443 & + +# Test with Host header +curl -H "Host: example.com" http://localhost:8080/ +``` + +**Option 2: NodePort** +```bash +# Patch service to NodePort +kubectl patch svc -n higress-system higress-gateway \ + -p '{"spec":{"type":"NodePort"}}' + +# Get assigned port +NODE_PORT=$(kubectl get svc -n higress-system higress-gateway \ + -o jsonpath='{.spec.ports[?(@.port==80)].nodePort}') + +# Test (use docker container IP for Kind) +curl -H "Host: example.com" http://localhost:${NODE_PORT}/ +``` + +**Option 3: Kind with Port Mapping (Requires cluster recreation)** +```yaml +# kind-config.yaml +kind: Cluster +apiVersion: kind.x-k8s.io/v1alpha4 +nodes: +- role: control-plane + extraPortMappings: + - containerPort: 30080 + hostPort: 80 + - containerPort: 30443 + hostPort: 443 +``` + ### Phase 4: Generate and Run Test Script After Higress is running, generate a test script covering all Ingress routes: @@ -181,6 +273,15 @@ Common replacements for nginx features: | Custom response | `custom-response` | | Request/Response transform | `transformer` | +#### Common Snippet Replacements + +| nginx snippet pattern | Higress solution | +|----------------------|------------------| +| Custom health endpoint (`location /health`) | WASM plugin: custom-location | +| Add response headers | WASM plugin: custom-response-headers | +| Request validation/blocking | WASM plugin with `OnHttpRequestHeaders` | +| Lua rate limiting | `key-rate-limit` plugin | + #### Custom WASM Plugin (If No Built-in Matches) When nginx snippets or Lua logic has no built-in equivalent: @@ -272,6 +373,78 @@ The test script will: **Only proceed with traffic cutover after all tests pass!** +## Troubleshooting + +### Common Issues + +#### Q1: Ingress created but routes return 404 +**Symptoms:** Ingress shows Ready, but curl returns 404 + +**Check:** +1. Verify IngressClass matches Higress config + ```bash + kubectl get ingress -o yaml | grep ingressClassName + ``` +2. Check controller logs + ```bash + kubectl logs -n higress-system -l app=higress-controller --tail=100 + ``` +3. Verify backend service is reachable + ```bash + kubectl run test --rm -it --image=curlimages/curl -- \ + curl http://..svc + ``` + +#### Q2: rewrite-target not working +**Symptoms:** Path not being rewritten, backend receives original path + +**Solution:** Ensure `use-regex: "true"` is also set: +```yaml +annotations: + nginx.ingress.kubernetes.io/rewrite-target: /$2 + nginx.ingress.kubernetes.io/use-regex: "true" +``` + +#### Q3: Snippet annotations silently ignored +**Symptoms:** nginx snippet features not working after migration + +**Cause:** Higress does not support snippet annotations (by design, for security) + +**Solution:** +- Check [references/builtin-plugins.md](references/builtin-plugins.md) for built-in alternatives +- Create custom WASM plugin (see Phase 6) + +#### Q4: TLS certificate issues +**Symptoms:** HTTPS not working or certificate errors + +**Check:** +1. Verify Secret exists and is type `kubernetes.io/tls` + ```bash + kubectl get secret -o yaml + ``` +2. Check TLS configuration in Ingress + ```bash + kubectl get ingress -o jsonpath='{.spec.tls}' + ``` + +### Useful Debug Commands + +```bash +# View Higress controller logs +kubectl logs -n higress-system -l app=higress-controller -c higress-core + +# View gateway access logs +kubectl logs -n higress-system -l app=higress-gateway | grep "GET\|POST" + +# Check Envoy config dump +kubectl exec -n higress-system deploy/higress-gateway -c istio-proxy -- \ + curl -s localhost:15000/config_dump | jq '.configs[2].dynamic_listeners' + +# View gateway stats +kubectl exec -n higress-system deploy/higress-gateway -c istio-proxy -- \ + curl -s localhost:15000/stats | grep http +``` + ## Rollback Since nginx keeps running during migration, rollback is simply switching traffic back: diff --git a/.claude/skills/nginx-to-higress-migration/references/annotation-mapping.md b/.claude/skills/nginx-to-higress-migration/references/annotation-mapping.md index 40e3ecb9a..53c0125a9 100644 --- a/.claude/skills/nginx-to-higress-migration/references/annotation-mapping.md +++ b/.claude/skills/nginx-to-higress-migration/references/annotation-mapping.md @@ -96,6 +96,40 @@ These nginx annotations work directly with Higress without any changes: | `nginx.ingress.kubernetes.io/proxy-ssl-secret` | `higress.io/proxy-ssl-secret` | | `nginx.ingress.kubernetes.io/proxy-ssl-verify` | `higress.io/proxy-ssl-verify` | +### TLS Protocol & Cipher Control + +Higress provides fine-grained TLS control via dedicated annotations: + +| nginx annotation | Higress annotation | Notes | +|------------------|-------------------|-------| +| `nginx.ingress.kubernetes.io/ssl-protocols` | (see below) | Use Higress-specific annotations | + +**Higress TLS annotations (no nginx equivalent - use these directly):** + +| Higress annotation | Description | Example value | +|-------------------|-------------|---------------| +| `higress.io/tls-min-protocol-version` | Minimum TLS version | `TLSv1.2` | +| `higress.io/tls-max-protocol-version` | Maximum TLS version | `TLSv1.3` | +| `higress.io/ssl-cipher` | Allowed cipher suites | `ECDHE-RSA-AES128-GCM-SHA256` | + +**Example: Restrict to TLS 1.2+** +```yaml +# nginx (using ssl-protocols) +annotations: + nginx.ingress.kubernetes.io/ssl-protocols: "TLSv1.2 TLSv1.3" + +# Higress (use dedicated annotations) +annotations: + higress.io/tls-min-protocol-version: "TLSv1.2" + higress.io/tls-max-protocol-version: "TLSv1.3" +``` + +**Example: Custom cipher suites** +```yaml +annotations: + higress.io/ssl-cipher: "ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384" +``` + ## Unsupported Annotations (Require WASM Plugin) These annotations have no direct Higress equivalent and require custom WASM plugins: