When IP_PROXY_PROVIDER_NAME in config.py is set to an unrecognised
value (e.g. typo "kuaidalli" instead of "kuaidaili"), IpProxyProvider.get()
returns None which is silently stored in ProxyIpPool.ip_provider. The
crash surfaces later as:
AttributeError: 'NoneType' object has no attribute 'get_proxy'
with no hint about what went wrong or how to fix it.
Add an explicit None-check in create_ip_pool() that raises ValueError
with the bad name and the list of valid options, so the root cause is
immediately visible. Also add a regression test covering the four
invalid-name edge cases (typo, empty string, wrong case, unknown name)
and the three valid names to confirm they are unaffected.
The PR adds API limit overrides and static proxy support, but the review found that the default proxy provider changed to an invalid static placeholder and the new API fields accepted unbounded values. This keeps the existing proxy default intact, makes static proxy explicit via config or CLI, validates API limit ranges, and adds focused regression coverage for both paths.
Constraint: PR branch must remain contributor-branch compatible and avoid adding dependencies
Rejected: Keep static as the default provider | breaks existing --enable_ip_proxy defaults with an invalid placeholder URL
Rejected: Accept arbitrary integer limits | lets API callers request negative or excessive crawl sizes
Confidence: high
Scope-risk: narrow
Directive: Do not change proxy provider defaults when adding new providers; new providers should be opt-in and covered by provider-specific tests
Tested: uv run pytest tests/test_api_limits.py tests/test_static_proxy_provider.py
Tested: uv run pytest tests
Tested: uv run pytest test/test_utils.py
Tested: uv run python -m compileall api cmd_arg config proxy tests
Tested: git diff --cached --check
Not-tested: Live crawler run against external platforms or real proxy vendor endpoints
Migrate remaining httpx.AsyncClient call sites in proxy/ package to
use make_async_client(), completing the DISABLE_SSL_VERIFY coverage
across all outbound HTTP requests in the project.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comprehensive translation of Chinese text to English across the entire codebase:
- api/: FastAPI server documentation and logger messages
- cache/: Cache abstraction layer comments and docstrings
- database/: Database models and MongoDB store documentation
- media_platform/: All platform crawlers (Bilibili, Douyin, Kuaishou, Tieba, Weibo, Xiaohongshu, Zhihu)
- model/: Data model documentation
- proxy/: Proxy pool and provider documentation
- store/: Data storage layer comments
- tools/: Utility functions and browser automation
- test/: Test file documentation
Preserved: Chinese disclaimer header (lines 10-18) for legal compliance
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
