程序员阿江(Relakkes)
d614ccf247
docs: translate comments and metadata to English
...
Update Chinese comments, variable descriptions, and metadata across
multiple configuration and core files to English. This improves
codebase accessibility for international developers. Additionally,
removed the sponsorship section from README files.
2026-02-12 05:30:11 +08:00
ouzhuowei
e54463ac78
处理子评论获取失败导致整个流程中断问题
...
Co-Authored-By: ouzhuowei <190020754@qq.com >
2026-02-10 17:53:30 +08:00
程序员阿江(Relakkes)
c309871485
refactor(xhs): improve login state check logic
2026-02-03 20:49:46 +08:00
程序员阿江(Relakkes)
6625663bde
feat: #823
2026-02-03 20:40:15 +08:00
wanzirong
f7d27ab43a
feat: 添加命令行参数支持
...
- 添加 --max_comments_per_post 参数用于控制每个帖子爬取的评论数量
- 添加 --xhs_sort_type 参数用于控制小红书排序方式
- 修复小红书 core.py 中 CRAWLER_MAX_COMMENTS_COUNT_SINGLENOTES 的导入方式
从直接导入改为通过 config 模块访问，使命令行参数能正确生效
2026-01-21 16:23:47 +08:00
bear
a59b385615
fix the login status error after scan the QR code
2026-01-09 14:11:47 +08:00
程序员阿江(Relakkes)
157ddfb21b
i18n: translate all Chinese comments, docstrings, and logger messages to English
...
Comprehensive translation of Chinese text to English across the entire codebase:
- api/: FastAPI server documentation and logger messages
- cache/: Cache abstraction layer comments and docstrings
- database/: Database models and MongoDB store documentation
- media_platform/: All platform crawlers (Bilibili, Douyin, Kuaishou, Tieba, Weibo, Xiaohongshu, Zhihu)
- model/: Data model documentation
- proxy/: Proxy pool and provider documentation
- store/: Data storage layer comments
- tools/: Utility functions and browser automation
- test/: Test file documentation
Preserved: Chinese disclaimer header (lines 10-18) for legal compliance
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2025-12-26 23:27:19 +08:00
MEI
ff9a1624f1
fix: params参数以及路径问题
2025-12-03 10:31:32 +08:00
程序员阿江(Relakkes)
f989ce0788
feat: xhs sign playwright version
2025-11-27 10:53:08 +08:00
程序员阿江(Relakkes)
6eef02d08c
feat: ip proxy expired check
2025-11-25 12:39:10 +08:00
程序员阿江(Relakkes)
ff8c92daad
chore: add copyright to every file
2025-11-18 12:24:02 +08:00
程序员阿江(Relakkes)
a1c5e07df8
fix: xhs sub comment bugfix #769
2025-11-17 11:47:33 +08:00
程序员阿江(Relakkes)
b6caa7a85e
refactor: add xhs creator params
2025-11-10 21:10:03 +08:00
程序员阿江(Relakkes)
1e3637f238
refactor: update xhs note detail
2025-11-10 18:13:51 +08:00
程序员阿江(Relakkes)
b5dab6d1e8
refactor: 使用 xhshow 替代 playwright 签名方案
...
感谢 @Cloxl/xhshow 开源项目
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-11-10 18:12:45 +08:00
程序员阿江(Relakkes)
3f5925e326
feat: update xhs sign
2025-10-27 19:06:07 +08:00
程序员阿江(Relakkes)
03e384bbe2
refactor: cdp模式下移除stealth注入
2025-10-19 15:32:03 +08:00
程序员阿江(Relakkes)
a9dd08680f
feat: xhs support creator url link
2025-10-18 07:20:09 +08:00
刘小龙
c87df59996
log client modify
2025-09-09 15:27:46 +08:00
程序员阿江(Relakkes)
2bce3593f7
feat: support time deplay for all platform
2025-09-02 16:43:09 +08:00
程序员阿江(Relakkes)
eb799e1fa7
refactor: xhs extractor
2025-09-02 14:50:32 +08:00
未来可欺
6a10d0d11c
原始的HTTPStatusError不能捕获像ConnectError、ReadError这些异常类型，本次提交修改了捕获异常的类型为httpx模块请求异常的基类：HTTPError，以便捕获在httpx.request方法中引发的任何异常（例如ip被封，服务器拒接连接），正确处理爬取媒体被中断时并不会导致爬取文本的中断逻辑
2025-08-06 11:24:51 +08:00
未来可欺
81f2dbe4ab
添加了对媒体资源服务器的异常处理，参见 issue #691
2025-08-05 13:11:00 +08:00
程序员阿江(Relakkes)
b9d30bbabb
fix : #693
2025-08-01 15:55:21 +08:00
未来可欺
0b81240aed
升级 httpx 版本至 0.28.1，并修改关键字参数 proxies 至 proxy
2025-07-31 22:48:02 +08:00
未来可欺
93a1c27fff
通过测试search模式，修复部分运行时的bug，并对能够爬取媒体的平台设置了较长的超时时间
2025-07-30 21:19:56 +08:00
未来可欺
173bc08a9d
添加了抖音存储视频以及图片的逻辑，并将config.py中ENABLE_GET_IMAGES参数更名为ENABLE_GET_MEIDAS，在此基础上略微修改存储逻辑
2025-07-30 18:24:08 +08:00
korruz
07a6e387ea
refactor: move format_proxy_info to utils and update crawler classes to use it
2025-07-29 14:16:24 +08:00
程序员阿江(Relakkes)
fc06c783f5
fix: fixed xhs req headers
2025-07-23 13:28:58 +08:00
程序员阿江(Relakkes)
a4d9aaa34a
refactor: xhs update
2025-07-21 21:26:16 +08:00
程序员阿江(Relakkes)
13b00f7a36
refactor: config update
2025-07-18 23:26:52 +08:00
gaoxiaobei
b913db64bb
refactor(config): move platform-specific configs to separate files
...
- Remove platform-specific configurations from base_config.py
- Create separate config files for each platform in their respective directories
- Update import statements in core files to use new platform-specific config modules
- Clean up unused and deprecated configuration options
2025-07-18 17:27:37 +08:00
程序员阿江(Relakkes)
e83b2422d9
feat: 支持playwright通过cdp协议连接本地chrome浏览器
...
docs: 增加uv来管理python依赖的文档
2025-06-25 23:22:39 +08:00
chimeElm
26a845581e
Update client.py
...
修复CRAWLER_MAX_NOTES_COUNT在爬取小红书作者帖子时失效的问题
2025-06-07 02:41:09 +08:00
crpa33
274d64aefc
处理xhs意外的评论信息为空的情况
...
报错就会打断我，我没辙
2025-04-02 11:59:27 +08:00
liudongkai
33e7ef016d
feat: xhs 非代理模式下增加随机等待间隔, db存储模式下增加存储xsec_token字段
2024-12-05 21:10:31 +08:00
leantli
e830ada574
feat: xhs comments add xsec_token
2024-12-03 18:25:21 +08:00
Relakkes
ca9b47ef63
fix: xhs 帖子详情优化
2024-11-27 09:41:24 +08:00
Relakkes
43dffeb2d1
feat: xhs帖子详情获取优化
2024-11-26 13:37:53 +08:00
lyx0727
705b810269
fix: xhs验证码处理，跳转链接修复
2024-11-01 22:08:46 +08:00
lyx0727
1a37df4d5e
fix: xhs出现验证码时报错，增加用户手动验证
2024-11-01 18:52:16 +08:00
程序员阿江(Relakkes)
5a27ad089c
Merge pull request #462 from FloRainRJY/xiaohongshu_comment_number_restrict
...
All_platform_comments_restrict
2024-10-24 15:31:13 +08:00
liugangdao
78c09c4ae1
fix:翻页时search id不变
2024-10-24 13:57:27 +08:00
unknown
7e53c4acfc
All_platform_comments_restrict
2024-10-23 16:32:02 +08:00
unknown
19269c66fd
xiaohongshu_comment_number_restrict
2024-10-22 20:33:10 +08:00
Relakkes
03e393949a
fix: xhs帖子详情问题更新
2024-10-20 00:59:08 +08:00
Relakkes
9fe3e47b0f
chore: 增加代码学习声明，严格禁止非法、禁止商业、不当用途
2024-10-20 00:43:25 +08:00
Relakkes
6dd3420743
fix : #423
2024-09-02 22:57:28 +08:00
Relakkes
d6fb255bdf
fix: xhs note detail error
2024-09-02 21:45:12 +08:00
Relakkes
65699aa1cb
feat: xhs支持获取评论的点赞数量
2024-08-24 06:07:33 +08:00
