wanzirong
f7d27ab43a
feat: 添加命令行参数支持
...
- 添加 --max_comments_per_post 参数用于控制每个帖子爬取的评论数量
- 添加 --xhs_sort_type 参数用于控制小红书排序方式
- 修复小红书 core.py 中 CRAWLER_MAX_COMMENTS_COUNT_SINGLENOTES 的导入方式
从直接导入改为通过 config 模块访问，使命令行参数能正确生效
2026-01-21 16:23:47 +08:00
bear
a59b385615
fix the login status error after scan the QR code
2026-01-09 14:11:47 +08:00
程序员阿江(Relakkes)
157ddfb21b
i18n: translate all Chinese comments, docstrings, and logger messages to English
...
Comprehensive translation of Chinese text to English across the entire codebase:
- api/: FastAPI server documentation and logger messages
- cache/: Cache abstraction layer comments and docstrings
- database/: Database models and MongoDB store documentation
- media_platform/: All platform crawlers (Bilibili, Douyin, Kuaishou, Tieba, Weibo, Xiaohongshu, Zhihu)
- model/: Data model documentation
- proxy/: Proxy pool and provider documentation
- store/: Data storage layer comments
- tools/: Utility functions and browser automation
- test/: Test file documentation
Preserved: Chinese disclaimer header (lines 10-18) for legal compliance
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2025-12-26 23:27:19 +08:00
MEI
ff9a1624f1
fix: params参数以及路径问题
2025-12-03 10:31:32 +08:00
程序员阿江(Relakkes)
f989ce0788
feat: xhs sign playwright version
2025-11-27 10:53:08 +08:00
程序员阿江(Relakkes)
6eef02d08c
feat: ip proxy expired check
2025-11-25 12:39:10 +08:00
程序员阿江(Relakkes)
ff8c92daad
chore: add copyright to every file
2025-11-18 12:24:02 +08:00
程序员阿江(Relakkes)
a1c5e07df8
fix: xhs sub comment bugfix #769
2025-11-17 11:47:33 +08:00
程序员阿江(Relakkes)
b6caa7a85e
refactor: add xhs creator params
2025-11-10 21:10:03 +08:00
程序员阿江(Relakkes)
1e3637f238
refactor: update xhs note detail
2025-11-10 18:13:51 +08:00
程序员阿江(Relakkes)
b5dab6d1e8
refactor: 使用 xhshow 替代 playwright 签名方案
...
感谢 @Cloxl/xhshow 开源项目
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-11-10 18:12:45 +08:00
程序员阿江(Relakkes)
3f5925e326
feat: update xhs sign
2025-10-27 19:06:07 +08:00
程序员阿江(Relakkes)
03e384bbe2
refactor: cdp模式下移除stealth注入
2025-10-19 15:32:03 +08:00
程序员阿江(Relakkes)
a9dd08680f
feat: xhs support creator url link
2025-10-18 07:20:09 +08:00
刘小龙
c87df59996
log client modify
2025-09-09 15:27:46 +08:00
程序员阿江(Relakkes)
2bce3593f7
feat: support time deplay for all platform
2025-09-02 16:43:09 +08:00
程序员阿江(Relakkes)
eb799e1fa7
refactor: xhs extractor
2025-09-02 14:50:32 +08:00
未来可欺
6a10d0d11c
原始的HTTPStatusError不能捕获像ConnectError、ReadError这些异常类型，本次提交修改了捕获异常的类型为httpx模块请求异常的基类：HTTPError，以便捕获在httpx.request方法中引发的任何异常（例如ip被封，服务器拒接连接），正确处理爬取媒体被中断时并不会导致爬取文本的中断逻辑
2025-08-06 11:24:51 +08:00
未来可欺
81f2dbe4ab
添加了对媒体资源服务器的异常处理，参见 issue #691
2025-08-05 13:11:00 +08:00
程序员阿江(Relakkes)
b9d30bbabb
fix : #693
2025-08-01 15:55:21 +08:00
未来可欺
0b81240aed
升级 httpx 版本至 0.28.1，并修改关键字参数 proxies 至 proxy
2025-07-31 22:48:02 +08:00
未来可欺
93a1c27fff
通过测试search模式，修复部分运行时的bug，并对能够爬取媒体的平台设置了较长的超时时间
2025-07-30 21:19:56 +08:00
未来可欺
173bc08a9d
添加了抖音存储视频以及图片的逻辑，并将config.py中ENABLE_GET_IMAGES参数更名为ENABLE_GET_MEIDAS，在此基础上略微修改存储逻辑
2025-07-30 18:24:08 +08:00
korruz
07a6e387ea
refactor: move format_proxy_info to utils and update crawler classes to use it
2025-07-29 14:16:24 +08:00
程序员阿江(Relakkes)
fc06c783f5
fix: fixed xhs req headers
2025-07-23 13:28:58 +08:00
程序员阿江(Relakkes)
a4d9aaa34a
refactor: xhs update
2025-07-21 21:26:16 +08:00
程序员阿江(Relakkes)
13b00f7a36
refactor: config update
2025-07-18 23:26:52 +08:00
gaoxiaobei
b913db64bb
refactor(config): move platform-specific configs to separate files
...
- Remove platform-specific configurations from base_config.py
- Create separate config files for each platform in their respective directories
- Update import statements in core files to use new platform-specific config modules
- Clean up unused and deprecated configuration options
2025-07-18 17:27:37 +08:00
程序员阿江(Relakkes)
e83b2422d9
feat: 支持playwright通过cdp协议连接本地chrome浏览器
...
docs: 增加uv来管理python依赖的文档
2025-06-25 23:22:39 +08:00
chimeElm
26a845581e
Update client.py
...
修复CRAWLER_MAX_NOTES_COUNT在爬取小红书作者帖子时失效的问题
2025-06-07 02:41:09 +08:00
crpa33
274d64aefc
处理xhs意外的评论信息为空的情况
...
报错就会打断我，我没辙
2025-04-02 11:59:27 +08:00
liudongkai
33e7ef016d
feat: xhs 非代理模式下增加随机等待间隔, db存储模式下增加存储xsec_token字段
2024-12-05 21:10:31 +08:00
leantli
e830ada574
feat: xhs comments add xsec_token
2024-12-03 18:25:21 +08:00
Relakkes
ca9b47ef63
fix: xhs 帖子详情优化
2024-11-27 09:41:24 +08:00
Relakkes
43dffeb2d1
feat: xhs帖子详情获取优化
2024-11-26 13:37:53 +08:00
lyx0727
705b810269
fix: xhs验证码处理，跳转链接修复
2024-11-01 22:08:46 +08:00
lyx0727
1a37df4d5e
fix: xhs出现验证码时报错，增加用户手动验证
2024-11-01 18:52:16 +08:00
程序员阿江(Relakkes)
5a27ad089c
Merge pull request #462 from FloRainRJY/xiaohongshu_comment_number_restrict
...
All_platform_comments_restrict
2024-10-24 15:31:13 +08:00
liugangdao
78c09c4ae1
fix:翻页时search id不变
2024-10-24 13:57:27 +08:00
unknown
7e53c4acfc
All_platform_comments_restrict
2024-10-23 16:32:02 +08:00
unknown
19269c66fd
xiaohongshu_comment_number_restrict
2024-10-22 20:33:10 +08:00
Relakkes
03e393949a
fix: xhs帖子详情问题更新
2024-10-20 00:59:08 +08:00
Relakkes
9fe3e47b0f
chore: 增加代码学习声明，严格禁止非法、禁止商业、不当用途
2024-10-20 00:43:25 +08:00
Relakkes
6dd3420743
fix : #423
2024-09-02 22:57:28 +08:00
Relakkes
d6fb255bdf
fix: xhs note detail error
2024-09-02 21:45:12 +08:00
Relakkes
65699aa1cb
feat: xhs支持获取评论的点赞数量
2024-08-24 06:07:33 +08:00
Relakkes
ab7d8142af
feat: weibo支持指定创作者主页
2024-08-24 05:52:11 +08:00
Relakkes
c70bd9e071
feat: 增加搜索词来源渠道
2024-08-23 08:29:24 +08:00
Relakkes
f371675d47
fix: xhs指定笔记ID获取方式增加解析html方式，原来的由于xsec_token导致失效
2024-08-11 22:37:10 +08:00
Relakkes
7229d29123
feat: xhs update
2024-08-04 14:54:03 +08:00
