程序员阿江(Relakkes)
157ddfb21b
i18n: translate all Chinese comments, docstrings, and logger messages to English
...
Comprehensive translation of Chinese text to English across the entire codebase:
- api/: FastAPI server documentation and logger messages
- cache/: Cache abstraction layer comments and docstrings
- database/: Database models and MongoDB store documentation
- media_platform/: All platform crawlers (Bilibili, Douyin, Kuaishou, Tieba, Weibo, Xiaohongshu, Zhihu)
- model/: Data model documentation
- proxy/: Proxy pool and provider documentation
- store/: Data storage layer comments
- tools/: Utility functions and browser automation
- test/: Test file documentation
Preserved: Chinese disclaimer header (lines 10-18) for legal compliance
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2025-12-26 23:27:19 +08:00
程序员阿江(Relakkes)
55d8c7783f
feat: webo full context support
2025-12-26 19:22:24 +08:00
程序员阿江(Relakkes)
ff1b681311
fix: weibo get note image fixed
2025-12-26 00:47:20 +08:00
程序员阿江(Relakkes)
6eef02d08c
feat: ip proxy expired check
2025-11-25 12:39:10 +08:00
程序员阿江(Relakkes)
ff8c92daad
chore: add copyright to every file
2025-11-18 12:24:02 +08:00
程序员阿江(Relakkes)
5288bddb42
refactor: weibo search #771
2025-11-17 17:24:47 +08:00
程序员阿江(Relakkes)
6dcfd7e0a5
refactor: weibo login
2025-11-17 17:11:35 +08:00
程序员阿江(Relakkes)
60cbb3e37d
fix: weibo container error #568
2025-11-06 19:43:09 +08:00
yt210
ef6948b305
新增存储到mongoDB
2025-11-06 10:40:30 +08:00
程序员阿江(Relakkes)
03e384bbe2
refactor: cdp模式下移除stealth注入
2025-10-19 15:32:03 +08:00
程序员阿江(Relakkes)
2bce3593f7
feat: support time deplay for all platform
2025-09-02 16:43:09 +08:00
未来可欺
81f2dbe4ab
添加了对媒体资源服务器的异常处理，参见 issue #691
2025-08-05 13:11:00 +08:00
未来可欺
0b81240aed
升级 httpx 版本至 0.28.1，并修改关键字参数 proxies 至 proxy
2025-07-31 22:48:02 +08:00
未来可欺
173bc08a9d
添加了抖音存储视频以及图片的逻辑，并将config.py中ENABLE_GET_IMAGES参数更名为ENABLE_GET_MEIDAS，在此基础上略微修改存储逻辑
2025-07-30 18:24:08 +08:00
korruz
07a6e387ea
refactor: move format_proxy_info to utils and update crawler classes to use it
2025-07-29 14:16:24 +08:00
程序员阿江(Relakkes)
13b00f7a36
refactor: config update
2025-07-18 23:26:52 +08:00
gaoxiaobei
b913db64bb
refactor(config): move platform-specific configs to separate files
...
- Remove platform-specific configurations from base_config.py
- Create separate config files for each platform in their respective directories
- Update import statements in core files to use new platform-specific config modules
- Clean up unused and deprecated configuration options
2025-07-18 17:27:37 +08:00
Lei Cao
355ed183dd
增加选择微博搜索类型的配置
2025-07-05 22:14:31 +00:00
程序员阿江(Relakkes)
848df2b491
feat: other platfrom support the cdp mode
2025-07-03 17:13:32 +08:00
Trojx
f9eedc59b1
fix：微博根据creator爬取note时，爬取评论失败。原因是解析的参数key有误
2024-11-29 10:47:40 +08:00
unknown
7e53c4acfc
All_platform_comments_restrict
2024-10-23 16:32:02 +08:00
Relakkes
9fe3e47b0f
chore: 增加代码学习声明，严格禁止非法、禁止商业、不当用途
2024-10-20 00:43:25 +08:00
Relakkes
ab7d8142af
feat: weibo支持指定创作者主页
2024-08-24 05:52:11 +08:00
Relakkes
c70bd9e071
feat: 增加搜索词来源渠道
2024-08-23 08:29:24 +08:00
Relakkes
1c2237a66f
fix: 微博登录问题修复
...
feat: 微博二级评论
2024-08-05 00:48:42 +08:00
Relakkes
d3eeccbaac
feat: logger record current search page
2024-06-24 22:24:51 +08:00
nelzomal
eace7d1750
improve base config reading command line arg logic
2024-06-09 18:51:36 +08:00
Relakkes
487afc8e0c
refactor: 修改导报顺心
2024-04-17 23:13:40 +08:00
Tianci-King
1115b0d90c
feat(core): 新增控制爬虫 参数起始页面的页数start_page;perf(argparse): 向命令行解析器添加程序参数起始页面页数和关键字
2024-04-12 00:52:47 +08:00
Er_Meng
9cd6efb916
使用isort对引用进行格式化排序 修改微博获取图片默认配置关闭
2024-04-10 09:54:28 +08:00
Er_Meng
16413c3074
新增对微博博客内照片获取的支持 文件存放路径data/weibo/images
2024-04-09 17:21:52 +08:00
Styunlen
40daa8d6f3
chore: fix wrong log output when weibo crawler finished
...
Scripts output "Bilibili crawler finished" when Weibo crawler finished.
2024-04-06 00:41:05 +08:00
leantli
68a60faa7f
chore: 简化判断方式
2024-04-04 00:11:22 +08:00
leantli
133f978477
fix: 修复爬取视频/帖子最大数设置值较低导致不爬取的问题
2024-04-03 12:18:23 +08:00
Relakkes
59cd9f67a0
feat: 支持评论模式是否开启爬取选项
2024-03-16 11:52:42 +08:00
Relakkes
894dabcf63
refactor: 数据存储重构，分离不同类型的存储实现
2024-01-14 22:06:31 +08:00
Relakkes
e31aebbdfb
fix: 修复代理Bug
2024-01-13 15:50:02 +08:00
Relakkes
4de14ad6a8
fix: 修复微博PC端登录后COOKIE在手机端无法使用的bug
2024-01-06 19:18:07 +08:00
Relakkes
38d6f10bf0
feat: 微博二维码登录done
2023-12-30 18:54:21 +08:00
Relakkes
eee81622ac
feat: 微博支持评论 & 指定帖子
2023-12-25 00:02:11 +08:00
Relakkes
c5b64fdbf5
feat: 微博爬虫帖子搜索完成
2023-12-24 17:57:48 +08:00
