程序员阿江(Relakkes)
fb66ef016d
docs: add vitepress-plugin-mermaid for Mermaid diagram rendering
...
- 添加 vitepress-plugin-mermaid 和 mermaid 依赖
- 更新 VitePress 配置以支持 Mermaid 图表渲染
- 在 sidebar 中添加项目架构文档链接
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2025-12-18 13:25:21 +08:00
程序员阿江(Relakkes)
26c511e35f
docs: add project architecture documentation with Mermaid diagrams
...
添加项目架构文档，包含：
- 系统架构总览图
- 数据流向图
- 爬虫基类体系和生命周期图
- 存储层架构图
- 代理、登录、缓存系统图
- 模块依赖关系图
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2025-12-18 13:16:32 +08:00
程序员阿江(Relakkes)
08fcf68b98
docs: update README.md
2025-12-17 12:12:29 +08:00
程序员阿江(Relakkes)
2426095123
docs: update README.md
2025-12-17 11:04:26 +08:00
程序员阿江(Relakkes)
3c75d4f1d0
docs: update docs style
2025-12-16 14:49:14 +08:00
程序员阿江(Relakkes)
332a07ce62
docs: update docs
2025-12-16 14:41:28 +08:00
程序员阿江(Relakkes)
8a0fd49b96
refactor: 抽离应用 runner 并优化退出清理
...
- 新增 tools/app_runner.py 统一信号/取消/清理超时逻辑
- main.py 精简为业务入口与资源清理实现
- CDPBrowserManager 不再覆盖已有 SIGINT/SIGTERM 处理器
2025-12-15 18:06:57 +08:00
程序员阿江(Relakkes)
9ade3b3eef
chore: use playwright sign xhs and update dependency
2025-12-09 14:47:48 +08:00
程序员阿江-Relakkes
2600c48359
fix: xhs sub comment sign error
...
fix: params参数以及路径问题
2025-12-03 11:02:52 +08:00
MEI
ff9a1624f1
fix: params参数以及路径问题
2025-12-03 10:31:32 +08:00
程序员阿江-Relakkes
630d4c1614
Merge pull request #789 from NanmiCoder/feature/test_new_rule_01
...
docs: update data store
2025-11-28 22:35:45 +08:00
程序员阿江(Relakkes)
f14242c239
docs: update data store
2025-11-28 22:21:20 +08:00
程序员阿江(Relakkes)
29832ded91
chore: add coderowner rules
2025-11-28 22:17:40 +08:00
程序员阿江(Relakkes)
11f2802624
docs: update README.md
2025-11-28 18:16:04 +08:00
程序员阿江-Relakkes
ab19494883
Merge pull request #785 from hsparks-codes/feat/update_readme
...
docs: Move data storage section to separate guide
2025-11-28 18:07:56 +08:00
hsparks.codes
2bc9297812
docs: Move data storage section to separate guide
...
- Create comprehensive data storage guide (docs/data_storage_guide.md)
- Update README.md with link to storage guide instead of full details
- Update README_en.md with link to storage guide
- Bilingual guide (Chinese and English) in single document
- Includes all storage options: CSV, JSON, Excel, SQLite, MySQL
- Detailed usage examples and documentation links
This change improves README readability by moving detailed storage
information to a dedicated document while keeping main README concise.
2025-11-28 10:18:09 +01:00
程序员阿江-Relakkes
ba64c8ff9c
Merge pull request #784 from NanmiCoder/feature/excel-export-and-tests
...
feat: excel store with other platform
2025-11-28 15:15:31 +08:00
程序员阿江-Relakkes
ebbf86d67b
Merge pull request #783 from hsparks-codes/feature/excel-export-and-tests
...
feat: Add Excel export functionality and unit tests
2025-11-28 15:14:25 +08:00
程序员阿江(Relakkes)
6e858c1a00
feat: excel store with other platform
2025-11-28 15:12:36 +08:00
hsparks.codes
324f09cf9f
fix: Update tests to handle openpyxl color format and ContextVar
...
- Fix header color assertion to check only RGB values (not alpha channel)
- Remove ContextVar mock as it cannot be patched in Python 3.11+
- All 17 tests now passing successfully
2025-11-28 05:04:00 +01:00
hsparks.codes
46ef86ddef
feat: Add Excel export functionality and unit tests
...
Features:
- Excel export with formatted multi-sheet workbooks (Contents, Comments, Creators)
- Professional styling: blue headers, auto-width columns, borders, text wrapping
- Smart export: empty sheets automatically removed
- Support for all platforms (xhs, dy, ks, bili, wb, tieba, zhihu)
Testing:
- Added pytest framework with asyncio support
- Unit tests for Excel store functionality
- Unit tests for store factory pattern
- Shared fixtures for test data
- Test coverage for edge cases
Documentation:
- Comprehensive Excel export guide (docs/excel_export_guide.md)
- Updated README.md and README_en.md with Excel examples
- Updated config comments to include excel option
Dependencies:
- Added openpyxl>=3.1.2 for Excel support
- Added pytest>=7.4.0 and pytest-asyncio>=0.21.0 for testing
This contribution adds immediate value for users who need data analysis
capabilities and establishes a testing foundation for future development.
2025-11-28 04:44:12 +01:00
程序员阿江-Relakkes
31a092c653
Merge pull request #782 from NanmiCoder/fix/xhs-sign-20251127
...
feat: xhs sign playwright version
2025-11-27 11:05:24 +08:00
程序员阿江(Relakkes)
f989ce0788
feat: xhs sign playwright version
2025-11-27 10:53:08 +08:00
程序员阿江-Relakkes
15b98fa511
ip proxy expired logic switch
...
Fix/proxy 20251125
2025-11-26 16:05:01 +08:00
程序员阿江(Relakkes)
f1e7124654
fix: proxy extract error
2025-11-26 16:01:54 +08:00
程序员阿江(Relakkes)
6eef02d08c
feat: ip proxy expired check
2025-11-25 12:39:10 +08:00
程序员阿江(Relakkes)
1da347cbf8
docs: update index.md
2025-11-22 09:12:25 +08:00
程序员阿江(Relakkes)
422cc92dd1
docs: update README
2025-11-22 08:20:09 +08:00
程序员阿江(Relakkes)
13d2302c9c
docs: update README
2025-11-18 17:56:55 +08:00
程序员阿江(Relakkes)
ff8c92daad
chore: add copyright to every file
2025-11-18 12:24:02 +08:00
程序员阿江(Relakkes)
5288bddb42
refactor: weibo search #771
2025-11-17 17:24:47 +08:00
程序员阿江(Relakkes)
6dcfd7e0a5
refactor: weibo login
2025-11-17 17:11:35 +08:00
程序员阿江(Relakkes)
e89a6d5781
feat: cdp browser cleanup after crawler done
2025-11-17 12:21:53 +08:00
程序员阿江(Relakkes)
a1c5e07df8
fix: xhs sub comment bugfix #769
2025-11-17 11:47:33 +08:00
程序员阿江(Relakkes)
b6caa7a85e
refactor: add xhs creator params
2025-11-10 21:10:03 +08:00
程序员阿江(Relakkes)
1e3637f238
refactor: update xhs note detail
2025-11-10 18:13:51 +08:00
程序员阿江(Relakkes)
b5dab6d1e8
refactor: 使用 xhshow 替代 playwright 签名方案
...
感谢 @Cloxl/xhshow 开源项目
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-11-10 18:12:45 +08:00
程序员阿江-Relakkes
54f23b8d1c
Merge pull request #768 from yangtao210/main
...
优化mongodb配置获取逻辑，移动存储基类位置。集成测试
2025-11-07 05:44:07 -05:00
yangtao210
58eb89f073
Merge branch 'NanmiCoder:main' into main
2025-11-07 17:44:09 +08:00
yt210
7888f4c6bd
优化mongodb配置获取逻辑，移动存储基类位置。集成测试
2025-11-07 17:42:50 +08:00
yt210
b61ec54a72
优化mongodb配置获取逻辑，移动存储基类位置。
2025-11-07 17:42:28 +08:00
程序员阿江(Relakkes)
60cbb3e37d
fix: weibo container error #568
2025-11-06 19:43:09 +08:00
程序员阿江-Relakkes
05a1782746
Merge pull request #764 from yangtao210/main
...
新增存储到mongoDB
2025-11-06 06:10:49 -05:00
yt210
ef6948b305
新增存储到mongoDB
2025-11-06 10:40:30 +08:00
程序员阿江(Relakkes)
45ec4b433a
docs: update
2025-11-06 00:08:03 +08:00
程序员阿江(Relakkes)
0074e975dd
fix: dy search
2025-11-04 00:14:16 +08:00
程序员阿江(Relakkes)
889fa01466
fix: bili词云图修复
2025-11-02 13:25:31 +08:00
程序员阿江(Relakkes)
3f5925e326
feat: update xhs sign
2025-10-27 19:06:07 +08:00
程序员阿江(Relakkes)
ed6e0bfb5f
refactor: tieba 改为浏览器获取数据
2025-10-19 17:09:55 +08:00
程序员阿江(Relakkes)
26a261bc09
Merge branch 'feature/config-refactor-20251018'
2025-10-19 15:32:42 +08:00
