103 Commits

Author	SHA1	Message	Date
刘小龙	c87df59996	log client modify	2025-09-09 15:27:46 +08:00
程序员阿江(Relakkes)	2bce3593f7	feat: support time deplay for all platform	2025-09-02 16:43:09 +08:00
程序员阿江(Relakkes)	eb799e1fa7	refactor: xhs extractor	2025-09-02 14:50:32 +08:00
未来可欺	6a10d0d11c	原始的HTTPStatusError不能捕获像ConnectError、ReadError这些异常类型，本次提交修改了捕获异常的类型为httpx模块请求异常的基类：HTTPError，以便捕获在httpx.request方法中引发的任何异常（例如ip被封，服务器拒接连接），正确处理爬取媒体被中断时并不会导致爬取文本的中断逻辑	2025-08-06 11:24:51 +08:00
未来可欺	81f2dbe4ab	添加了对媒体资源服务器的异常处理，参见 issue #691	2025-08-05 13:11:00 +08:00
程序员阿江(Relakkes)	b9d30bbabb	fix: #693	2025-08-01 15:55:21 +08:00
未来可欺	0b81240aed	升级 httpx 版本至 0.28.1，并修改关键字参数 proxies 至 proxy	2025-07-31 22:48:02 +08:00
未来可欺	93a1c27fff	通过测试search模式，修复部分运行时的bug，并对能够爬取媒体的平台设置了较长的超时时间	2025-07-30 21:19:56 +08:00
未来可欺	173bc08a9d	添加了抖音存储视频以及图片的逻辑，并将config.py中ENABLE_GET_IMAGES参数更名为ENABLE_GET_MEIDAS，在此基础上略微修改存储逻辑	2025-07-30 18:24:08 +08:00
korruz	07a6e387ea	refactor: move format_proxy_info to utils and update crawler classes to use it	2025-07-29 14:16:24 +08:00
程序员阿江(Relakkes)	fc06c783f5	fix: fixed xhs req headers	2025-07-23 13:28:58 +08:00
程序员阿江(Relakkes)	a4d9aaa34a	refactor: xhs update	2025-07-21 21:26:16 +08:00
程序员阿江(Relakkes)	13b00f7a36	refactor: config update	2025-07-18 23:26:52 +08:00
gaoxiaobei	b913db64bb	refactor(config): move platform-specific configs to separate files ... - Remove platform-specific configurations from base_config.py - Create separate config files for each platform in their respective directories - Update import statements in core files to use new platform-specific config modules - Clean up unused and deprecated configuration options	2025-07-18 17:27:37 +08:00
程序员阿江(Relakkes)	e83b2422d9	feat: 支持playwright通过cdp协议连接本地chrome浏览器 ... docs: 增加uv来管理python依赖的文档	2025-06-25 23:22:39 +08:00
chimeElm	26a845581e	Update client.py ... 修复CRAWLER_MAX_NOTES_COUNT在爬取小红书作者帖子时失效的问题	2025-06-07 02:41:09 +08:00
crpa33	274d64aefc	处理xhs意外的评论信息为空的情况 ... 报错就会打断我，我没辙	2025-04-02 11:59:27 +08:00
liudongkai	33e7ef016d	feat: xhs 非代理模式下增加随机等待间隔, db存储模式下增加存储xsec_token字段	2024-12-05 21:10:31 +08:00
leantli	e830ada574	feat: xhs comments add xsec_token	2024-12-03 18:25:21 +08:00
Relakkes	ca9b47ef63	fix: xhs 帖子详情优化	2024-11-27 09:41:24 +08:00
Relakkes	43dffeb2d1	feat: xhs帖子详情获取优化	2024-11-26 13:37:53 +08:00
lyx0727	705b810269	fix: xhs验证码处理，跳转链接修复	2024-11-01 22:08:46 +08:00
lyx0727	1a37df4d5e	fix: xhs出现验证码时报错，增加用户手动验证	2024-11-01 18:52:16 +08:00
程序员阿江(Relakkes)	5a27ad089c	Merge pull request #462 from FloRainRJY/xiaohongshu_comment_number_restrict ... All_platform_comments_restrict	2024-10-24 15:31:13 +08:00
liugangdao	78c09c4ae1	fix:翻页时search id不变	2024-10-24 13:57:27 +08:00
unknown	7e53c4acfc	All_platform_comments_restrict	2024-10-23 16:32:02 +08:00
unknown	19269c66fd	xiaohongshu_comment_number_restrict	2024-10-22 20:33:10 +08:00
Relakkes	03e393949a	fix: xhs帖子详情问题更新	2024-10-20 00:59:08 +08:00
Relakkes	9fe3e47b0f	chore: 增加代码学习声明，严格禁止非法、禁止商业、不当用途	2024-10-20 00:43:25 +08:00
Relakkes	6dd3420743	fix: #423	2024-09-02 22:57:28 +08:00
Relakkes	d6fb255bdf	fix: xhs note detail error	2024-09-02 21:45:12 +08:00
Relakkes	65699aa1cb	feat: xhs支持获取评论的点赞数量	2024-08-24 06:07:33 +08:00
Relakkes	ab7d8142af	feat: weibo支持指定创作者主页	2024-08-24 05:52:11 +08:00
Relakkes	c70bd9e071	feat: 增加搜索词来源渠道	2024-08-23 08:29:24 +08:00
Relakkes	f371675d47	fix: xhs指定笔记ID获取方式增加解析html方式，原来的由于xsec_token导致失效	2024-08-11 22:37:10 +08:00
Relakkes	7229d29123	feat: xhs update	2024-08-04 14:54:03 +08:00
Wenbo Lu	9fb8f42f69	fetch_creator_notes_detail, id ->note_id	2024-07-26 21:55:31 +08:00
Relakkes	573ca9a659	feat: xhs笔记详情更新	2024-07-25 00:44:46 +08:00
AuYeung	1fd7827e36	When the query has no content, terminate the loop early	2024-07-18 20:44:40 +08:00
helloteemo	b95dc2c125	fix: 小红书下载新版本使用>3.10特性, 降低使用版本	2024-07-12 09:50:03 +08:00
helloteemo	6545a15ff3	feature: 支持小红书图片、视频下载	2024-07-11 22:56:30 +08:00
helloteemo	e71690a985	fix: 解决小红书图片水印问题	2024-07-11 17:39:48 +08:00
Relakkes	d3eeccbaac	feat: logger record current search page	2024-06-24 22:24:51 +08:00
nelzomal	eace7d1750	improve base config reading command line arg logic	2024-06-09 18:51:36 +08:00
Relakkes	4bba1447f8	feat: cache impl done	2024-06-02 19:57:13 +08:00
Relakkes	e64df93edd	feat: 由于xhs和dy现在检测playwright二维码登录了，大概率会出现滑块或者手机验证，增加登录态检测时间为5min，预留足够的时间手动过验证码。	2024-05-15 23:23:30 +08:00
Henry He	a2dca888ac	fix: 修复 f-string 双引号问题	2024-04-26 11:02:55 +08:00
bigfa	99d53cd945	fix:兼容网页端上传的图片原图获取	2024-04-24 14:16:35 +08:00
Relakkes	87eb8aa6a7	fix: #230	2024-04-13 20:18:04 +08:00
程序员阿江-Relakkes	a341dc2aff	Merge pull request #229 from Tianci-King/main ... feat(core): 新增控制爬虫参数起始页面的页数start_page;perf(argparse): 向命令行解析器添加程序参数…	2024-04-13 13:37:35 +08:00