mirror of
https://github.com/NanmiCoder/MediaCrawler.git
synced 2026-07-24 05:10:30 +08:00
fix: 将平台业务ID统一改为String并自动建表
- 抖音、B站、快手、微博的帖子/视频/评论ID从BigInteger改为String, 避免PostgreSQL下字符串写入BIGINT报错及未来ID溢出风险 - B站dynamic_id改为String,修复API返回id_str被强转int导致的精度丢失 - 知乎提取器对content_id/question_id显式str()转换 - main.py启动数据库保存模式时自动建表,无需手动--init_db - 同步更新相关老化测试
This commit is contained in:
@@ -128,8 +128,7 @@ class BiliDbStoreImplement(AbstractStore):
|
||||
Args:
|
||||
content_item: content item dict
|
||||
"""
|
||||
video_id = int(content_item.get("video_id"))
|
||||
content_item["video_id"] = video_id
|
||||
video_id = content_item.get("video_id")
|
||||
content_item["liked_count"] = int(content_item.get("liked_count", 0) or 0)
|
||||
content_item["create_time"] = int(content_item.get("create_time", 0) or 0)
|
||||
|
||||
@@ -154,9 +153,7 @@ class BiliDbStoreImplement(AbstractStore):
|
||||
Args:
|
||||
comment_item: comment item dict
|
||||
"""
|
||||
comment_id = int(comment_item.get("comment_id"))
|
||||
comment_item["comment_id"] = comment_id
|
||||
comment_item["video_id"] = int(comment_item.get("video_id", 0) or 0)
|
||||
comment_id = comment_item.get("comment_id")
|
||||
comment_item["create_time"] = int(comment_item.get("create_time", 0) or 0)
|
||||
comment_item["like_count"] = str(comment_item.get("like_count", "0"))
|
||||
comment_item["sub_comment_count"] = str(comment_item.get("sub_comment_count", "0"))
|
||||
@@ -191,8 +188,7 @@ class BiliDbStoreImplement(AbstractStore):
|
||||
Args:
|
||||
dynamic_item: dynamic item dict
|
||||
"""
|
||||
dynamic_id = int(dynamic_item.get("dynamic_id"))
|
||||
dynamic_item["dynamic_id"] = dynamic_id
|
||||
dynamic_id = dynamic_item.get("dynamic_id")
|
||||
|
||||
async with get_session() as session:
|
||||
result = await session.execute(select(BilibiliUpDynamic).where(BilibiliUpDynamic.dynamic_id == dynamic_id))
|
||||
|
||||
Reference in New Issue
Block a user