70 Commits

Author SHA1 Message Date
suyiiyii 94daf74359 web api 初步完工 2024-09-20 15:39:22 +08:00
suyiiyii c85e77c801 web api 更新 2024-09-20 00:25:40 +08:00
suyiiyii d890d32bba web api 初稿 2024-09-19 21:16:56 +08:00
suyiiyii 76f271584f web api 初稿 2024-09-19 20:28:16 +08:00
suyiiyii 6cbc6f7d4d 🔀 merge 2024-09-19 10:00:34 +08:00
github-actions[bot] 908de2c5ca 🔖 Release 0.9.4 2024-09-17 15:10:51 +00:00
felinae98 76be8454f3 🔖 release 0.9.4 2024-09-17 23:09:39 +08:00
github-actions[bot] 7845ef8c74 📝 Update changelog 2024-09-17 14:59:38 +00:00
Azide 088e7a439f 新增可以在 fsm 抛出错误后重置 fsm 的装饰器工具 2024-09-17 22:59:09 +08:00
suyiiyii c784417ecc 初步实现删除cookie的单元测试 2024-09-13 20:01:54 +08:00
suyiiyii dd802a9c17 初步实现添加cookie的单元测试 2024-09-13 19:12:19 +08:00
suyiiyii 2cfd58373f pytest db_config 2024-09-13 14:12:02 +08:00
suyiiyii 4791fb69e0 ♻️ 重构 get_cookie 方法 2024-09-13 11:34:45 +08:00
suyiiyii 4b8d6a9379 🐛 fix 2024-09-13 10:11:54 +08:00
suyiiyii b25fcd9ac2 尝试添加一种可以跳过当前请求的方式 2024-09-13 01:07:34 +08:00
suyiiyii 16331b50d5 根据数据库的修改对应的逻辑,同时移除多余的init_cookie和_check_cookie 2024-09-13 01:00:53 +08:00
suyiiyii af246df222 俺又来改数据库哩 2024-09-13 00:35:14 +08:00
suyiiyii 4f73f8a08c 🐛 弃用_cookie_client_manger_,改用issubclass判断是否为CookieClientManager 2024-09-13 00:26:28 +08:00
suyiiyii 5111baa89c 🐛 应用部分推荐的重命名 2024-09-13 00:11:49 +08:00
suyiiyii f865cef427 🐛 调整日志等级 2024-09-12 23:33:11 +08:00
renovate[bot] ab5236ee37 ⬆️ Update all non-major dependencies (#621)
* ⬆️ Update all non-major dependencies

* 💄 auto fix by pre-commit hooks

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-09-12 13:28:17 +08:00
suyiiyii 318ba8fb3c 🔀 merge 2024-09-09 18:41:01 +08:00
suyiiyii 2c2c9a091c ♻️ del_cookie 2024-09-09 18:36:18 +08:00
suyiiyii 4a5e00c094 :recycles: ddel_cookie_target 2024-09-09 18:32:53 +08:00
suyiiyii 404b1e445c :recycles: add_cookie_target 2024-09-09 18:23:23 +08:00
suyiiyii d43d042618 :recycles: add_cookie 2024-09-09 11:35:39 +08:00
suyiiyii 65a5976897 :recycles: CookieClientManager 2024-09-09 11:06:59 +08:00
suyiiyii f959e3ee08 :recycles: 仿照 platform_manager 添加 site_manager 2024-09-09 11:01:34 +08:00
suyiiyii 4db7e7b911 :recycles: DBConfig中 替换platform_name为site_name 2024-09-08 18:38:38 +08:00
suyiiyii 275bc0cb53 :recycles: 注释掉cookie相关代码,使得bison可以正常运行 2024-09-08 18:21:57 +08:00
suyiiyii ce1f1bbedb 又来改数据库了( 2024-09-08 18:17:06 +08:00
suyiiyii 7c9e191f40 删除cookie 对话 2024-09-08 15:56:44 +08:00
suyiiyii 940301a6fc 取消关联cookie 对话 2024-09-08 15:17:19 +08:00
suyiiyii 61dcf879ce ♻️ 整理代码 2024-09-08 13:03:47 +08:00
suyiiyii a6227828e3 ♻️ 整理代码 2024-09-08 12:59:44 +08:00
suyiiyii bbc5492193 为匿名cookie设置标签 2024-09-08 12:22:06 +08:00
suyiiyii eddd3e42a1 修改_choose_cookie的逻辑以支持no target的Platform 2024-09-06 11:33:39 +08:00
suyiiyii 06079b98f7 关联cookie是不显示匿名cookie 2024-09-06 10:16:24 +08:00
suyiiyii 418a941448 添加不合法cookie的提示 2024-09-06 10:08:08 +08:00
suyiiyii afd1bee762 将get_cookie_friendly_name和valid_cookie移动到ccm内部 2024-09-06 01:08:32 +08:00
suyiiyii b61bde6e3f 关联cookie时,只显示支持的订阅 2024-09-06 00:39:57 +08:00
suyiiyii 6537f01a34 集中判断是否为CookieClientManager 2024-09-06 00:34:52 +08:00
suyiiyii 0ce2893911 匿名cookie和用户cookie一起调度 2024-09-06 00:25:33 +08:00
suyiiyii cf3966e69b 添加cookie只显示支持的Platform 2024-09-06 00:14:35 +08:00
suyiiyii 370fc250f0 数据库Cookie表添加is_universal属性 2024-09-05 19:32:47 +08:00
suyiiyii 3bd0867f0e 数据库Cookie表添加cd属性 2024-09-05 16:07:59 +08:00
suyiiyii 01435eeded 提出assemble_client方法 2024-09-04 01:19:53 +08:00
pre-commit-ci[bot] ded3e34259 ⬆️ auto update by pre-commit hooks (#620)
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.6.0 → v0.6.3](https://github.com/astral-sh/ruff-pre-commit/compare/v0.6.0...v0.6.3)
- [github.com/pre-commit/mirrors-eslint: v9.8.0 → v9.9.1](https://github.com/pre-commit/mirrors-eslint/compare/v9.8.0...v9.9.1)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-09-03 13:35:36 +08:00
renovate[bot] 68f7e3b72a ⬆️ Update all non-major dependencies (#595)
* ⬆️ Update all non-major dependencies

* 💄 auto fix by pre-commit hooks

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-09-03 13:34:49 +08:00
suyiiyii 498e7d60d4 根据条件选择Cookie 和 状态回写数据库 2024-09-03 13:15:18 +08:00
suyiiyii 055ed6e02a 使用闭包实现client hook 2024-09-03 10:52:20 +08:00
suyiiyii 4ce6b85f79 weibo 带 cookie mvp 2024-09-03 10:00:53 +08:00
suyiiyii 7901b845ea 初步实现携带cookie请求 2024-09-02 23:13:29 +08:00
github-actions[bot] ccbed746da 📝 Update changelog 2024-09-01 14:34:29 +00:00
dependabot[bot] c523b3a811 ⬆️ Bump webpack from 5.88.2 to 5.94.0 in /admin-frontend (#619)
Bumps [webpack](https://github.com/webpack/webpack) from 5.88.2 to 5.94.0.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](https://github.com/webpack/webpack/compare/v5.88.2...v5.94.0)

---
updated-dependencies:
- dependency-name: webpack
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-01 22:33:03 +08:00
suyiiyii d6205867b8 无权限用户尝试添加订阅时返回提示信息 (#617)
*  无权限用户尝试添加订阅时返回提示信息

*  添加no_permission_matcher相关的单元测试

* 🐛 优化无权限提示的排版

* 🐛 移除没有必要的命令
2024-09-01 22:32:47 +08:00
github-actions[bot] 83cd0a741e 📝 Update changelog 2024-09-01 14:30:29 +00:00
Azide b8b49a5ce5 🐛 B站请求策略阶段行为优化 (#610)
* 🐛 调整ruff的pytest警告

* 🐛 调整导入关系警告

* 🐛 删除奇怪无用的赋值和取值逻辑

*  不同测试部分所用变量应加以区分

* 🐛 subs_io model添加默认值

* 🐛 修完所有的 ruff PT001 警告

* 🔧 按ruff建议修改ruff配置

warning: The top-level linter settings are deprecated in favour of their counterparts in the `lint` section. Please update the following options in `pyproject.toml`:
  - 'ignore' -> 'lint.ignore'
  - 'select' -> 'lint.select'

* 🔊 降低B站请求策略非Raise阶段API352的日志等级

* 🐛 Raise阶段应该 raise err

* 🔊 日志添加平台名

* 🐛 bzhanhan调度继续降低至60s

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-09-01 22:30:02 +08:00
github-actions[bot] afcc536334 📝 Update changelog 2024-09-01 14:24:42 +00:00
suyiiyii cf38500be7 🐛 Rss 不再删除格式化字符 2024-09-01 22:23:02 +08:00
suyiiyii 1cd778c2e0 ♻️ cookie 组件不再与 user 关联 2024-08-31 23:07:43 +08:00
suyiiyii c828fd94e4 ♻️ cookie 组件不再与 user 关联 2024-08-31 23:04:49 +08:00
github-actions[bot] 7d80b44d2a 📝 Update changelog 2024-08-30 02:26:50 +00:00
uy_sun 60e6f05cf4 fix tests 2024-08-30 10:26:22 +08:00
felinae98 8a21ca2a1c 🐛 forbid adding platform that needs browser in no-browser env 2024-08-30 10:26:22 +08:00
suyiiyii ffae6f2ec5 添加cookie的时候显示关联的cookie 2024-08-26 18:09:26 +08:00
suyiiyii 6f20dbf358 支持对话关联cookie到订阅目标 2024-08-26 17:32:36 +08:00
suyiiyii b655eff755 支持对话添加cookie 2024-08-26 10:37:03 +08:00
suyiiyii c264ad374b 🐛 stash 2024-08-22 21:50:25 +08:00
suyiiyii 7913f7485a 添加cookie相关的数据库表 2024-08-22 20:55:39 +08:00
47 changed files with 10691 additions and 7106 deletions
+2 -2
View File
@@ -7,7 +7,7 @@ ci:
autoupdate_commit_msg: ":arrow_up: auto update by pre-commit hooks"
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.6.0
rev: v0.6.3
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
@@ -34,7 +34,7 @@ repos:
stages: [commit]
- repo: https://github.com/pre-commit/mirrors-eslint
rev: v9.8.0
rev: v9.9.1
hooks:
- id: eslint
additional_dependencies:
+6 -1
View File
@@ -1,9 +1,14 @@
# Change Log
## 最近更新
## v0.9.4
### Bug 修复
- FSM 内部执行外部函数出现异常时不应崩溃 [@AzideCupric](https://github.com/AzideCupric) ([#616](https://github.com/MountainDash/nonebot-bison/pull/616))
- 无权限用户尝试添加订阅时返回提示信息 [@suyiiyii](https://github.com/suyiiyii) ([#617](https://github.com/MountainDash/nonebot-bison/pull/617))
- B站请求策略阶段行为优化 [@AzideCupric](https://github.com/AzideCupric) ([#610](https://github.com/MountainDash/nonebot-bison/pull/610))
- Rss 不再删除格式化字符 [@suyiiyii](https://github.com/suyiiyii) ([#615](https://github.com/MountainDash/nonebot-bison/pull/615))
- forbid adding platform that needs browser in no-browser env [@felinae98](https://github.com/felinae98) ([#609](https://github.com/MountainDash/nonebot-bison/pull/609))
- 修正项目的代码警告 [@AzideCupric](https://github.com/AzideCupric) ([#614](https://github.com/MountainDash/nonebot-bison/pull/614))
- 修复 anonymous_site() 无法正确工作的问题 [@felinae98](https://github.com/felinae98) ([#606](https://github.com/MountainDash/nonebot-bison/pull/606))
+16 -16
View File
@@ -5,22 +5,22 @@
"homepage": "bison",
"proxy": "http://127.0.0.1:8080",
"dependencies": {
"@arco-design/web-react": "^2.63.1",
"@babel/core": "^7.24.7",
"@arco-design/web-react": "^2.64.0",
"@babel/core": "^7.25.2",
"@babel/plugin-syntax-flow": "^7.24.7",
"@babel/plugin-transform-react-jsx": "^7.24.7",
"@babel/plugin-transform-react-jsx": "^7.25.2",
"@reduxjs/toolkit": "^1.9.7",
"@testing-library/dom": "^10.4.0",
"@testing-library/react": "^16.0.0",
"@testing-library/react": "^16.0.1",
"@testing-library/user-event": "^14.5.2",
"@types/jest": "^29.5.12",
"@types/node": "^20.14.10",
"@types/react": "^18.3.3",
"@types/node": "^20.16.5",
"@types/react": "^18.3.5",
"@types/react-dom": "^18.3.0",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"react-redux": "^9.1.2",
"react-router-dom": "^6.24.1",
"react-router-dom": "^6.26.1",
"react-scripts": "5.0.1",
"redux": "^5.0.1",
"redux-persist": "^6.0.0",
@@ -53,17 +53,17 @@
]
},
"devDependencies": {
"@testing-library/jest-dom": "^6.4.6",
"@typescript-eslint/eslint-plugin": "^8.0.0",
"@typescript-eslint/parser": "^8.0.0",
"eslint": "^9.6.0",
"@testing-library/jest-dom": "^6.5.0",
"@typescript-eslint/eslint-plugin": "^8.4.0",
"@typescript-eslint/parser": "^8.4.0",
"eslint": "^9.10.0",
"eslint-config-airbnb": "^19.0.4",
"eslint-config-airbnb-typescript": "^18.0.0",
"eslint-import-resolver-typescript": "^3.6.1",
"eslint-plugin-import": "^2.29.1",
"eslint-plugin-jsx-a11y": "^6.9.0",
"eslint-plugin-react": "^7.34.3",
"eslint-import-resolver-typescript": "^3.6.3",
"eslint-plugin-import": "^2.30.0",
"eslint-plugin-jsx-a11y": "^6.10.0",
"eslint-plugin-react": "^7.35.2",
"eslint-plugin-react-hooks": "^4.6.2",
"eslint-plugin-react-redux": "^4.1.0"
"eslint-plugin-react-redux": "^4.2.0"
}
}
+8001 -6050
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -1,4 +1,4 @@
FROM node:20.15.1 as frontend
FROM node:20.17.0 as frontend
ADD . /app
WORKDIR /app/admin-frontend
RUN yarn && yarn build
+1 -1
View File
@@ -1,4 +1,4 @@
# syntax=docker/dockerfile:1.8
# syntax=docker/dockerfile:1.9
FROM python:3.11-slim-bullseye as base
FROM base as builder
+1 -1
View File
@@ -1,4 +1,4 @@
# syntax=docker/dockerfile:1.8
# syntax=docker/dockerfile:1.9
FROM python:3.11-slim-bullseye as base
FROM base as builder
+75 -4
View File
@@ -1,3 +1,5 @@
from typing import cast
import nonebot
from fastapi import status
from fastapi.routing import APIRouter
@@ -12,14 +14,18 @@ from ..apis import check_sub_target
from .jwt import load_jwt, pack_jwt
from ..types import Target as T_Target
from ..utils.get_bot import get_groups
from ..platform import platform_manager
from .token_manager import token_manager
from ..config.db_config import SubscribeDupException
from ..platform import site_manager, platform_manager
from ..utils.site import CookieClientManager, is_cookie_client_manager
from ..config import NoSuchUserException, NoSuchTargetException, NoSuchSubscribeException, config
from .types import (
Cookie,
TokenResp,
GlobalConf,
SiteConfig,
StatusResp,
CookieTarget,
SubscribeResp,
PlatformConfig,
AddSubscribeReq,
@@ -54,16 +60,20 @@ async def check_is_superuser(token_obj: dict = Depends(get_jwt_obj)):
@router.get("/global_conf")
async def get_global_conf() -> GlobalConf:
res = {}
platform_res = {}
for platform_name, platform in platform_manager.items():
res[platform_name] = PlatformConfig(
platform_res[platform_name] = PlatformConfig(
platformName=platform_name,
categories=platform.categories,
enabledTag=platform.enable_tag,
site_name=platform.site.name,
name=platform.name,
hasTarget=getattr(platform, "has_target"),
)
return GlobalConf(platformConf=res)
site_res = {}
for site_name, site in site_manager.items():
site_res[site_name] = SiteConfig(name=site_name, enable_cookie=is_cookie_client_manager(site.client_mgr))
return GlobalConf(platformConf=platform_res, siteConf=site_res)
async def get_admin_groups(qq: int):
@@ -197,3 +207,64 @@ async def update_weigth_config(platformName: str, target: str, weight_config: We
except NoSuchTargetException:
raise HTTPException(status.HTTP_400_BAD_REQUEST, "no such subscribe")
return StatusResp(ok=True, msg="")
@router.get("/cookie", dependencies=[Depends(check_is_superuser)])
async def get_cookie(site_name: str = None, target: str = None) -> list[Cookie]:
cookies_in_db = await config.get_cookie(site_name, is_anonymous=False)
client_mgr = cast(CookieClientManager, site_manager[site_name].client_mgr)
friendly_names = [await client_mgr.get_cookie_friendly_name(x) for x in cookies_in_db]
return [
Cookie(
id=cookies_in_db[i].id,
friendly_name=friendly_names[i],
site_name=cookies_in_db[i].site_name,
last_usage=cookies_in_db[i].last_usage,
status=cookies_in_db[i].status,
cd_milliseconds=cookies_in_db[i].cd_milliseconds,
is_universal=cookies_in_db[i].is_universal,
is_anonymous=cookies_in_db[i].is_anonymous,
tags=cookies_in_db[i].tags,
)
for i in range(len(cookies_in_db))
]
@router.post("/cookie", dependencies=[Depends(check_is_superuser)])
async def add_cookie(site_name: str, content: str) -> StatusResp:
client_mgr = cast(CookieClientManager, site_manager[site_name].client_mgr)
await client_mgr.add_user_cookie(content)
return StatusResp(ok=True, msg="")
@router.delete("/cookie/{cookie_id}", dependencies=[Depends(check_is_superuser)])
async def delete_cookie_by_id(cookie_id: int) -> StatusResp:
await config.delete_cookie_by_id(cookie_id)
return StatusResp(ok=True, msg="")
@router.get("/cookie_target", dependencies=[Depends(check_is_superuser)])
async def get_cookie_target(
site_name: str | None = None, target: str | None = None, cookie_id: int | None = None
) -> list[CookieTarget]:
cookie_targets = await config.get_cookie_target()
# TODO: filter in SQL
return [
x
for x in cookie_targets
if (site_name is None or x.cookie.site_name == site_name)
and (target is None or x.target.target == target)
and (cookie_id is None or x.cookie.id == cookie_id)
]
@router.post("/cookie_target", dependencies=[Depends(check_is_superuser)])
async def add_cookie_target(platform_name: str, target: str, cookie_id: int) -> StatusResp:
await config.add_cookie_target(target, platform_name, cookie_id)
return StatusResp(ok=True, msg="")
@router.delete("/cookie_target", dependencies=[Depends(check_is_superuser)])
async def del_cookie_target(platform_name: str, target: str, cookie_id: int) -> StatusResp:
await config.delete_cookie_target(target, platform_name, cookie_id)
return StatusResp(ok=True, msg="")
+37
View File
@@ -6,14 +6,22 @@ class PlatformConfig(BaseModel):
categories: dict[int, str]
enabledTag: bool
platformName: str
site_name: str
hasTarget: bool
class SiteConfig(BaseModel):
name: str
enable_cookie: bool
AllPlatformConf = dict[str, PlatformConfig]
AllSiteConf = dict[str, SiteConfig]
class GlobalConf(BaseModel):
platformConf: AllPlatformConf
siteConf: AllSiteConf
class TokenResp(BaseModel):
@@ -50,3 +58,32 @@ class AddSubscribeReq(BaseModel):
class StatusResp(BaseModel):
ok: bool
msg: str
from typing import Any
from datetime import datetime
from pydantic import BaseModel
class Target(BaseModel):
platform_name: str
target_name: str
target: str
class Cookie(BaseModel):
id: int
site_name: str
friendly_name: str
last_usage: datetime
status: str
cd_milliseconds: int
is_universal: bool
is_anonymous: bool
tags: dict[str, Any]
class CookieTarget(BaseModel):
target: Target
cookie_id: int
+105 -2
View File
@@ -12,8 +12,8 @@ from nonebot_plugin_datastore import create_session
from ..types import Tag
from ..types import Target as T_Target
from .utils import NoSuchTargetException
from .db_model import User, Target, Subscribe, ScheduleTimeWeight
from .utils import NoSuchTargetException, DuplicateCookieTargetException
from .db_model import User, Cookie, Target, Subscribe, CookieTarget, ScheduleTimeWeight
from ..types import Category, UserSubInfo, WeightConfig, TimeWeightConfig, PlatformWeightConfigResp
@@ -259,5 +259,108 @@ class DBConfig:
)
return res
async def get_cookie(
self,
site_name: str | None = None,
target: T_Target | None = None,
is_universal: bool | None = None,
is_anonymous: bool | None = None,
) -> Sequence[Cookie]:
"""获取满足传入条件的所有 cookie"""
async with create_session() as sess:
query = select(Cookie).distinct()
if is_universal is not None:
query = query.where(Cookie.is_universal == is_universal)
if is_anonymous is not None:
query = query.where(Cookie.is_anonymous == is_anonymous)
if site_name:
query = query.where(Cookie.site_name == site_name)
query = query.outerjoin(CookieTarget).options(selectinload(Cookie.targets))
res = (await sess.scalars(query)).all()
if target:
# 如果指定了 target,过滤掉不满足要求的cookie
query = select(CookieTarget.cookie_id).join(Target).where(Target.target == target)
ids = set((await sess.scalars(query)).all())
# 如果指定了 target 且未指定 is_universal,则添加返回 universal cookie
res = [cookie for cookie in res if cookie.id in ids or cookie.is_universal]
return res
async def add_cookie(self, cookie: Cookie) -> int:
async with create_session() as sess:
sess.add(cookie)
await sess.commit()
await sess.refresh(cookie)
return cookie.id
async def update_cookie(self, cookie: Cookie):
async with create_session() as sess:
cookie_in_db: Cookie | None = await sess.scalar(select(Cookie).where(Cookie.id == cookie.id))
if not cookie_in_db:
raise ValueError(f"cookie {cookie.id} not found")
cookie_in_db.content = cookie.content
cookie_in_db.last_usage = cookie.last_usage
cookie_in_db.status = cookie.status
cookie_in_db.tags = cookie.tags
await sess.commit()
async def delete_cookie_by_id(self, cookie_id: int):
async with create_session() as sess:
cookie = await sess.scalar(
select(Cookie)
.where(Cookie.id == cookie_id)
.outerjoin(CookieTarget)
.options(selectinload(Cookie.targets))
)
if len(cookie.targets) > 0:
raise Exception(f"cookie {cookie.id} in use")
await sess.execute(delete(Cookie).where(Cookie.id == cookie_id))
await sess.commit()
async def add_cookie_target(self, target: T_Target, platform_name: str, cookie_id: int):
"""通过 cookie_id 可以唯一确定一个 Cookie,通过 target 和 platform_name 可以唯一确定一个 Target"""
async with create_session() as sess:
target_obj = await sess.scalar(
select(Target).where(Target.platform_name == platform_name, Target.target == target)
)
# check if relation exists
cookie_target = await sess.scalar(
select(CookieTarget).where(CookieTarget.target == target_obj, CookieTarget.cookie_id == cookie_id)
)
if cookie_target:
raise DuplicateCookieTargetException()
cookie_obj = await sess.scalar(select(Cookie).where(Cookie.id == cookie_id))
cookie_target = CookieTarget(target=target_obj, cookie=cookie_obj)
sess.add(cookie_target)
await sess.commit()
async def delete_cookie_target(self, target: T_Target, platform_name: str, cookie_id: int):
async with create_session() as sess:
target_obj = await sess.scalar(
select(Target).where(Target.platform_name == platform_name, Target.target == target)
)
cookie_obj = await sess.scalar(select(Cookie).where(Cookie.id == cookie_id))
await sess.execute(
delete(CookieTarget).where(CookieTarget.target == target_obj, CookieTarget.cookie == cookie_obj)
)
await sess.commit()
async def delete_cookie_target_by_id(self, cookie_target_id: int):
async with create_session() as sess:
await sess.execute(delete(CookieTarget).where(CookieTarget.id == cookie_target_id))
await sess.commit()
async def get_cookie_target(self) -> list[CookieTarget]:
async with create_session() as sess:
query = (
select(CookieTarget)
.outerjoin(Target)
.options(selectinload(CookieTarget.target))
.outerjoin(Cookie)
.options(selectinload(CookieTarget.cookie))
)
res = list((await sess.scalars(query)).all())
res.sort(key=lambda x: (x.target.platform_name, x.cookie_id, x.target_id))
return res
config = DBConfig()
+40 -1
View File
@@ -1,4 +1,5 @@
import datetime
from typing import Any
from pathlib import Path
from nonebot_plugin_saa import PlatformTarget
@@ -6,7 +7,7 @@ from sqlalchemy.dialects.postgresql import JSONB
from nonebot.compat import PYDANTIC_V2, ConfigDict
from nonebot_plugin_datastore import get_plugin_data
from sqlalchemy.orm import Mapped, relationship, mapped_column
from sqlalchemy import JSON, String, ForeignKey, UniqueConstraint
from sqlalchemy import JSON, String, DateTime, ForeignKey, UniqueConstraint
from ..types import Tag, Category
@@ -36,6 +37,7 @@ class Target(Model):
subscribes: Mapped[list["Subscribe"]] = relationship(back_populates="target")
time_weight: Mapped[list["ScheduleTimeWeight"]] = relationship(back_populates="target")
cookies: Mapped[list["CookieTarget"]] = relationship(back_populates="target")
class ScheduleTimeWeight(Model):
@@ -66,3 +68,40 @@ class Subscribe(Model):
target: Mapped[Target] = relationship(back_populates="subscribes")
user: Mapped[User] = relationship(back_populates="subscribes")
class Cookie(Model):
id: Mapped[int] = mapped_column(primary_key=True)
site_name: Mapped[str] = mapped_column(String(100))
content: Mapped[str] = mapped_column(String(1024))
# 最后使用的时刻
last_usage: Mapped[datetime.datetime] = mapped_column(DateTime, default=datetime.datetime(1970, 1, 1))
# Cookie 当前的状态
status: Mapped[str] = mapped_column(String(20), default="")
# 使用一次之后,需要的冷却时间
cd_milliseconds: Mapped[int] = mapped_column(default=0)
# 是否是通用 Cookie(对所有Target都有效)
is_universal: Mapped[bool] = mapped_column(default=False)
# 是否是匿名 Cookie
is_anonymous: Mapped[bool] = mapped_column(default=False)
# 标签,扩展用
tags: Mapped[dict[str, Any]] = mapped_column(JSON().with_variant(JSONB, "postgresql"), default={})
targets: Mapped[list["CookieTarget"]] = relationship(back_populates="cookie")
@property
def cd(self) -> datetime.timedelta:
return datetime.timedelta(milliseconds=self.cd_milliseconds)
@cd.setter
def cd(self, value: datetime.timedelta):
self.cd_milliseconds = int(value.total_seconds() * 1000)
class CookieTarget(Model):
id: Mapped[int] = mapped_column(primary_key=True)
target_id: Mapped[int] = mapped_column(ForeignKey("nonebot_bison_target.id", ondelete="CASCADE"))
cookie_id: Mapped[int] = mapped_column(ForeignKey("nonebot_bison_cookie.id", ondelete="CASCADE"))
target: Mapped[Target] = relationship(back_populates="cookies")
cookie: Mapped[Cookie] = relationship(back_populates="targets")
@@ -0,0 +1,62 @@
"""empty message
Revision ID: ef796b74b0fe
Revises: f9baef347cc8
Create Date: 2024-09-13 00:34:08.601438
"""
import sqlalchemy as sa
from alembic import op
from sqlalchemy import Text
from sqlalchemy.dialects import postgresql
# revision identifiers, used by Alembic.
revision = "ef796b74b0fe"
down_revision = "f9baef347cc8"
branch_labels = None
depends_on = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
op.create_table(
"nonebot_bison_cookie",
sa.Column("id", sa.Integer(), nullable=False),
sa.Column("site_name", sa.String(length=100), nullable=False),
sa.Column("content", sa.String(length=1024), nullable=False),
sa.Column("last_usage", sa.DateTime(), nullable=False),
sa.Column("status", sa.String(length=20), nullable=False),
sa.Column("cd_milliseconds", sa.Integer(), nullable=False),
sa.Column("is_universal", sa.Boolean(), nullable=False),
sa.Column("is_anonymous", sa.Boolean(), nullable=False),
sa.Column("tags", sa.JSON().with_variant(postgresql.JSONB(astext_type=Text()), "postgresql"), nullable=False),
sa.PrimaryKeyConstraint("id", name=op.f("pk_nonebot_bison_cookie")),
)
op.create_table(
"nonebot_bison_cookietarget",
sa.Column("id", sa.Integer(), nullable=False),
sa.Column("target_id", sa.Integer(), nullable=False),
sa.Column("cookie_id", sa.Integer(), nullable=False),
sa.ForeignKeyConstraint(
["cookie_id"],
["nonebot_bison_cookie.id"],
name=op.f("fk_nonebot_bison_cookietarget_cookie_id_nonebot_bison_cookie"),
ondelete="CASCADE",
),
sa.ForeignKeyConstraint(
["target_id"],
["nonebot_bison_target.id"],
name=op.f("fk_nonebot_bison_cookietarget_target_id_nonebot_bison_target"),
ondelete="CASCADE",
),
sa.PrimaryKeyConstraint("id", name=op.f("pk_nonebot_bison_cookietarget")),
)
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
op.drop_table("nonebot_bison_cookietarget")
op.drop_table("nonebot_bison_cookie")
# ### end Alembic commands ###
+4
View File
@@ -8,3 +8,7 @@ class NoSuchSubscribeException(Exception):
class NoSuchTargetException(Exception):
pass
class DuplicateCookieTargetException(Exception):
pass
+21
View File
@@ -3,6 +3,8 @@ from pkgutil import iter_modules
from collections import defaultdict
from importlib import import_module
from ..utils import Site
from ..plugin_config import plugin_config
from .platform import Platform, make_no_target_group
_package_dir = str(Path(__file__).resolve().parent)
@@ -22,3 +24,22 @@ for name, platform_list in _platform_list.items():
platform_manager[name] = platform_list[0]
else:
platform_manager[name] = make_no_target_group(platform_list)
def _get_unavailable_platforms() -> dict[str, str]:
res = {}
for name, platform in platform_manager.items():
if platform.site.require_browser and not plugin_config.bison_use_browser:
res[name] = "需要启用 bison_use_browser"
return res
# platform => reason for not available
unavailable_paltforms: dict[str, str] = _get_unavailable_platforms()
site_manager: dict[str, type[Site]] = {}
for site in Site.registry:
if not hasattr(site, "name"):
continue
site_manager[site.name] = site
+62 -1
View File
@@ -2,10 +2,24 @@ import sys
import asyncio
import inspect
from enum import Enum
from functools import wraps
from dataclasses import dataclass
from collections.abc import Set as AbstractSet
from collections.abc import Callable, Sequence, Awaitable, AsyncGenerator
from typing import TYPE_CHECKING, Any, Generic, TypeVar, Protocol, TypeAlias, TypedDict, NamedTuple, runtime_checkable
from typing import (
TYPE_CHECKING,
Any,
Generic,
TypeVar,
Protocol,
ParamSpec,
TypeAlias,
TypedDict,
NamedTuple,
Concatenate,
overload,
runtime_checkable,
)
from nonebot import logger
@@ -17,6 +31,7 @@ TAddon = TypeVar("TAddon", contravariant=True)
TState = TypeVar("TState", contravariant=True)
TEvent = TypeVar("TEvent", contravariant=True)
TFSM = TypeVar("TFSM", bound="FSM", contravariant=True)
P = ParamSpec("P")
class StateError(Exception): ...
@@ -163,6 +178,52 @@ class FSM(Generic[TState, TEvent, TAddon]):
self.started = False
del self.machine
self.current_state = self.graph["initial"]
self.machine = self._core()
logger.trace("FSM closed")
@overload
def reset_on_exception(
func: Callable[Concatenate[TFSM, P], Awaitable[ActionReturn]],
) -> Callable[Concatenate[TFSM, P], Awaitable[ActionReturn]]:
"""自动在发生异常后重置 FSM"""
@overload
def reset_on_exception(
auto_start: bool = False,
) -> Callable[
[Callable[Concatenate[TFSM, P], Awaitable[ActionReturn]]], Callable[Concatenate[TFSM, P], Awaitable[ActionReturn]]
]:
"""自动在异常后重置 FSM,当 auto_start 为 True 时,自动启动 FSM"""
# 参考自 dataclasses.dataclass 的实现
def reset_on_exception(func=None, /, *, auto_start=False): # pyright: ignore[reportInconsistentOverload]
def warp(func: Callable[Concatenate[TFSM, P], Awaitable[ActionReturn]]):
return __reset_clear_up(func, auto_start)
# 判断调用的是 @reset_on_exception 还是 @reset_on_exception(...)
if func is None:
# 调用的是带括号的
return warp
# 调用的是不带括号的
return warp(func)
def __reset_clear_up(func: Callable[Concatenate[TFSM, P], Awaitable[ActionReturn]], auto_start: bool):
@wraps(func)
async def wrapper(fsm_self: TFSM, *args: P.args, **kwargs: P.kwargs) -> ActionReturn:
try:
return await func(fsm_self, *args, **kwargs)
except Exception as e:
logger.error(f"Exception in {func.__name__}: {e}")
await fsm_self.reset()
if auto_start and not fsm_self.started:
await fsm_self.start()
raise e
return wrapper
+13 -4
View File
@@ -14,7 +14,7 @@ from httpx import URL as HttpxURL
from nonebot_bison.types import Target
from .models import DynRawPost
from .fsm import FSM, Condition, StateGraph, Transition, ActionReturn
from .fsm import FSM, Condition, StateGraph, Transition, ActionReturn, reset_on_exception
if TYPE_CHECKING:
from .platforms import Bilibili
@@ -218,6 +218,11 @@ class RetryFSM(FSM[RetryState, RetryEvent, RetryAddon[TBilibili]]):
self.addon.reset_all()
await super().reset()
@override
@reset_on_exception
async def emit(self, event: RetryEvent):
await super().emit(event)
# FIXME: 拿出来是方便测试了,但全局单例会导致所有被装饰的函数共享状态,有待改进
_retry_fsm = RetryFSM(RETRY_GRAPH, RetryAddon["Bilibili"]())
@@ -236,15 +241,19 @@ def retry_for_352(api_func: Callable[[TBilibili, Target], Awaitable[list[DynRawP
case RetryState.NROMAL | RetryState.REFRESH | RetryState.RAISE:
try:
res = await api_func(bls, *args, **kwargs)
except ApiCode352Error:
logger.error("API 352 错误")
except ApiCode352Error as e:
logger.warning("本次 Bilibili API 请求返回 352 错误")
await _retry_fsm.emit(RetryEvent.REQUEST_AND_RAISE)
if _retry_fsm.current_state == RetryState.RAISE:
raise e
return []
else:
await _retry_fsm.emit(RetryEvent.REQUEST_AND_SUCCESS)
return res
case RetryState.BACKOFF:
logger.warning("回避中,不请求")
logger.warning("本次 Bilibili 请求回避中,不请求")
await _retry_fsm.emit(RetryEvent.IN_BACKOFF_TIME)
return []
case _:
+1 -1
View File
@@ -68,7 +68,7 @@ class BilibiliClientManager(ClientManager):
class BilibiliSite(Site):
name = "bilibili.com"
schedule_setting = {"seconds": 50}
schedule_setting = {"seconds": 60}
schedule_type = "interval"
client_mgr = BilibiliClientManager
require_browser = True
+1 -16
View File
@@ -16,7 +16,7 @@ from nonebot_plugin_saa import PlatformTarget
from ..post import Post
from ..utils import Site, ProcessContext
from ..plugin_config import plugin_config
from ..types import Tag, Target, RawPost, SubUnit, Category
from ..types import Tag, Target, RawPost, SubUnit, Category, RegistryMeta
class CategoryNotSupport(Exception):
@@ -29,21 +29,6 @@ class CategoryNotRecognize(Exception):
"""raise in get_category, when you don't know the category of post"""
class RegistryMeta(type):
def __new__(cls, name, bases, namespace, **kwargs):
return super().__new__(cls, name, bases, namespace)
def __init__(cls, name, bases, namespace, **kwargs):
if kwargs.get("base"):
# this is the base class
cls.registry = []
elif not kwargs.get("abstract"):
# this is the subclass
cls.registry.append(cls)
super().__init__(name, bases, namespace, **kwargs)
P = ParamSpec("P")
R = TypeVar("R")
+6 -4
View File
@@ -9,13 +9,15 @@ from bs4 import BeautifulSoup as bs
from ..post import Post
from .platform import NewMessage
from ..types import Target, RawPost
from ..utils import Site, text_fletten, text_similarity
from ..utils import Site, text_similarity
from ..utils.site import create_cookie_client_manager
class RssSite(Site):
name = "rss"
schedule_type = "interval"
schedule_setting = {"seconds": 30}
client_mgr = create_cookie_client_manager("rss")
class RssPost(Post):
@@ -32,7 +34,7 @@ class RssPost(Post):
for p in soup.find_all("p"):
p.insert_after("\n")
return text_fletten(soup.get_text())
return soup.get_text()
class Rss(NewMessage):
@@ -63,7 +65,7 @@ class Rss(NewMessage):
return post.id
async def get_sub_list(self, target: Target) -> list[RawPost]:
client = await self.ctx.get_client()
client = await self.ctx.get_client(target)
res = await client.get(target, timeout=10.0)
feed = feedparser.parse(res)
entries = feed.entries
@@ -82,7 +84,7 @@ class Rss(NewMessage):
async def parse(self, raw_post: RawPost) -> Post:
title = raw_post.get("title", "")
soup = bs(raw_post.description, "html.parser")
desc = soup.text.strip()
desc = raw_post.description
title, desc = self._text_process(title, desc)
pics = [x.attrs["src"] for x in soup("img")]
if raw_post.get("media_content"):
+6 -2
View File
@@ -13,6 +13,7 @@ from bs4 import BeautifulSoup as bs
from ..post import Post
from .platform import NewMessage
from ..utils import Site, http_client
from ..utils.site import create_cookie_client_manager
from ..types import Tag, Target, RawPost, ApiError, Category
_HEADER = {
@@ -39,6 +40,7 @@ class WeiboSite(Site):
name = "weibo.com"
schedule_type = "interval"
schedule_setting = {"seconds": 3}
client_mgr = create_cookie_client_manager(name)
class Weibo(NewMessage):
@@ -78,9 +80,11 @@ class Weibo(NewMessage):
raise cls.ParseTargetException(prompt="正确格式:\n1. 用户数字UID\n2. https://weibo.com/u/xxxx")
async def get_sub_list(self, target: Target) -> list[RawPost]:
client = await self.ctx.get_client()
client = await self.ctx.get_client(target)
header = {"Referer": f"https://m.weibo.cn/u/{target}", "MWeibo-Pwa": "1", "X-Requested-With": "XMLHttpRequest"}
# 获取 cookie 见 https://docs.rsshub.app/zh/deploy/config#%E5%BE%AE%E5%8D%9A
params = {"containerid": "107603" + target}
res = await client.get("https://m.weibo.cn/api/container/getIndex?", params=params, timeout=4.0)
res = await client.get("https://m.weibo.cn/api/container/getIndex?", headers=header, params=params, timeout=4.0)
res_data = json.loads(res.text)
if not res_data["ok"] and res_data["msg"] != "这里还没有内容":
raise ApiError(res.request.url)
+6
View File
@@ -1,3 +1,5 @@
from typing import cast
from nonebot.log import logger
from ..utils import Site
@@ -7,6 +9,7 @@ from ..config.db_model import Target
from ..types import Target as T_Target
from ..platform import platform_manager
from ..plugin_config import plugin_config
from ..utils.site import CookieClientManager, is_cookie_client_manager
scheduler_dict: dict[type[Site], Scheduler] = {}
@@ -30,6 +33,9 @@ async def init_scheduler():
else:
_schedule_class_platform_dict[site].append(platform_name)
for site, target_list in _schedule_class_dict.items():
if is_cookie_client_manager(site.client_mgr):
client_mgr = cast(CookieClientManager, site.client_mgr)
await client_mgr.refresh_anonymous_cookie()
if not plugin_config.bison_use_browser and site.require_browser:
logger.warning(f"{site.name} requires browser, it will not schedule.")
continue
+3
View File
@@ -12,6 +12,7 @@ from ..send import send_msgs
from ..types import Target, SubUnit
from ..platform import platform_manager
from ..utils import Site, ProcessContext
from ..utils.site import SkipRequestException
@dataclass
@@ -107,6 +108,8 @@ class Scheduler:
schedulable.platform_name, schedulable.target
)
to_send = await platform_obj.do_fetch_new_post(SubUnit(schedulable.target, send_userinfo_list))
except SkipRequestException as err:
logger.debug(f"skip request: {err}")
except Exception as err:
records = context.gen_req_records()
for record in records:
+59 -2
View File
@@ -14,6 +14,10 @@ from nonebot.adapters.onebot.v11.event import PrivateMessageEvent
from .add_sub import do_add_sub
from .del_sub import do_del_sub
from .query_sub import do_query_sub
from .add_cookie import do_add_cookie
from .del_cookie import do_del_cookie
from .add_cookie_target import do_add_cookie_target
from .del_cookie_target import do_del_cookie_target
from .utils import common_platform, admin_permission, gen_handle_cancel, configurable_to_me, set_target_user_info
add_sub_matcher = on_command(
@@ -26,12 +30,10 @@ add_sub_matcher = on_command(
add_sub_matcher.handle()(set_target_user_info)
do_add_sub(add_sub_matcher)
query_sub_matcher = on_command("查询订阅", rule=configurable_to_me, priority=5, block=True)
query_sub_matcher.handle()(set_target_user_info)
do_query_sub(query_sub_matcher)
del_sub_matcher = on_command(
"删除订阅",
rule=configurable_to_me,
@@ -42,6 +44,46 @@ del_sub_matcher = on_command(
del_sub_matcher.handle()(set_target_user_info)
do_del_sub(del_sub_matcher)
add_cookie_matcher = on_command(
"添加cookie",
aliases={"添加Cookie"},
rule=to_me(),
permission=SUPERUSER,
priority=5,
block=True,
)
do_add_cookie(add_cookie_matcher)
add_cookie_target_matcher = on_command(
"关联cookie",
aliases={"关联Cookie"},
rule=to_me(),
permission=SUPERUSER,
priority=5,
block=True,
)
do_add_cookie_target(add_cookie_target_matcher)
del_cookie_target_matcher = on_command(
"取消关联cookie",
aliases={"取消关联Cookie"},
rule=to_me(),
permission=SUPERUSER,
priority=5,
block=True,
)
do_del_cookie_target(del_cookie_target_matcher)
del_cookie_matcher = on_command(
"删除cookie",
aliases={"删除Cookie"},
rule=to_me(),
permission=SUPERUSER,
priority=5,
block=True,
)
do_del_cookie(del_cookie_matcher)
group_manage_matcher = on_command("群管理", rule=to_me(), permission=SUPERUSER, priority=4, block=True)
group_handle_cancel = gen_handle_cancel(group_manage_matcher, "已取消")
@@ -108,10 +150,25 @@ async def do_dispatch_command(
asyncio.create_task(new_matcher_ins.run(bot, event, state))
no_permission_matcher = on_command(
"添加订阅", rule=configurable_to_me, aliases={"删除订阅", "群管理"}, priority=8, block=True
)
@no_permission_matcher.handle()
async def send_no_permission():
await no_permission_matcher.finish("您没有权限进行此操作,请联系 Bot 管理员")
__all__ = [
"common_platform",
"add_sub_matcher",
"query_sub_matcher",
"del_sub_matcher",
"group_manage_matcher",
"no_permission_matcher",
"add_cookie_matcher",
"add_cookie_target_matcher",
"del_cookie_target_matcher",
"del_cookie_matcher",
]
+69
View File
@@ -0,0 +1,69 @@
from typing import cast
from nonebot.typing import T_State
from nonebot.matcher import Matcher
from nonebot.params import Arg, ArgPlainText
from nonebot.adapters import Message, MessageTemplate
from ..platform import platform_manager
from .utils import common_platform, gen_handle_cancel
from ..utils.site import CookieClientManager, is_cookie_client_manager
def do_add_cookie(add_cookie: type[Matcher]):
handle_cancel = gen_handle_cancel(add_cookie, "已中止添加cookie")
@add_cookie.handle()
async def init_promote(state: T_State):
state["_prompt"] = (
"请输入想要添加 Cookie 的平台,目前支持,请输入冒号左边的名称:\n"
+ "".join(
[
f"{platform_name}: {platform_manager[platform_name].name}\n"
for platform_name in common_platform
if is_cookie_client_manager(platform_manager[platform_name].site.client_mgr)
]
)
+ "要查看全部平台请输入:“全部”\n中止添加cookie过程请输入:“取消”"
)
@add_cookie.got("platform", MessageTemplate("{_prompt}"), [handle_cancel])
async def parse_platform(state: T_State, platform: str = ArgPlainText()) -> None:
if platform == "全部":
message = "全部平台\n" + "\n".join(
[
f"{platform_name}: {platform.name}"
for platform_name, platform in platform_manager.items()
if is_cookie_client_manager(platform_manager[platform_name].site.client_mgr)
]
)
await add_cookie.reject(message)
elif platform == "取消":
await add_cookie.finish("已中止添加cookie")
elif platform in platform_manager:
state["platform"] = platform
state["site"] = platform_manager[platform].site
else:
await add_cookie.reject("平台输入错误")
@add_cookie.handle()
async def prepare_get_id(state: T_State):
state["_prompt"] = "请输入 Cookie"
@add_cookie.got("cookie", MessageTemplate("{_prompt}"), [handle_cancel])
async def got_cookie(state: T_State, cookie: Message = Arg()):
client_mgr: type[CookieClientManager] = cast(
type[CookieClientManager], platform_manager[state["platform"]].site.client_mgr
)
cookie_text = cookie.extract_plain_text()
if not await client_mgr.validate_cookie(cookie_text):
await add_cookie.reject(state["site"].cookie_format_prompt)
state["cookie"] = cookie_text
@add_cookie.handle()
async def add_cookie_process(state: T_State):
client_mgr = cast(CookieClientManager, platform_manager[state["platform"]].site.client_mgr)
await client_mgr.add_user_cookie(state["cookie"])
await add_cookie.finish(
f"已添加 Cookie: {state['cookie']} 到平台 {state['platform']}" + "\n请使用“关联cookie”为 Cookie 关联订阅"
)
@@ -0,0 +1,75 @@
from typing import cast
from nonebot.typing import T_State
from nonebot.matcher import Matcher
from nonebot.params import ArgPlainText
from nonebot_plugin_saa import MessageFactory
from nonebot.internal.adapter import MessageTemplate
from ..config import config
from ..utils import parse_text
from ..platform import platform_manager
from ..utils.site import CookieClientManager
from .utils import gen_handle_cancel, generate_sub_list_text
def do_add_cookie_target(add_cookie_target_matcher: type[Matcher]):
handle_cancel = gen_handle_cancel(add_cookie_target_matcher, "已中止关联 cookie")
@add_cookie_target_matcher.handle()
async def init_promote(state: T_State):
res = await generate_sub_list_text(
add_cookie_target_matcher, state, is_index=True, is_show_cookie=True, is_hide_no_cookie_platfrom=True
)
res += "请输入要关联 cookie 的订阅的序号\n输入'取消'中止"
await MessageFactory(await parse_text(res)).send()
@add_cookie_target_matcher.got("target_idx", parameterless=[handle_cancel])
async def got_target_idx(state: T_State, target_idx: str = ArgPlainText()):
try:
target_idx = int(target_idx)
state["target"] = state["sub_table"][target_idx]
state["site"] = platform_manager[state["target"]["platform_name"]].site
except Exception:
await add_cookie_target_matcher.reject("序号错误")
@add_cookie_target_matcher.handle()
async def init_promote_cookie(state: T_State):
# 获取 site 的所有用户 cookie,再排除掉已经关联的 cookie,剩下的就是可以关联的 cookie
cookies = await config.get_cookie(site_name=state["site"].name, is_anonymous=False)
associated_cookies = await config.get_cookie(
target=state["target"]["target"],
site_name=state["site"].name,
is_anonymous=False,
)
associated_cookie_ids = {cookie.id for cookie in associated_cookies}
cookies = [cookie for cookie in cookies if cookie.id not in associated_cookie_ids]
if not cookies:
await add_cookie_target_matcher.finish(
"当前平台暂无可关联的 Cookie,请使用“添加cookie”命令添加或检查已关联的 Cookie"
)
state["cookies"] = cookies
client_mgr = cast(CookieClientManager, state["site"].client_mgr)
state["_prompt"] = "请选择一个 Cookie,已关联的 Cookie 不会显示\n" + "\n".join(
[f"{idx}. {await client_mgr.get_cookie_friendly_name(cookie)}" for idx, cookie in enumerate(cookies, 1)]
)
@add_cookie_target_matcher.got("cookie_idx", MessageTemplate("{_prompt}"), [handle_cancel])
async def got_cookie_idx(state: T_State, cookie_idx: str = ArgPlainText()):
try:
cookie_idx = int(cookie_idx)
state["cookie"] = state["cookies"][cookie_idx - 1]
except Exception:
await add_cookie_target_matcher.reject("序号错误")
@add_cookie_target_matcher.handle()
async def add_cookie_target_process(state: T_State):
await config.add_cookie_target(state["target"]["target"], state["target"]["platform_name"], state["cookie"].id)
cookie = state["cookie"]
client_mgr = cast(CookieClientManager, state["site"].client_mgr)
await add_cookie_target_matcher.finish(
f"已关联 Cookie: {await client_mgr.get_cookie_friendly_name(cookie)} "
f"到订阅 {state['site'].name} {state['target']['target']}"
)
+3 -1
View File
@@ -9,9 +9,9 @@ from nonebot_plugin_saa import Text, PlatformTarget, SupportedAdapters
from ..types import Target
from ..config import config
from ..apis import check_sub_target
from ..platform import Platform, platform_manager
from ..config.db_config import SubscribeDupException
from .utils import common_platform, ensure_user_info, gen_handle_cancel
from ..platform import Platform, platform_manager, unavailable_paltforms
def do_add_sub(add_sub: type[Matcher]):
@@ -39,6 +39,8 @@ def do_add_sub(add_sub: type[Matcher]):
elif platform == "取消":
await add_sub.finish("已中止订阅")
elif platform in platform_manager:
if platform in unavailable_paltforms:
await add_sub.finish(f"无法订阅 {platform}{unavailable_paltforms[platform]}")
state["platform"] = platform
else:
await add_sub.reject("平台输入错误")
+48
View File
@@ -0,0 +1,48 @@
from nonebot.typing import T_State
from nonebot.matcher import Matcher
from nonebot.params import EventPlainText
from nonebot_plugin_saa import MessageFactory
from ..config import config
from ..utils import parse_text
from ..platform import site_manager
from .utils import gen_handle_cancel
def do_del_cookie(del_cookie: type[Matcher]):
handle_cancel = gen_handle_cancel(del_cookie, "删除中止")
@del_cookie.handle()
async def send_list(state: T_State):
cookies = await config.get_cookie(is_anonymous=False)
if not cookies:
await del_cookie.finish("暂无已添加的 Cookie\n请使用“添加cookie”命令添加")
res = "已添加的 Cookie 为:\n"
state["cookie_table"] = {}
for index, cookie in enumerate(cookies, 1):
state["cookie_table"][index] = cookie
client_mgr = site_manager[cookie.site_name].client_mgr
friendly_name = await client_mgr.get_cookie_friendly_name(cookie)
res += f"{index} {cookie.site_name} {friendly_name} {len(cookie.targets)}个关联\n"
if res[-1] != "\n":
res += "\n"
res += "请输入要删除的 Cookie 的序号\n输入'取消'中止"
await MessageFactory(await parse_text(res)).send()
@del_cookie.receive(parameterless=[handle_cancel])
async def do_del(
state: T_State,
index_str: str = EventPlainText(),
):
try:
index = int(index_str)
cookie = state["cookie_table"][index]
if cookie.targets:
await del_cookie.reject("只能删除未关联的 Cookie,请使用“取消关联cookie”命令取消关联")
await config.delete_cookie_by_id(cookie.id)
except KeyError:
await del_cookie.reject("序号错误")
except Exception:
await del_cookie.reject("删除错误")
else:
await del_cookie.finish("删除成功")
@@ -0,0 +1,51 @@
from typing import cast
from nonebot.typing import T_State
from nonebot.matcher import Matcher
from nonebot.params import EventPlainText
from nonebot_plugin_saa import MessageFactory
from ..config import config
from ..utils import parse_text
from .utils import gen_handle_cancel
from ..platform import platform_manager
from ..utils.site import CookieClientManager
def do_del_cookie_target(del_cookie_target: type[Matcher]):
handle_cancel = gen_handle_cancel(del_cookie_target, "取消关联中止")
@del_cookie_target.handle()
async def send_list(state: T_State):
cookie_targets = await config.get_cookie_target()
if not cookie_targets:
await del_cookie_target.finish("暂无已关联 Cookie\n请使用“添加cookie”命令添加关联")
res = "已关联的 Cookie 为:\n"
state["cookie_target_table"] = {}
for index, cookie_target in enumerate(cookie_targets, 1):
client_mgr = cast(CookieClientManager, platform_manager[cookie_target.target.platform_name].site.client_mgr)
friendly_name = await client_mgr.get_cookie_friendly_name(cookie_target.cookie)
state["cookie_target_table"][index] = {
"platform_name": cookie_target.target.platform_name,
"target": cookie_target.target,
"friendly_name": friendly_name,
"cookie_target": cookie_target,
}
res += f"{index} {cookie_target.target.platform_name} {cookie_target.target.target_name} {friendly_name}\n"
if res[-1] != "\n":
res += "\n"
res += "请输入要删除的关联的序号\n输入'取消'中止"
await MessageFactory(await parse_text(res)).send()
@del_cookie_target.receive(parameterless=[handle_cancel])
async def do_del(
state: T_State,
index_str: str = EventPlainText(),
):
try:
index = int(index_str)
await config.delete_cookie_target_by_id(state["cookie_target_table"][index]["cookie_target"].id)
except Exception:
await del_cookie_target.reject("删除错误")
else:
await del_cookie_target.finish("删除成功")
+64 -3
View File
@@ -1,16 +1,21 @@
import contextlib
from typing import Annotated
from itertools import groupby
from operator import attrgetter
from typing import Annotated, cast
from nonebot.rule import Rule
from nonebot.adapters import Event
from nonebot.typing import T_State
from nonebot.matcher import Matcher
from nonebot.permission import SUPERUSER
from nonebot_plugin_saa import extract_target
from nonebot.params import Depends, EventToMe, EventPlainText
from nonebot_plugin_saa import PlatformTarget, extract_target
from ..platform import platform_manager
from ..config import config
from ..types import Category
from ..plugin_config import plugin_config
from ..platform import site_manager, platform_manager
from ..utils.site import CookieClientManager, is_cookie_client_manager
def _configurable_to_me(to_me: bool = EventToMe()):
@@ -60,3 +65,59 @@ def admin_permission():
permission = permission | GROUP_ADMIN | GROUP_OWNER
return permission
async def generate_sub_list_text(
matcher: type[Matcher],
state: T_State,
user_info: PlatformTarget = None,
is_index=False,
is_show_cookie=False,
is_hide_no_cookie_platfrom=False,
):
"""根据配置参数,生产订阅列表文本,同时将订阅信息存入state["sub_table"]"""
if user_info:
sub_list = await config.list_subscribe(user_info)
else:
sub_list = await config.list_subs_with_all_info()
sub_list = [
next(group)
for key, group in groupby(sorted(sub_list, key=attrgetter("target_id")), key=attrgetter("target_id"))
]
if is_hide_no_cookie_platfrom:
sub_list = [
sub
for sub in sub_list
if is_cookie_client_manager(platform_manager.get(sub.target.platform_name).site.client_mgr)
]
if not sub_list:
await matcher.finish("暂无已订阅账号\n请使用“添加订阅”命令添加订阅")
res = "订阅的帐号为:\n"
state["sub_table"] = {}
for index, sub in enumerate(sub_list, 1):
state["sub_table"][index] = {
"platform_name": sub.target.platform_name,
"target": sub.target.target,
}
res += f"{index} " if is_index else ""
res += f"{sub.target.platform_name} {sub.target.target_name} {sub.target.target}\n"
if platform := platform_manager.get(sub.target.platform_name):
if platform.categories:
res += " [{}]".format(", ".join(platform.categories[Category(x)] for x in sub.categories)) + "\n"
if platform.enable_tag:
if sub.tags:
res += " {}".format(", ".join(sub.tags)) + "\n"
if is_show_cookie:
target_cookies = await config.get_cookie(
target=sub.target.target, site_name=platform.site.name, is_anonymous=False
)
if target_cookies:
res += " 关联的 Cookie\n"
for cookie in target_cookies:
client_mgr = cast(CookieClientManager, site_manager[platform.site.name].client_mgr)
res += f" \t{await client_mgr.get_cookie_friendly_name(cookie)}\n"
else:
res += f" (平台 {sub.target.platform_name} 已失效,请删除此订阅)"
return res
+15
View File
@@ -58,3 +58,18 @@ class ApiError(Exception):
class SubUnit(NamedTuple):
sub_target: Target
user_sub_infos: list[UserSubInfo]
class RegistryMeta(type):
def __new__(cls, name, bases, namespace, **kwargs):
return super().__new__(cls, name, bases, namespace)
def __init__(cls, name, bases, namespace, **kwargs):
if kwargs.get("base"):
# this is the base class
cls.registry = []
elif not kwargs.get("abstract"):
# this is the subclass
cls.registry.append(cls)
super().__init__(name, bases, namespace, **kwargs)
+9 -9
View File
@@ -1,25 +1,25 @@
import difflib
import re
import sys
import difflib
import nonebot
from nonebot.plugin import require
from bs4 import BeautifulSoup as bs
from nonebot.log import logger, default_format
from nonebot.plugin import require
from nonebot_plugin_saa import Text, Image, MessageSegmentFactory
from .site import Site as Site
from ..plugin_config import plugin_config
from .image import pic_merge as pic_merge
from .context import ProcessContext as ProcessContext
from .http import http_client as http_client
from .image import capture_html as capture_html
from .site import ClientManager as ClientManager
from .image import text_to_image as text_to_image
from .site import anonymous_site as anonymous_site
from .context import ProcessContext as ProcessContext
from .image import is_pics_mergable as is_pics_mergable
from .image import pic_merge as pic_merge
from .image import pic_url_to_image as pic_url_to_image
from .image import text_to_image as text_to_image
from .site import ClientManager as ClientManager
from .site import DefaultClientManager as DefaultClientManager
from .site import Site as Site
from .site import anonymous_site as anonymous_site
from ..plugin_config import plugin_config
class Singleton(type):
+2 -1
View File
@@ -22,8 +22,9 @@ class ProcessContext:
async def _log_to_ctx(r: Response):
self._log_response(r)
existing_hooks = client.event_hooks["response"]
hooks = {
"response": [_log_to_ctx],
"response": [*existing_hooks, _log_to_ctx],
}
client.event_hooks = hooks
+122 -2
View File
@@ -1,10 +1,17 @@
import json
from typing import Literal
from json import JSONDecodeError
from abc import ABC, abstractmethod
from datetime import datetime, timedelta
import httpx
from httpx import AsyncClient
from nonebot.log import logger
from ..types import Target
from ..config import config
from .http import http_client
from ..config.db_model import Cookie
from ..types import Target, RegistryMeta
class ClientManager(ABC):
@@ -35,12 +42,121 @@ class DefaultClientManager(ClientManager):
pass
class Site:
class CookieClientManager(ClientManager):
_site_name: str
_default_cd: int = timedelta(seconds=10)
@classmethod
async def refresh_anonymous_cookie(cls):
"""移除已有的匿名cookie,添加一个新的匿名cookie"""
anonymous_cookies = await config.get_cookie(cls._site_name, is_anonymous=True)
anonymous_cookie = Cookie(site_name=cls._site_name, content="{}", is_universal=True, is_anonymous=True)
for cookie in anonymous_cookies:
if not cookie.is_anonymous:
continue
await config.delete_cookie_by_id(cookie.id)
anonymous_cookie.id = cookie.id # 保持原有的id
anonymous_cookie.last_usage = datetime.now() # 使得第一次请求优先使用用户 cookie
await config.add_cookie(anonymous_cookie)
@classmethod
async def add_user_cookie(cls, content: str):
"""添加用户 cookie"""
cookie = Cookie(site_name=cls._site_name, content=content)
cookie.cd = cls._default_cd
await config.add_cookie(cookie)
@classmethod
async def validate_cookie(cls, content: str) -> bool:
"""验证 cookie 内容是否有效,添加 cookie 时用,可根据平台的具体情况进行重写"""
try:
data = json.loads(content)
if not isinstance(data, dict):
return False
except JSONDecodeError:
return False
return True
@classmethod
async def get_cookie_friendly_name(cls, cookie: Cookie) -> str:
"""获取 cookie 的友好名字,用于展示"""
from . import text_fletten
return text_fletten(f"{cookie.site_name} [{cookie.content[:10]}]")
def _generate_hook(self, cookie: Cookie) -> callable:
"""hook 函数生成器,用于回写请求状态到数据库"""
async def _response_hook(resp: httpx.Response):
if resp.status_code == 200:
logger.trace(f"请求成功: {cookie.id} {resp.request.url}")
cookie.status = "success"
else:
logger.warning(f"请求失败:{cookie.id} {resp.request.url}, 状态码: {resp.status_code}")
cookie.status = "failed"
cookie.last_usage = datetime.now()
await config.update_cookie(cookie)
return _response_hook
async def _choose_cookie(self, target: Target | None) -> Cookie:
"""选择 cookie 的具体算法"""
cookies = await config.get_cookie(self._site_name, target)
cookies = (cookie for cookie in cookies if cookie.last_usage + cookie.cd < datetime.now())
cookie = min(cookies, key=lambda x: x.last_usage)
return cookie
async def get_client(self, target: Target | None) -> AsyncClient:
"""获取 client,根据 target 选择 cookie"""
client = http_client()
cookie = await self._choose_cookie(target)
if cookie.is_universal:
logger.trace(f"平台 {self._site_name} 未获取到用户cookie, 使用匿名cookie")
else:
logger.trace(f"平台 {self._site_name} 获取到用户cookie: {cookie.id}")
return await self._assemble_client(client, cookie)
async def _assemble_client(self, client, cookie) -> AsyncClient:
"""组装 client,可以自定义 cookie 对象的 content 装配到 client 中的方式"""
cookies = httpx.Cookies()
if cookie:
cookies.update(json.loads(cookie.content))
client.cookies = cookies
client.event_hooks = {"response": [self._generate_hook(cookie)]}
return client
async def get_client_for_static(self) -> AsyncClient:
return http_client()
async def get_query_name_client(self) -> AsyncClient:
return http_client()
async def refresh_client(self):
pass
def is_cookie_client_manager(manger: type[ClientManager]) -> bool:
return issubclass(manger, CookieClientManager)
def create_cookie_client_manager(site_name: str) -> type[CookieClientManager]:
"""创建一个平台特化的 CookieClientManger"""
return type(
"CookieClientManager",
(CookieClientManager,),
{"_site_name": site_name},
)
class Site(metaclass=RegistryMeta, base=True):
schedule_type: Literal["date", "interval", "cron"]
schedule_setting: dict
name: str
client_mgr: type[ClientManager] = DefaultClientManager
require_browser: bool = False
registry: list[type["Site"]]
cookie_format_prompt = "无效的 Cookie,请检查后重新输入,详情见<待添加的文档>"
def __str__(self):
return f"[{self.name}]-{self.name}-{self.schedule_setting}"
@@ -56,3 +172,7 @@ def anonymous_site(schedule_type: Literal["date", "interval", "cron"], schedule_
"client_mgr": DefaultClientManager,
},
)
class SkipRequestException(Exception):
pass
+4 -4
View File
@@ -11,9 +11,9 @@
"docs:update-package": "pnpm dlx vp-update"
},
"devDependencies": {
"@vuepress/bundler-vite": "2.0.0-rc.14",
"vue": "^3.4.31",
"vuepress": "2.0.0-rc.14",
"vuepress-theme-hope": "2.0.0-rc.50"
"@vuepress/bundler-vite": "2.0.0-rc.15",
"vue": "^3.5.3",
"vuepress": "2.0.0-rc.15",
"vuepress-theme-hope": "2.0.0-rc.52"
}
}
+429 -434
View File
File diff suppressed because it is too large Load Diff
Generated
+532 -515
View File
File diff suppressed because it is too large Load Diff
+23 -23
View File
@@ -1,6 +1,6 @@
[tool.poetry]
name = "nonebot-bison"
version = "0.9.3"
version = "0.9.4"
description = "Subscribe message from social medias"
authors = ["felinae98 <felinae225@qq.com>"]
license = "MIT"
@@ -24,40 +24,40 @@ classifiers = [
python = ">=3.10,<4.0.0"
beautifulsoup4 = ">=4.12.3"
feedparser = "^6.0.11"
httpx = ">=0.27.0"
nonebot2 = { extras = ["fastapi"], version = "^2.3.2" }
httpx = ">=0.27.2"
nonebot2 = { extras = ["fastapi"], version = "^2.3.3" }
nonebot-adapter-onebot = "^2.4.4"
nonebot-plugin-htmlrender = ">=0.3.3"
nonebot-plugin-htmlrender = ">=0.3.5"
nonebot-plugin-datastore = ">=1.3.0,<2.0.0"
nonebot-plugin-apscheduler = ">=0.5.0"
nonebot-plugin-send-anything-anywhere = ">=0.6.1,<0.7.0"
pillow = ">=8.4.0,<11.0"
pyjwt = "^2.8.0"
python-socketio = "^5.11.3"
nonebot-plugin-send-anything-anywhere = ">=0.7.1,<0.7.2"
pillow = ">=10.4.0,<11.0"
pyjwt = "^2.9.0"
python-socketio = "^5.11.4"
tinydb = "^4.8.0"
qrcode = "^7.4.2"
pydantic = ">=1.10.17,<3.0.0,!=2.5.0,!=2.5.1"
lxml = ">=5.2.2"
yarl = ">=1.9.4"
hishel = "^0.0.20"
expiringdictx = "^1.0.1"
rapidfuzz = "^3.9.3"
pydantic = ">=2.9.0,<3.0.0,!=2.5.0,!=2.5.1"
lxml = ">=5.3.0"
yarl = ">=1.11.0"
hishel = "^0.0.30"
expiringdictx = "^1.1.0"
rapidfuzz = "^3.9.7"
[tool.poetry.group.dev.dependencies]
black = ">=23.12.1,<25.0"
black = ">=24.8.0,<25.0"
ipdb = "^0.13.13"
isort = "^5.13.2"
nonemoji = "^0.1.4"
nb-cli = "^1.4.1"
pre-commit = "^3.7.1"
ruff = "^0.6.0"
nb-cli = "^1.4.2"
pre-commit = "^3.8.0"
ruff = "^0.6.4"
[tool.poetry.group.test.dependencies]
flaky = "^3.8.1"
nonebug = "^0.3.7"
nonebug-saa = "^0.4.1"
pytest = ">=7.4.4,<9.0.0"
pytest-asyncio = ">=0.23.7,<0.24.0"
pytest = ">=8.3.2,<9.0.0"
pytest-asyncio = ">=0.24.0,<0.24.1"
pytest-cov = ">=5.0.0,<6"
pytest-mock = "^3.14.0"
pytest-xdist = { extras = ["psutil"], version = "^3.6.1" }
@@ -68,10 +68,10 @@ freezegun = "^1.5.1"
optional = true
[tool.poetry.group.docker.dependencies]
nb-cli = "^1.4.1"
nonebot2 = { extras = ["fastapi", "aiohttp"], version = "^2.3.2" }
nb-cli = "^1.4.2"
nonebot2 = { extras = ["fastapi", "aiohttp"], version = "^2.3.3" }
nonebot-adapter-red = "^0.9.0"
nonebot-adapter-qq = "^1.4.4"
nonebot-adapter-qq = "^1.5.1"
poetry-core = "^1.9.0"
[tool.poetry.extras]
View File
+123
View File
@@ -0,0 +1,123 @@
import json
from typing import cast
from datetime import datetime
import pytest
from nonebug import App
async def test_cookie(app: App, init_scheduler):
from nonebot_plugin_saa import TargetQQGroup
from nonebot_bison.platform import site_manager
from nonebot_bison.config.db_config import config
from nonebot_bison.types import Target as T_Target
from nonebot_bison.utils.site import CookieClientManager
from nonebot_bison.config.utils import DuplicateCookieTargetException
target = T_Target("weibo_id")
platform_name = "weibo"
await config.add_subscribe(
TargetQQGroup(group_id=123),
target=target,
target_name="weibo_name",
platform_name=platform_name,
cats=[],
tags=[],
)
site = site_manager["weibo.com"]
client_mgr = cast(CookieClientManager, site.client_mgr)
# 刷新匿名cookie
await client_mgr.refresh_anonymous_cookie()
cookies = await config.get_cookie(site_name=site.name)
assert len(cookies) == 1
# 添加用户cookie
await client_mgr.add_user_cookie(json.dumps({"test_cookie": "1"}))
await client_mgr.add_user_cookie(json.dumps({"test_cookie": "2"}))
cookies = await config.get_cookie(site_name=site.name)
assert len(cookies) == 3
cookies = await config.get_cookie(site_name=site.name, is_anonymous=False)
assert len(cookies) == 2
# 单个target,多个cookie
await config.add_cookie_target(target, platform_name, cookies[0].id)
await config.add_cookie_target(target, platform_name, cookies[1].id)
cookies = await config.get_cookie(site_name=site.name, target=target)
assert len(cookies) == 3
cookies = await config.get_cookie(site_name=site.name, target=target, is_anonymous=False)
assert len(cookies) == 2
cookies = await config.get_cookie(site_name=site.name, target=target, is_universal=False)
assert len(cookies) == 2
# 测试不同的target
target2 = T_Target("weibo_id2")
await config.add_subscribe(
TargetQQGroup(group_id=123),
target=target2,
target_name="weibo_name2",
platform_name=platform_name,
cats=[],
tags=[],
)
await client_mgr.add_user_cookie(json.dumps({"test_cookie": "3"}))
cookies = await config.get_cookie(site_name=site.name, is_anonymous=False)
# 多个target,多个cookie
await config.add_cookie_target(target2, platform_name, cookies[0].id)
await config.add_cookie_target(target2, platform_name, cookies[2].id)
cookies = await config.get_cookie(site_name=site.name, target=target2)
assert len(cookies) == 3
# 重复关联 target
with pytest.raises(DuplicateCookieTargetException) as e:
await config.add_cookie_target(target2, platform_name, cookies[2].id)
assert isinstance(e.value, DuplicateCookieTargetException)
cookies = await config.get_cookie(site_name=site.name, target=target2, is_anonymous=False)
assert len(cookies) == 2
# 有关联的cookie不能删除
with pytest.raises(Exception, match="cookie") as e:
await config.delete_cookie_by_id(cookies[1].id)
cookies = await config.get_cookie(site_name=site.name, target=target2, is_anonymous=False)
assert len(cookies) == 2
await config.delete_cookie_target(target2, platform_name, cookies[1].id)
await config.delete_cookie_by_id(cookies[1].id)
cookies = await config.get_cookie(site_name=site.name, target=target2, is_anonymous=False)
assert len(cookies) == 1
cookie = cookies[0]
cookie_id = cookie.id
cookie.last_usage = datetime(2024, 9, 13)
cookie.status = "test"
await config.update_cookie(cookie)
cookies = await config.get_cookie(site_name=site.name, target=target2, is_anonymous=False)
assert len(cookies) == 1
assert cookies[0].id == cookie_id
assert cookies[0].last_usage == datetime(2024, 9, 13)
assert cookies[0].status == "test"
# 不存在的 cookie_id
cookie.id = 114514
with pytest.raises(ValueError, match="cookie") as e:
await config.update_cookie(cookie)
# 获取所有关联对象
cookie_targets = await config.get_cookie_target()
assert len(cookie_targets) == 3
# 删除关联对象
await config.delete_cookie_target_by_id(cookie_targets[0].id)
cookie_targets = await config.get_cookie_target()
assert len(cookie_targets) == 2
+10
View File
@@ -18,6 +18,7 @@ def pytest_configure(config: pytest.Config) -> None:
"superusers": {"10001"},
"command_start": {""},
"log_level": "TRACE",
"bison_use_browser": True,
}
@@ -113,3 +114,12 @@ async def use_legacy_config(app: App):
# 清除单例的缓存
Singleton._instances.clear()
@pytest.fixture
async def _no_browser(app: App, mocker: MockerFixture):
from nonebot_bison.plugin_config import plugin_config
from nonebot_bison.platform import _get_unavailable_platforms
mocker.patch.object(plugin_config, "bison_use_browser", False)
mocker.patch("nonebot_bison.platform.unavailable_paltforms", _get_unavailable_platforms())
+94 -1
View File
@@ -58,6 +58,92 @@ def without_dynamic(app: App):
)
@pytest.mark.asyncio
async def test_reset_on_exception(app: App):
from strenum import StrEnum
from nonebot_bison.platform.bilibili.fsm import FSM, StateGraph, Transition, ActionReturn, reset_on_exception
class State(StrEnum):
A = "A"
B = "B"
C = "C"
class Event(StrEnum):
A = "A"
B = "B"
C = "C"
class Addon:
pass
async def raction(from_: State, event: Event, to: State, addon: Addon) -> ActionReturn:
logger.info(f"action: {from_} -> {to}")
raise RuntimeError("test")
async def action(from_: State, event: Event, to: State, addon: Addon) -> ActionReturn:
logger.info(f"action: {from_} -> {to}")
graph: StateGraph[State, Event, Addon] = {
"transitions": {
State.A: {
Event.A: Transition(raction, State.B),
Event.B: Transition(action, State.C),
},
State.B: {
Event.B: Transition(action, State.C),
},
State.C: {
Event.C: Transition(action, State.A),
},
},
"initial": State.A,
}
addon = Addon()
class AFSM(FSM[State, Event, Addon]):
@reset_on_exception(auto_start=True)
async def emit(self, event: Event):
return await super().emit(event)
fsm = AFSM(graph, addon)
await fsm.start()
with pytest.raises(RuntimeError):
await fsm.emit(Event.A)
assert fsm.started is True
await fsm.emit(Event.B)
await fsm.emit(Event.C)
class BFSM(FSM[State, Event, Addon]):
@reset_on_exception
async def emit(self, event: Event):
return await super().emit(event)
fsm = BFSM(graph, addon)
await fsm.start()
with pytest.raises(RuntimeError):
await fsm.emit(Event.A)
assert fsm.started is False
with pytest.raises(TypeError, match="can't send non-None value to a just-started async generator"):
await fsm.emit(Event.B)
class CFSM(FSM[State, Event, Addon]): ...
fsm = CFSM(graph, addon)
await fsm.start()
with pytest.raises(RuntimeError):
await fsm.emit(Event.A)
assert fsm.started is True
with pytest.raises(StopAsyncIteration):
await fsm.emit(Event.B)
@pytest.mark.asyncio
async def test_retry_for_352(app: App, mocker: MockerFixture):
from nonebot_bison.post import Post
@@ -183,7 +269,7 @@ async def test_retry_for_352(app: App, mocker: MockerFixture):
fakebili.set_raise352(True)
for state in test_state_list:
for state in test_state_list[:-3]:
logger.info(f"\n\nnow state should be {state}")
assert _retry_fsm.current_state == state
@@ -194,6 +280,13 @@ async def test_retry_for_352(app: App, mocker: MockerFixture):
if state == RetryState.BACKOFF:
freeze_start += timedelta_length * (_retry_fsm.addon.backoff_count + 1) ** 2
for state in test_state_list[-3:]:
logger.info(f"\n\nnow state should be {state}")
assert _retry_fsm.current_state == state
with pytest.raises(ApiCode352Error):
await fakebili.get_sub_list(Target("t1")) # type: ignore
assert client_mgr.refresh_client_call_count == 4 * 3 + 3 # refresh + raise
assert client_mgr.get_client_call_count == 2 + 4 * 3 + 3 # previous + refresh + raise
+18 -4
View File
@@ -88,9 +88,21 @@ async def test_fetch_new_1(
assert post1.title is None
assert (
post1.content
== "【#統合戦略】 引き続き新テーマ「ミヅキと紺碧の樹」の新要素及びシステムの変更点を一部ご紹介します!"
" 今回は「灯火」、「ダイス」、「記号認識」、「鍵」についてです。詳細は添付の画像をご確認ください。"
"#アークナイツ https://t.co/ARmptV0Zvu"
== "【#統合戦略】 <br />引き続き新テーマ「ミヅキと紺碧の樹」の新要素及びシステムの変更点を一部ご紹介します! "
"<br /><br />"
"今回は「灯火」、「ダイス」、「記号認識」、「鍵」についてです。<br />詳細は添付の画像をご確認ください。"
"<br /><br />"
"#アークナイツ https://t.co/ARmptV0Zvu<br />"
'<img src="https://pbs.twimg.com/media/FwZG9YAacAIXDw2?format=jpg&amp;name=orig" />'
)
plain_content = await post1.get_plain_content()
assert (
plain_content == "【#統合戦略】 \n"
"引き続き新テーマ「ミヅキと紺碧の樹」の新要素及びシステムの変更点を一部ご紹介します! \n\n"
"今回は「灯火」、「ダイス」、「記号認識」、「鍵」についてです。\n"
"詳細は添付の画像をご確認ください。\n\n"
"#アークナイツ https://t.co/ARmptV0Zvu\n"
"[图片]"
)
@@ -174,7 +186,9 @@ async def test_fetch_new_4(
assert len(res2[0][1]) == 1
post1 = res2[0][1][0]
assert post1.url == "https://wallhaven.cc/w/85rjej"
assert post1.content == "85rjej.jpg"
assert post1.content == '<img alt="loading" class="lazyload" src="https://th.wallhaven.cc/small/85/85rjej.jpg" />'
plain_content = await post1.get_plain_content()
assert plain_content == "[图片]"
def test_similar_text_process():
+45
View File
@@ -615,3 +615,48 @@ async def test_add_with_bilibili_bangumi_target_parser(app: App, init_scheduler)
assert sub.tags == []
assert sub.target.platform_name == "bilibili-bangumi"
assert sub.target.target_name == "汉化日记 第三季"
@pytest.mark.asyncio
async def test_subscribe_platform_requires_browser(app: App, mocker: MockerFixture):
from nonebot.adapters.onebot.v11.event import Sender
from nonebot.adapters.onebot.v11.message import Message
from nonebot_bison.plugin_config import plugin_config
from nonebot_bison.sub_manager import add_sub_matcher, common_platform
from nonebot_bison.platform import platform_manager, unavailable_paltforms
mocker.patch.object(plugin_config, "bison_use_browser", False)
mocker.patch.dict(unavailable_paltforms, {"bilibili": "需要启用 bison_use_browser"})
async with app.test_matcher(add_sub_matcher) as ctx:
bot = ctx.create_bot()
event_1 = fake_group_message_event(
message=Message("添加订阅"),
sender=Sender(card="", nickname="test", role="admin"),
to_me=True,
)
ctx.receive_event(bot, event_1)
ctx.should_pass_rule()
ctx.should_call_send(
event_1,
BotReply.add_reply_on_platform(platform_manager=platform_manager, common_platform=common_platform),
True,
)
event_2 = fake_group_message_event(
message=Message("全部"), sender=Sender(card="", nickname="test", role="admin")
)
ctx.receive_event(bot, event_2)
ctx.should_rejected()
ctx.should_call_send(
event_2,
BotReply.add_reply_on_platform_input_allplatform(platform_manager),
True,
)
event_3 = fake_group_message_event(message=Message("bilibili"), sender=fake_admin_user)
ctx.receive_event(bot, event_3)
ctx.should_call_send(
event_3,
BotReply.add_reply_platform_unavailable("bilibili", "需要启用 bison_use_browser"),
True,
)
+212
View File
@@ -0,0 +1,212 @@
import json
from nonebug.app import App
from pytest_mock import MockerFixture
from ..utils import BotReply, fake_superuser, fake_admin_user, fake_private_message_event
async def test_add_cookie_rule(app: App, mocker: MockerFixture):
from nonebot.adapters.onebot.v11.bot import Bot
from nonebot.adapters.onebot.v11.message import Message
from nonebot_bison.plugin_config import plugin_config
from nonebot_bison.sub_manager import add_cookie_matcher
mocker.patch.object(plugin_config, "bison_to_me", True)
async with app.test_matcher(add_cookie_matcher) as ctx:
bot = ctx.create_bot(base=Bot)
event = fake_private_message_event(message=Message("添加cookie"), sender=fake_superuser)
ctx.receive_event(bot, event)
ctx.should_pass_rule()
ctx.should_pass_permission()
async with app.test_matcher(add_cookie_matcher) as ctx:
bot = ctx.create_bot(base=Bot)
event = fake_private_message_event(message=Message("添加cookie"), sender=fake_admin_user)
ctx.receive_event(bot, event)
ctx.should_not_pass_rule()
ctx.should_pass_permission()
async def test_add_cookie_target_no_cookie(app: App, mocker: MockerFixture):
from nonebot.adapters.onebot.v11.bot import Bot
from nonebot.adapters.onebot.v11.message import Message
from nonebot_bison.sub_manager import add_cookie_target_matcher
async with app.test_matcher(add_cookie_target_matcher) as ctx:
bot = ctx.create_bot(base=Bot)
from nonebug_saa import should_send_saa
from nonebot_plugin_saa import TargetQQGroup, MessageFactory
from nonebot_bison.config import config
from nonebot_bison.types import Target as T_Target
target = T_Target("weibo_id")
platform_name = "weibo"
await config.add_subscribe(
TargetQQGroup(group_id=123),
target=target,
target_name="weibo_name",
platform_name=platform_name,
cats=[],
tags=[],
)
event_1 = fake_private_message_event(
message=Message("关联cookie"), sender=fake_superuser, to_me=True, user_id=fake_superuser.user_id
)
ctx.receive_event(bot, event_1)
ctx.should_pass_rule()
should_send_saa(
ctx,
MessageFactory(
"订阅的帐号为:\n1 weibo weibo_name weibo_id\n []\n请输入要关联 cookie 的订阅的序号\n输入'取消'中止"
),
bot,
event=event_1,
)
event_2 = fake_private_message_event(
message=Message("1"), sender=fake_superuser, to_me=True, user_id=fake_superuser.user_id
)
ctx.receive_event(bot, event_2)
ctx.should_pass_rule()
ctx.should_call_send(
event_2,
"当前平台暂无可关联的 Cookie,请使用“添加cookie”命令添加或检查已关联的 Cookie",
True,
)
async def test_add_cookie(app: App, mocker: MockerFixture):
from nonebot.adapters.onebot.v11.bot import Bot
from nonebot.adapters.onebot.v11.message import Message
from nonebot_bison.platform import platform_manager
from nonebot_bison.sub_manager import common_platform, add_cookie_matcher, add_cookie_target_matcher
async with app.test_matcher(add_cookie_matcher) as ctx:
bot = ctx.create_bot(base=Bot)
event_1 = fake_private_message_event(
message=Message("添加Cookie"), sender=fake_superuser, to_me=True, user_id=fake_superuser.user_id
)
ctx.receive_event(bot, event_1)
ctx.should_pass_rule()
ctx.should_call_send(
event_1,
BotReply.add_reply_on_add_cookie(platform_manager, common_platform),
True,
)
event_2 = fake_private_message_event(
message=Message("全部"), sender=fake_superuser, to_me=True, user_id=fake_superuser.user_id
)
ctx.receive_event(bot, event_2)
ctx.should_pass_rule()
ctx.should_rejected()
ctx.should_call_send(
event_2,
BotReply.add_reply_on_add_cookie_input_allplatform(platform_manager),
True,
)
event_3 = fake_private_message_event(
message=Message("weibo"), sender=fake_superuser, to_me=True, user_id=fake_superuser.user_id
)
ctx.receive_event(bot, event_3)
ctx.should_pass_rule()
ctx.should_call_send(event_3, BotReply.add_reply_on_input_cookie)
event_4_err = fake_private_message_event(
message=Message("test"), sender=fake_superuser, to_me=True, user_id=fake_superuser.user_id
)
ctx.receive_event(bot, event_4_err)
ctx.should_call_send(event_4_err, "无效的 Cookie,请检查后重新输入,详情见<待添加的文档>", True)
ctx.should_rejected()
event_4_ok = fake_private_message_event(
message=Message(json.dumps({"cookie": "test"})),
sender=fake_superuser,
to_me=True,
user_id=fake_superuser.user_id,
)
ctx.receive_event(bot, event_4_ok)
ctx.should_pass_rule()
ctx.should_call_send(
event_4_ok, '已添加 Cookie: {"cookie": "test"} 到平台 weibo\n请使用“关联cookie”为 Cookie 关联订阅', True
)
async with app.test_matcher(add_cookie_target_matcher) as ctx:
from nonebug_saa import should_send_saa
from nonebot_plugin_saa import TargetQQGroup, MessageFactory
from nonebot_bison.config import config
from nonebot_bison.types import Target as T_Target
target = T_Target("weibo_id")
platform_name = "weibo"
await config.add_subscribe(
TargetQQGroup(group_id=123),
target=target,
target_name="weibo_name",
platform_name=platform_name,
cats=[],
tags=[],
)
bot = ctx.create_bot(base=Bot)
event_1 = fake_private_message_event(
message=Message("关联cookie"), sender=fake_superuser, to_me=True, user_id=fake_superuser.user_id
)
ctx.receive_event(bot, event_1)
ctx.should_pass_rule()
should_send_saa(
ctx,
MessageFactory(
"订阅的帐号为:\n1 weibo weibo_name weibo_id\n []\n请输入要关联 cookie 的订阅的序号\n输入'取消'中止"
),
bot,
event=event_1,
)
event_2_err = fake_private_message_event(
message=Message("2"), sender=fake_superuser, to_me=True, user_id=fake_superuser.user_id
)
ctx.receive_event(bot, event_2_err)
ctx.should_call_send(event_2_err, "序号错误", True)
ctx.should_rejected()
event_2_ok = fake_private_message_event(
message=Message("1"), sender=fake_superuser, to_me=True, user_id=fake_superuser.user_id
)
ctx.receive_event(bot, event_2_ok)
ctx.should_pass_rule()
ctx.should_call_send(event_2_ok, '请选择一个 Cookie,已关联的 Cookie 不会显示\n1. weibo.com [{"cookie":]', True)
event_3_err = fake_private_message_event(
message=Message("2"), sender=fake_superuser, to_me=True, user_id=fake_superuser.user_id
)
ctx.receive_event(bot, event_3_err)
ctx.should_call_send(event_3_err, "序号错误", True)
ctx.should_rejected()
event_3_ok = fake_private_message_event(
message=Message("1"), sender=fake_superuser, to_me=True, user_id=fake_superuser.user_id
)
ctx.receive_event(bot, event_3_ok)
ctx.should_pass_rule()
ctx.should_call_send(event_3_ok, '已关联 Cookie: weibo.com [{"cookie":] 到订阅 weibo.com weibo_id', True)
async def test_add_cookie_target_no_target(app: App, mocker: MockerFixture):
from nonebot.adapters.onebot.v11.bot import Bot
from nonebot.adapters.onebot.v11.message import Message
from nonebot_bison.sub_manager import add_cookie_target_matcher
async with app.test_matcher(add_cookie_target_matcher) as ctx:
bot = ctx.create_bot(base=Bot)
event_1 = fake_private_message_event(
message=Message("关联cookie"), sender=fake_superuser, to_me=True, user_id=fake_superuser.user_id
)
ctx.receive_event(bot, event_1)
ctx.should_pass_rule()
ctx.should_call_send(
event_1,
"暂无已订阅账号\n请使用“添加订阅”命令添加订阅",
True,
)
+133
View File
@@ -0,0 +1,133 @@
import json
from nonebug.app import App
from ..utils import fake_superuser, fake_private_message_event
async def test_del_cookie_err(app: App):
from nonebug_saa import should_send_saa
from nonebot.adapters.onebot.v11.bot import Bot
from nonebot.adapters.onebot.v11.message import Message
from nonebot_plugin_saa import TargetQQGroup, MessageFactory
from nonebot_bison.config import config
from nonebot_bison.config.db_model import Cookie
from nonebot_bison.types import Target as T_Target
from nonebot_bison.sub_manager import del_cookie_matcher
async with app.test_matcher(del_cookie_matcher) as ctx:
bot = ctx.create_bot(base=Bot)
event = fake_private_message_event(
message=Message("删除cookie"), sender=fake_superuser, to_me=True, user_id=fake_superuser.user_id
)
ctx.receive_event(bot, event)
ctx.should_pass_rule()
ctx.should_pass_permission()
ctx.should_call_send(event, "暂无已添加的 Cookie\n请使用“添加cookie”命令添加", True)
async with app.test_matcher(del_cookie_matcher) as ctx:
bot = ctx.create_bot(base=Bot)
target = T_Target("weibo_id")
platform_name = "weibo"
await config.add_subscribe(
TargetQQGroup(group_id=123),
target=target,
target_name="weibo_name",
platform_name=platform_name,
cats=[],
tags=[],
)
await config.add_cookie(Cookie(content=json.dumps({"cookie": "test"}), site_name="weibo.com"))
cookies = await config.get_cookie(is_anonymous=False)
await config.add_cookie_target(target, platform_name, cookies[0].id)
event_1 = fake_private_message_event(
message=Message("删除cookie"), sender=fake_superuser, to_me=True, user_id=fake_superuser.user_id
)
ctx.receive_event(bot, event_1)
ctx.should_pass_rule()
ctx.should_pass_permission()
should_send_saa(
ctx,
MessageFactory(
'已添加的 Cookie 为:\n1 weibo.com weibo.com [{"cookie":] '
"1个关联\n请输入要删除的 Cookie 的序号\n输入'取消'中止"
),
bot,
event=event_1,
)
event_2_err = fake_private_message_event(
message=Message("2"), sender=fake_superuser, to_me=True, user_id=fake_superuser.user_id
)
ctx.receive_event(bot, event_2_err)
ctx.should_call_send(event_2_err, "序号错误", True)
ctx.should_rejected()
event_2 = fake_private_message_event(
message=Message("1"), sender=fake_superuser, to_me=True, user_id=fake_superuser.user_id
)
ctx.receive_event(bot, event_2)
ctx.should_pass_rule()
ctx.should_call_send(event_2, "只能删除未关联的 Cookie,请使用“取消关联cookie”命令取消关联", True)
ctx.should_call_send(event_2, "删除错误", True)
ctx.should_rejected()
async def test_del_cookie(app: App):
from nonebug_saa import should_send_saa
from nonebot.adapters.onebot.v11.bot import Bot
from nonebot.adapters.onebot.v11.message import Message
from nonebot_plugin_saa import TargetQQGroup, MessageFactory
from nonebot_bison.config import config
from nonebot_bison.config.db_model import Cookie
from nonebot_bison.types import Target as T_Target
from nonebot_bison.sub_manager import del_cookie_matcher
async with app.test_matcher(del_cookie_matcher) as ctx:
bot = ctx.create_bot(base=Bot)
event = fake_private_message_event(
message=Message("删除cookie"), sender=fake_superuser, to_me=True, user_id=fake_superuser.user_id
)
ctx.receive_event(bot, event)
ctx.should_pass_rule()
ctx.should_pass_permission()
ctx.should_call_send(event, "暂无已添加的 Cookie\n请使用“添加cookie”命令添加", True)
async with app.test_matcher(del_cookie_matcher) as ctx:
bot = ctx.create_bot(base=Bot)
target = T_Target("weibo_id")
platform_name = "weibo"
await config.add_subscribe(
TargetQQGroup(group_id=123),
target=target,
target_name="weibo_name",
platform_name=platform_name,
cats=[],
tags=[],
)
await config.add_cookie(Cookie(content=json.dumps({"cookie": "test"}), site_name="weibo.com"))
event_1 = fake_private_message_event(
message=Message("删除cookie"), sender=fake_superuser, to_me=True, user_id=fake_superuser.user_id
)
ctx.receive_event(bot, event_1)
ctx.should_pass_rule()
ctx.should_pass_permission()
should_send_saa(
ctx,
MessageFactory(
'已添加的 Cookie 为:\n1 weibo.com weibo.com [{"cookie":]'
" 0个关联\n请输入要删除的 Cookie 的序号\n输入'取消'中止"
),
bot,
event=event_1,
)
event_2 = fake_private_message_event(
message=Message("1"), sender=fake_superuser, to_me=True, user_id=fake_superuser.user_id
)
ctx.receive_event(bot, event_2)
ctx.should_pass_rule()
ctx.should_pass_permission()
ctx.should_call_send(event_2, "删除成功", True)
+45
View File
@@ -0,0 +1,45 @@
import pytest
from nonebug import App
from ..utils import BotReply, fake_admin_user, fake_group_message_event
@pytest.mark.asyncio
async def test_with_permission(app: App):
from nonebot.adapters.onebot.v11.bot import Bot
from nonebot.adapters.onebot.v11.message import Message
from nonebot_bison.platform import platform_manager
from nonebot_bison.sub_manager import add_sub_matcher, common_platform, no_permission_matcher
async with app.test_matcher([add_sub_matcher, no_permission_matcher]) as ctx:
bot = ctx.create_bot(base=Bot)
event = fake_group_message_event(message=Message("添加订阅"), sender=fake_admin_user, to_me=True)
ctx.receive_event(bot, event)
ctx.should_call_send(
event,
BotReply.add_reply_on_platform(platform_manager, common_platform),
True,
)
ctx.should_pass_rule()
ctx.should_pass_permission()
@pytest.mark.asyncio
async def test_without_permission(app: App):
from nonebot.adapters.onebot.v11.bot import Bot
from nonebot.adapters.onebot.v11.message import Message
from nonebot_bison.sub_manager import add_sub_matcher, no_permission_matcher
async with app.test_matcher([add_sub_matcher, no_permission_matcher]) as ctx:
bot = ctx.create_bot(base=Bot)
event = fake_group_message_event(message=Message("添加订阅"), to_me=True)
ctx.receive_event(bot, event)
ctx.should_call_send(
event,
BotReply.no_permission,
True,
)
ctx.should_pass_rule()
ctx.should_pass_permission()
+36
View File
@@ -89,6 +89,7 @@ add_reply_on_id_input_search = (
class BotReply:
@staticmethod
def add_reply_on_platform(platform_manager, common_platform):
return (
@@ -146,6 +147,10 @@ class BotReply:
extra_text = ("1." + target_promot + "\n2.") if target_promot else ""
return extra_text + base_text
@staticmethod
def add_reply_platform_unavailable(platform: str, reason: str) -> str:
return f"无法订阅 {platform}{reason}"
add_reply_on_id_input_error = "id输入错误"
add_reply_on_target_parse_input_error = "不能从你的输入中提取出id,请检查你输入的内容是否符合预期"
add_reply_on_platform_input_error = "平台输入错误"
@@ -154,3 +159,34 @@ class BotReply:
)
add_reply_on_tags_need_more_info = "订阅标签直接输入标签内容\n屏蔽标签请在标签名称前添加~号\n详见https://nonebot-bison.netlify.app/usage/#%E5%B9%B3%E5%8F%B0%E8%AE%A2%E9%98%85%E6%A0%87%E7%AD%BE-tag"
add_reply_abort = "已中止订阅"
no_permission = "您没有权限进行此操作,请联系 Bot 管理员"
@staticmethod
def add_reply_on_add_cookie(platform_manager, common_platform):
from nonebot_bison.utils.site import is_cookie_client_manager
return (
"请输入想要添加 Cookie 的平台,目前支持,请输入冒号左边的名称:\n"
+ "".join(
[
f"{platform_name}: {platform_manager[platform_name].name}\n"
for platform_name in common_platform
if is_cookie_client_manager(platform_manager[platform_name].site.client_mgr)
]
)
+ "要查看全部平台请输入:“全部”\n中止添加cookie过程请输入:“取消”"
)
@staticmethod
def add_reply_on_add_cookie_input_allplatform(platform_manager):
from nonebot_bison.utils.site import is_cookie_client_manager
return "全部平台\n" + "\n".join(
[
f"{platform_name}: {platform.name}"
for platform_name, platform in platform_manager.items()
if is_cookie_client_manager(platform.site.client_mgr)
]
)
add_reply_on_input_cookie = "请输入 Cookie"