From b94f6cc31844e9307e355fc81f387ea42501a014 Mon Sep 17 00:00:00 2001 From: panda361 <35888512+panda361@users.noreply.github.com> Date: Thu, 18 Feb 2021 14:01:36 +0800 Subject: [PATCH 01/11] update readme Fix typo --- README.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index c3a3965..5269f26 100644 --- a/README.md +++ b/README.md @@ -9,13 +9,13 @@ * 文字 * 不支持视频 * 不支持转发的内容 -* bilibili +* Bilibili * 图片 * 专栏 * 文字 * 视频链接 * 不支持转发的内容 -* rss +* RSS * 从description中提取图片 * 文字 @@ -37,26 +37,26 @@ * 查询订阅：`查询订阅` * 删除订阅（仅管理员和群主）：`删除订阅` -平台代码包含：weibo，bilibili，rss +平台代码包含：Weibo，Bilibili，RSS
各平台uid 下面均以pc站点为例 -* weibo +* Weibo * 对于一般用户主页`https://weibo.com/u/6441489862?xxxxxxxxxxxxxxx`，`/u/`后面的数字即为uid * 对于有个性域名的用户如：`https://weibo.com/arknights`，需要点击左侧信息标签下“更多”，链接为`https://weibo.com/6279793937/about`，其中中间数字即为uid -* bilibili +* Bilibili * 主页链接一般为`https://space.bilibili.com/161775300?xxxxxxxxxx`，数字即为uid -* rss - * rss链接即为uid +* RSS + * RSS链接即为uid
### 文字转图片 -因为可能要发送长文本，所以bot很可能被风控，如有需要请开启以图片形式发送文字，本项目使用的图片转文字方法是chromium（经典杀鸡用牛刀）。 +因为可能要发送长文本，所以bot很可能被风控，如有需要请开启以图片形式发送文字，本项目使用的文字转图片方法是Chromium（经典杀鸡用牛刀）。 -如果确定要开启推荐自行安装chromium，设置使用本地chromium，并且保证服务器有比较大的内存。 +如果确定要开启推荐自行安装Chromium，设置使用本地Chromium，并且保证服务器有比较大的内存。 ## 功能 -* 定时爬取制定网站 +* 定时爬取指定网站 * 通过图片发送文本，防止风控 * 使用队列限制发送频率 From f34a5dca9bbda127cfc9959b68100bfc40a5e896 Mon Sep 17 00:00:00 2001 From: felinae98 <731499577@qq.com> Date: Sat, 20 Feb 2021 21:23:13 +0800 Subject: [PATCH 02/11] update to v0.2.3 --- pyproject.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pyproject.toml b/pyproject.toml index c01f96b..359971e 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [tool.poetry] name = "nonebot-hk-reporter" -version = "0.2.2" +version = "0.2.3" description = "Subscribe message from social medias" authors = ["felinae98 "] license = "MIT" From b92201f61f41fde28e3bad1c7f52c258a6c6b956 Mon Sep 17 00:00:00 2001 From: felinae98 <731499577@qq.com> Date: Wed, 24 Feb 2021 20:57:07 +0800 Subject: [PATCH 03/11] Create LICENSE --- LICENSE | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 LICENSE diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..e857061 --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2021 felinae98 + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. From bf8ca7f6da2419e1fe4cb58768c7f384b07c49ff Mon Sep 17 00:00:00 2001 From: felinae98 <731499577@qq.com> Date: Wed, 24 Feb 2021 21:01:06 +0800 Subject: [PATCH 04/11] update readme --- README.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index c3a3965..30b8387 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,8 @@ -# hk-reporter 通用订阅推送插件 +
+

hk-reporter
通用订阅推送插件

+![https://badgen.net/github/license/felinae98/nonebot-hk-reporter]() +![https://badgen.net/pypi/v/nonebot-hk-reporter](https://pypi.org/project/nonebot-hk-reporter/) +
## 简介 一款自动爬取各种站点，社交平台更新动态，并将信息推送到QQ的机器人。基于 [`NoneBot2`](https://github.com/nonebot/nonebot2 ) 开发（诞生于明日方舟的蹲饼活动） @@ -25,6 +29,14 @@ 本项目可作为单独插件使用，仅包含订阅相关功能（绝对simple和stupid），也可直接克隆项目进行使用（包含自动同意superuser，自动接受入群邀请等功能） 作为插件使用请安装`nonebot-hk-reporter`包，并在`bot.py`中加载`nonebot_hk_reporter`插件；或直接克隆本项目进行使用 配置与安装请参考[nonebot2文档](https://v2.nonebot.dev/) +
+Docker部署方法 +Docker镜像地址为`felinae98/nonebot-hk-reporter`对应main分支，`felinae98/nonebot-hk-reporter:arknights`对应arknights分支。例子： +```bash +docker run --name nonebot-hk-reporter --network -d -e 'SUPERUSERS=[]' -v :/data -e 'hk_reporter_config_path=/data' -e 'HK_REPORTER_USE_PIC=True' -e 'HK_REPORTER_USE_LOCAL=True' +``` +go-cqhttp镜像可使用`felinae98/go-cqhttp-ffmpeg`（数据目录为`/data`），需要注意，两个容器需要在同一个network中。 +
### 配置变量 * `HK_REPORTER_CONFIG_PATH` (str) 配置文件保存目录，如果不设置，则为当前目录下的`data`文件夹 From 019bbb29ba9f5c0455d7d34e1d6df0503197deb5 Mon Sep 17 00:00:00 2001 From: felinae98 <731499577@qq.com> Date: Wed, 24 Feb 2021 21:08:18 +0800 Subject: [PATCH 05/11] Update README.md --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 30b8387..92015fe 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,9 @@

hk-reporter
通用订阅推送插件

-![https://badgen.net/github/license/felinae98/nonebot-hk-reporter]() -![https://badgen.net/pypi/v/nonebot-hk-reporter](https://pypi.org/project/nonebot-hk-reporter/)
+![LICENSE](https://badgen.net/github/license/felinae98/nonebot-hk-reporter) +![https://badgen.net/pypi/v/nonebot-hk-reporter](https://pypi.org/project/nonebot-hk-reporter/) + ## 简介 一款自动爬取各种站点，社交平台更新动态，并将信息推送到QQ的机器人。基于 [`NoneBot2`](https://github.com/nonebot/nonebot2 ) 开发（诞生于明日方舟的蹲饼活动） From 18cd4fc65c68eb314f15ffc29c29f393208b1bca Mon Sep 17 00:00:00 2001 From: felinae98 <731499577@qq.com> Date: Wed, 24 Feb 2021 21:20:30 +0800 Subject: [PATCH 06/11] Update README.md --- README.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 92015fe..9367a69 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,13 @@

hk-reporter
通用订阅推送插件

-
-![LICENSE](https://badgen.net/github/license/felinae98/nonebot-hk-reporter) -![https://badgen.net/pypi/v/nonebot-hk-reporter](https://pypi.org/project/nonebot-hk-reporter/) + +[![pypi](https://badgen.net/pypi/v/nonebot-hk-reporter)](https://pypi.org/project/nonebot-hk-reporter/) +[![qq group](https://img.shields.io/badge/QQ%E7%BE%A4-868610060-orange )](https://qm.qq.com/cgi-bin/qm/qr?k=pXYMGB_e8b6so3QTqgeV6lkKDtEeYE4f&jump_from=webapi) + + + ## 简介 一款自动爬取各种站点，社交平台更新动态，并将信息推送到QQ的机器人。基于 [`NoneBot2`](https://github.com/nonebot/nonebot2 ) 开发（诞生于明日方舟的蹲饼活动） From 0a49d9fde08a5cbdc74e66eb0d955c7194a5c35c Mon Sep 17 00:00:00 2001 From: felinae98 <731499577@qq.com> Date: Wed, 24 Feb 2021 21:22:25 +0800 Subject: [PATCH 07/11] Update README.md --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 657352c..d8b466a 100644 --- a/README.md +++ b/README.md @@ -35,9 +35,10 @@ 配置与安装请参考[nonebot2文档](https://v2.nonebot.dev/)
Docker部署方法 + Docker镜像地址为`felinae98/nonebot-hk-reporter`对应main分支，`felinae98/nonebot-hk-reporter:arknights`对应arknights分支。例子： ```bash -docker run --name nonebot-hk-reporter --network -d -e 'SUPERUSERS=[]' -v :/data -e 'hk_reporter_config_path=/data' -e 'HK_REPORTER_USE_PIC=True' -e 'HK_REPORTER_USE_LOCAL=True' +docker run --name nonebot-hk-reporter --network -d -e 'SUPERUSERS=[]' -v :/data -e 'hk_reporter_config_path=/data' -e 'HK_REPORTER_USE_PIC=True' -e 'HK_REPORTER_USE_LOCAL=True felinae98/nonebot-hk-reporter' ``` go-cqhttp镜像可使用`felinae98/go-cqhttp-ffmpeg`（数据目录为`/data`），需要注意，两个容器需要在同一个network中。
From 713bcaf32e85901a6451fffd9cb0340532e3ebb5 Mon Sep 17 00:00:00 2001 From: felinae98 <731499577@qq.com> Date: Wed, 24 Feb 2021 21:23:08 +0800 Subject: [PATCH 08/11] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d8b466a..37f18bb 100644 --- a/README.md +++ b/README.md @@ -38,7 +38,7 @@ Docker镜像地址为`felinae98/nonebot-hk-reporter`对应main分支，`felinae98/nonebot-hk-reporter:arknights`对应arknights分支。例子： ```bash -docker run --name nonebot-hk-reporter --network -d -e 'SUPERUSERS=[]' -v :/data -e 'hk_reporter_config_path=/data' -e 'HK_REPORTER_USE_PIC=True' -e 'HK_REPORTER_USE_LOCAL=True felinae98/nonebot-hk-reporter' +docker run --name nonebot-hk-reporter --network -d -e 'SUPERUSERS=[]' -v :/data -e 'hk_reporter_config_path=/data' -e 'HK_REPORTER_USE_PIC=True' -e 'HK_REPORTER_USE_LOCAL=True' felinae98/nonebot-hk-reporter ``` go-cqhttp镜像可使用`felinae98/go-cqhttp-ffmpeg`（数据目录为`/data`），需要注意，两个容器需要在同一个network中。 From b14604fd3a0ecf610c02336d57d892dfe184a5ad Mon Sep 17 00:00:00 2001 From: felinae98 <731499577@qq.com> Date: Fri, 26 Feb 2021 19:41:05 +0800 Subject: [PATCH 09/11] update to 0.2.4, fix unable to use by nbcli mentioned in #2 --- pyproject.toml | 6 +++--- .../{hk_reporter => nonebot_hk_reporter}/__init__.py | 0 src/plugins/{hk_reporter => nonebot_hk_reporter}/config.py | 0 .../{hk_reporter => nonebot_hk_reporter}/config_manager.py | 0 .../platform/__init__.py | 0 .../platform/bilibili.py | 0 .../platform/platform.py | 0 .../{hk_reporter => nonebot_hk_reporter}/platform/rss.py | 0 .../{hk_reporter => nonebot_hk_reporter}/platform/utils.py | 0 .../{hk_reporter => nonebot_hk_reporter}/platform/weibo.py | 0 .../{hk_reporter => nonebot_hk_reporter}/plugin_config.py | 0 src/plugins/{hk_reporter => nonebot_hk_reporter}/post.py | 0 .../{hk_reporter => nonebot_hk_reporter}/scheduler.py | 0 src/plugins/{hk_reporter => nonebot_hk_reporter}/send.py | 0 src/plugins/{hk_reporter => nonebot_hk_reporter}/types.py | 0 src/plugins/{hk_reporter => nonebot_hk_reporter}/utils.py | 0 16 files changed, 3 insertions(+), 3 deletions(-) rename src/plugins/{hk_reporter => nonebot_hk_reporter}/__init__.py (100%) rename src/plugins/{hk_reporter => nonebot_hk_reporter}/config.py (100%) rename src/plugins/{hk_reporter => nonebot_hk_reporter}/config_manager.py (100%) rename src/plugins/{hk_reporter => nonebot_hk_reporter}/platform/__init__.py (100%) rename src/plugins/{hk_reporter => nonebot_hk_reporter}/platform/bilibili.py (100%) rename src/plugins/{hk_reporter => nonebot_hk_reporter}/platform/platform.py (100%) rename src/plugins/{hk_reporter => nonebot_hk_reporter}/platform/rss.py (100%) rename src/plugins/{hk_reporter => nonebot_hk_reporter}/platform/utils.py (100%) rename src/plugins/{hk_reporter => nonebot_hk_reporter}/platform/weibo.py (100%) rename src/plugins/{hk_reporter => nonebot_hk_reporter}/plugin_config.py (100%) rename src/plugins/{hk_reporter => nonebot_hk_reporter}/post.py (100%) rename src/plugins/{hk_reporter => nonebot_hk_reporter}/scheduler.py (100%) rename src/plugins/{hk_reporter => nonebot_hk_reporter}/send.py (100%) rename src/plugins/{hk_reporter => nonebot_hk_reporter}/types.py (100%) rename src/plugins/{hk_reporter => nonebot_hk_reporter}/utils.py (100%) diff --git a/pyproject.toml b/pyproject.toml index 359971e..3d76c77 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [tool.poetry] name = "nonebot-hk-reporter" -version = "0.2.3" +version = "0.2.4" description = "Subscribe message from social medias" authors = ["felinae98 "] license = "MIT" @@ -8,8 +8,8 @@ homepage = "https://github.com/felinae98/nonebot-hk-reporter" keywords = ["nonebot", "nonebot2", "qqbot"] readme = "README.md" packages = [ - { include = "hk_reporter/*.py", from = "./src/plugins/" }, - { include = "hk_reporter/platform/*.py", from = "./src/plugins/" } + { include = "nonebot_hk_reporter/*.py", from = "./src/plugins/" }, + { include = "nonebot_hk_reporter/platform/*.py", from = "./src/plugins/" } ] classifiers = [ "Development Status :: 2 - Pre-Alpha", diff --git a/src/plugins/hk_reporter/__init__.py b/src/plugins/nonebot_hk_reporter/__init__.py similarity index 100% rename from src/plugins/hk_reporter/__init__.py rename to src/plugins/nonebot_hk_reporter/__init__.py diff --git a/src/plugins/hk_reporter/config.py b/src/plugins/nonebot_hk_reporter/config.py similarity index 100% rename from src/plugins/hk_reporter/config.py rename to src/plugins/nonebot_hk_reporter/config.py diff --git a/src/plugins/hk_reporter/config_manager.py b/src/plugins/nonebot_hk_reporter/config_manager.py similarity index 100% rename from src/plugins/hk_reporter/config_manager.py rename to src/plugins/nonebot_hk_reporter/config_manager.py diff --git a/src/plugins/hk_reporter/platform/__init__.py b/src/plugins/nonebot_hk_reporter/platform/__init__.py similarity index 100% rename from src/plugins/hk_reporter/platform/__init__.py rename to src/plugins/nonebot_hk_reporter/platform/__init__.py diff --git a/src/plugins/hk_reporter/platform/bilibili.py b/src/plugins/nonebot_hk_reporter/platform/bilibili.py similarity index 100% rename from src/plugins/hk_reporter/platform/bilibili.py rename to src/plugins/nonebot_hk_reporter/platform/bilibili.py diff --git a/src/plugins/hk_reporter/platform/platform.py b/src/plugins/nonebot_hk_reporter/platform/platform.py similarity index 100% rename from src/plugins/hk_reporter/platform/platform.py rename to src/plugins/nonebot_hk_reporter/platform/platform.py diff --git a/src/plugins/hk_reporter/platform/rss.py b/src/plugins/nonebot_hk_reporter/platform/rss.py similarity index 100% rename from src/plugins/hk_reporter/platform/rss.py rename to src/plugins/nonebot_hk_reporter/platform/rss.py diff --git a/src/plugins/hk_reporter/platform/utils.py b/src/plugins/nonebot_hk_reporter/platform/utils.py similarity index 100% rename from src/plugins/hk_reporter/platform/utils.py rename to src/plugins/nonebot_hk_reporter/platform/utils.py diff --git a/src/plugins/hk_reporter/platform/weibo.py b/src/plugins/nonebot_hk_reporter/platform/weibo.py similarity index 100% rename from src/plugins/hk_reporter/platform/weibo.py rename to src/plugins/nonebot_hk_reporter/platform/weibo.py diff --git a/src/plugins/hk_reporter/plugin_config.py b/src/plugins/nonebot_hk_reporter/plugin_config.py similarity index 100% rename from src/plugins/hk_reporter/plugin_config.py rename to src/plugins/nonebot_hk_reporter/plugin_config.py diff --git a/src/plugins/hk_reporter/post.py b/src/plugins/nonebot_hk_reporter/post.py similarity index 100% rename from src/plugins/hk_reporter/post.py rename to src/plugins/nonebot_hk_reporter/post.py diff --git a/src/plugins/hk_reporter/scheduler.py b/src/plugins/nonebot_hk_reporter/scheduler.py similarity index 100% rename from src/plugins/hk_reporter/scheduler.py rename to src/plugins/nonebot_hk_reporter/scheduler.py diff --git a/src/plugins/hk_reporter/send.py b/src/plugins/nonebot_hk_reporter/send.py similarity index 100% rename from src/plugins/hk_reporter/send.py rename to src/plugins/nonebot_hk_reporter/send.py diff --git a/src/plugins/hk_reporter/types.py b/src/plugins/nonebot_hk_reporter/types.py similarity index 100% rename from src/plugins/hk_reporter/types.py rename to src/plugins/nonebot_hk_reporter/types.py diff --git a/src/plugins/hk_reporter/utils.py b/src/plugins/nonebot_hk_reporter/utils.py similarity index 100% rename from src/plugins/hk_reporter/utils.py rename to src/plugins/nonebot_hk_reporter/utils.py From c47b0c4425adf3c8ec918448fcf5dd71d897d47f Mon Sep 17 00:00:00 2001 From: felinae98 <731499577@qq.com> Date: Fri, 26 Feb 2021 20:16:16 +0800 Subject: [PATCH 10/11] use tempfile, test --- src/plugins/nonebot_hk_reporter/utils.py | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/src/plugins/nonebot_hk_reporter/utils.py b/src/plugins/nonebot_hk_reporter/utils.py index 509466b..b43329d 100644 --- a/src/plugins/nonebot_hk_reporter/utils.py +++ b/src/plugins/nonebot_hk_reporter/utils.py @@ -7,6 +7,7 @@ import base64 from pyppeteer import launch from html import escape from hashlib import sha256 +from tempfile import NamedTemporaryFile from .plugin_config import plugin_config @@ -44,14 +45,14 @@ class Render(metaclass=Singleton): return str(data) async def text_to_pic(self, text: str) -> str: - hash_text = sha256(text.encode()).hexdigest()[:20] lines = text.split('\n') parsed_lines = list(map(lambda x: '

{}

'.format(escape(x)), lines)) html_text = '
{}
'.format(''.join(parsed_lines)) - with open('/tmp/text-{}.html'.format(hash_text), 'w') as f: - f.write(html_text) - data = await self.render('file:///tmp/text-{}.html'.format(hash_text), target='div') - os.remove('/tmp/text-{}.html'.format(hash_text)) + with NamedTemporaryFile('wt', suffix='.html', delete=False) as tmp: + tmp_path = tmp.name + tmp.write(html_text) + data = await self.render('file://{}'.format(tmp_path), target='div') + os.remove(tmp_path) return data async def text_to_pic_cqcode(self, text:str) -> str: @@ -68,3 +69,8 @@ async def parse_text(text: str): else: return text +async def test(): + ren = Render() + res = await ren.text_to_pic('12333333') + logger.debug(res) +nonebot.get_driver().on_startup(test) From 574c950afbc5e4e81d78516e09beb6e2097b2248 Mon Sep 17 00:00:00 2001 From: felinae98 <731499577@qq.com> Date: Fri, 26 Feb 2021 20:47:58 +0800 Subject: [PATCH 11/11] update to 0.2.5, suport windows --- pyproject.toml | 2 +- src/plugins/nonebot_hk_reporter/utils.py | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/pyproject.toml b/pyproject.toml index 3d76c77..d5e1996 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [tool.poetry] name = "nonebot-hk-reporter" -version = "0.2.4" +version = "0.2.5" description = "Subscribe message from social medias" authors = ["felinae98 "] license = "MIT" diff --git a/src/plugins/nonebot_hk_reporter/utils.py b/src/plugins/nonebot_hk_reporter/utils.py index b43329d..4fbcfd9 100644 --- a/src/plugins/nonebot_hk_reporter/utils.py +++ b/src/plugins/nonebot_hk_reporter/utils.py @@ -5,6 +5,7 @@ import nonebot from nonebot import logger import base64 from pyppeteer import launch +from pyppeteer.chromium_downloader import check_chromium, download_chromium from html import escape from hashlib import sha256 from tempfile import NamedTemporaryFile @@ -20,6 +21,10 @@ class Singleton(type): supported_target_type = ('weibo', 'bilibili', 'rss') +if not plugin_config.hk_reporter_use_local and not check_chromium(): + os.environ['PYPPETEER_DOWNLOAD_HOST'] = 'http://npm.taobao.org/mirrors' + download_chromium() + class Render(metaclass=Singleton): def __init__(self): @@ -62,15 +67,10 @@ class Render(metaclass=Singleton): # logger.debug(code) return code -async def parse_text(text: str): +async def parse_text(text: str) -> str: + 'return raw text if don\'t use pic, otherwise return rendered opcode' if plugin_config.hk_reporter_use_pic: - r = Render() - return await r.text_to_pic_cqcode(text) + render = Render() + return await render.text_to_pic_cqcode(text) else: return text - -async def test(): - ren = Render() - res = await ren.text_to_pic('12333333') - logger.debug(res) -nonebot.get_driver().on_startup(test)