18 Commits

Author SHA1 Message Date
suyiiyii e1c97aacad Update pyproject.toml
docker optional-dependencies
2024-12-23 01:03:27 +08:00
suyiiyii 2645314899 ♻️ Install playwright before pytest 2024-12-23 00:51:34 +08:00
suyiiyii 979f6b4cb2 ♻️ 锁依赖 2024-12-23 00:45:50 +08:00
suyiiyii 1274093622 🐛 change "proxies" to "proxy" in http.py 2024-12-22 21:32:55 +08:00
suyiiyii 0f1fe1515f ♻️ 怎么还是限制死了是单引号的 ( 2024-12-22 21:27:38 +08:00
suyiiyii 99c949691e ♻️ 全部换成字符串试试 2024-12-22 21:23:59 +08:00
suyiiyii 5bd67ad7ad ♻️ 忘记改类型了 ( 2024-12-22 21:21:22 +08:00
suyiiyii cac5e09f71 ♻️ 为什么会报这个错误呢 /home/runner/work/nonebot-bison/nonebot-bison/./.github/actions/setup-python/action.yml (Line: 28, Col: 7): Required property is missing: shell 2024-12-22 21:19:10 +08:00
suyiiyii 08c8bb955f ♻️ 将所有 poetry 替换为 uv 2024-12-22 21:15:32 +08:00
suyiiyii 6c8b37f085 ♻️ 更新 action workflow 2024-12-22 21:02:32 +08:00
suyiiyii 8550ff975a 🚀 在现有 Dockerfile 的基础上,修改无浏览器镜像 2024-12-22 20:34:40 +08:00
suyiiyii 8c6e77dd94 🚀 使用 带前端,sentry,浏览器的版本作为默认Dockerfile 2024-12-22 19:20:04 +08:00
suyiiyii 93b6055391 🚀 优化 dockerfile 2024-12-22 19:06:35 +08:00
BalconyJH 5a5b27d285 💚 Update docker build and remove obsolete dockerfile. 2024-12-22 17:35:32 +08:00
Azide c8df2b3cc2 🚧 重写 Dockerfile 2024-12-19 23:55:12 +08:00
Azide e20aba9040 ♻️ 迁移到 uv 2024-12-19 23:54:42 +08:00
suyiiyii 5a7396bada 👷 fix case and PYTHONPATH env warning in Dockerfile
test-build / Build Frontend (push) Has been cancelled
test-build / Smoke-test Coverage (macos-latest, 3.10) (push) Has been cancelled
test-build / Smoke-test Coverage (macos-latest, 3.11) (push) Has been cancelled
test-build / Smoke-test Coverage (macos-latest, 3.12) (push) Has been cancelled
test-build / Smoke-test Coverage (ubuntu-latest, 3.10) (push) Has been cancelled
test-build / Smoke-test Coverage (ubuntu-latest, 3.11) (push) Has been cancelled
test-build / Smoke-test Coverage (ubuntu-latest, 3.12) (push) Has been cancelled
test-build / Smoke-test Coverage (windows-latest, 3.10) (push) Has been cancelled
test-build / Smoke-test Coverage (windows-latest, 3.11) (push) Has been cancelled
test-build / Smoke-test Coverage (windows-latest, 3.12) (push) Has been cancelled
test-build / All-test Coverage (macos-latest, 3.10) (push) Has been cancelled
test-build / All-test Coverage (macos-latest, 3.11) (push) Has been cancelled
test-build / All-test Coverage (macos-latest, 3.12) (push) Has been cancelled
test-build / All-test Coverage (ubuntu-latest, 3.10) (push) Has been cancelled
test-build / All-test Coverage (ubuntu-latest, 3.11) (push) Has been cancelled
test-build / All-test Coverage (ubuntu-latest, 3.12) (push) Has been cancelled
test-build / All-test Coverage (windows-latest, 3.10) (push) Has been cancelled
test-build / All-test Coverage (windows-latest, 3.11) (push) Has been cancelled
test-build / All-test Coverage (windows-latest, 3.12) (push) Has been cancelled
pydantic1-compat-test / pydantic1 test (ubuntu-latest, 3.11) (push) Has been cancelled
Ruff Lint / Ruff Lint (push) Has been cancelled
test-build / Docker main (push) Has been cancelled
test-build / Docker main sentry (push) Has been cancelled
2024-12-17 16:29:10 +08:00
Azide 08ad5c288c 🧑‍💻 使用 Ruff 包办所有格式化和检查 (#663) 2024-12-17 10:52:21 +08:00
141 changed files with 4496 additions and 6391 deletions
+13 -8
View File
@@ -1,19 +1,24 @@
*.ini
*.yaml
#*.yaml
*.yml
*.md
*.json
*.toml
#*.json
*.xml
tests
node_modules
admin-frontend
#admin-frontend
data*
htmlcov
docker
dist
docs
.*
!pyproject.toml
!README.md
venv
.venv
/admin-frontend/node_modules
/LICENSE
/.editorconfig
/.gitattributes
/.prettierignore
/.prettierrc
/.devcontainer/
/.github/
+14 -3
View File
@@ -6,6 +6,10 @@ inputs:
description: Python version
required: false
default: "3.10"
install-deps:
description: Install dependencies
required: false
default: "false"
runs:
using: "composite"
@@ -14,7 +18,14 @@ runs:
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}
- name: Setup Poetry
uses: Gr1N/setup-poetry@v9
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v4
with:
poetry-version: "1.7.1"
version: "latest"
enable-cache: true
- name: Install dependencies
if: ${{ inputs.install-deps == 'true' }}
run: uv sync --frozen
shell: bash
+14 -24
View File
@@ -11,7 +11,7 @@ on:
- nonebot_bison/**
- tests/**
- pyproject.toml
- poetry.lock
- uv.lock
- docker.env.prod
- .github/**
pull_request:
@@ -21,7 +21,7 @@ on:
- nonebot_bison/**
- tests/**
- pyproject.toml
- poetry.lock
- uv.lock
- docker.env.prod
- .github/**
types:
@@ -72,12 +72,13 @@ jobs:
uses: ./.github/actions/setup-python
with:
python-version: ${{ matrix.python-version }}
install-deps: "true"
- name: Install prerequisites
run: poetry install
- name: Install playwright
run: uv run playwright install
- name: Run Pytest
run: poetry run pytest --cov-report xml --cov=./nonebot_bison -k 'not compare and not render' -n auto
run: uv run pytest --cov-report xml --cov=./nonebot_bison -k 'not compare and not render' -n auto
- name: Upload coverage report
uses: codecov/codecov-action@v5
@@ -106,12 +107,13 @@ jobs:
uses: ./.github/actions/setup-python
with:
python-version: ${{ matrix.python-version }}
install-deps: "true"
- name: Install prerequisites
run: poetry install
- name: Install playwright
run: uv run playwright install
- name: Run Pytest
run: poetry run pytest --cov-report xml --cov=./nonebot_bison -k 'not compare' -n auto
run: uv run pytest --cov-report xml --cov=./nonebot_bison -k 'not compare' -n auto
- name: Upload coverage report
uses: codecov/codecov-action@v5
@@ -127,12 +129,6 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Download frontend files
uses: actions/download-artifact@v4
with:
name: frontend
path: ./nonebot_bison/admin_page/dist
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
@@ -144,7 +140,7 @@ jobs:
if: github.event_name == 'pull_request'
with:
context: .
file: ./docker/Dockerfile_with_frontend
file: ./docker/Dockerfile
push: false
load: true
tags: felinae98/nonebot-bison:dummy
@@ -174,7 +170,7 @@ jobs:
if: github.event_name != 'pull_request'
with:
context: .
file: ./docker/Dockerfile_with_frontend
file: ./docker/Dockerfile
push: true
tags: felinae98/nonebot-bison:${{ env.GIT_BRANCH_NAME }}
cache-from: type=gha
@@ -188,12 +184,6 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Download frontend files
uses: actions/download-artifact@v4
with:
name: frontend
path: ./nonebot_bison/admin_page/dist
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
@@ -215,8 +205,8 @@ jobs:
uses: docker/build-push-action@v6
with:
context: .
file: ./docker/Dockerfile_with_frontend_sentry
file: ./docker/Dockerfile_without_browser
push: ${{ github.event_name != 'pull_request' }}
tags: felinae98/nonebot-bison:${{ env.GIT_BRANCH_NAME }}-sentry
tags: felinae98/nonebot-bison:${{ env.GIT_BRANCH_NAME }}-nobrowser
cache-from: type=gha
cache-to: type=gha,mode=max
+3 -33
View File
@@ -9,35 +9,12 @@ on:
type: string
jobs:
build-frontend:
name: Build Frontend
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build Frontend
uses: ./.github/actions/build-frontend
- name: Upload dist
uses: actions/upload-artifact@v4
with:
name: frontend
path: ./admin-frontend/build/
docker-main:
name: Docker main
runs-on: ubuntu-latest
needs: [build-frontend]
steps:
- uses: actions/checkout@v4
- name: Download frontend files
uses: actions/download-artifact@v4
with:
name: frontend
path: ./src/plugins/nonebot_bison/admin_page/dist
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
@@ -54,7 +31,7 @@ jobs:
uses: docker/build-push-action@v6
with:
context: .
file: ./docker/Dockerfile_with_frontend
file: ./docker/Dockerfile
push: true
tags: felinae98/nonebot-bison:${{ inputs.dockerTag }}
cache-from: type=gha
@@ -63,16 +40,9 @@ jobs:
docker-main-sentry:
name: Docker main sentry
runs-on: ubuntu-latest
needs: [build-frontend]
steps:
- uses: actions/checkout@v4
- name: Download frontend files
uses: actions/download-artifact@v4
with:
name: frontend
path: ./src/plugins/nonebot_bison/admin_page/dist
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
@@ -89,7 +59,7 @@ jobs:
uses: docker/build-push-action@v6
with:
context: .
file: ./docker/Dockerfile_with_frontend_sentry
tags: felinae98/nonebot-bison:${{ inputs.dockerTag }}-sentry
file: ./docker/Dockerfile_without_browser
tags: felinae98/nonebot-bison:${{ inputs.dockerTag }}-nobrowser
cache-from: type=gha
cache-to: type=gha,mode=max
+7 -4
View File
@@ -11,7 +11,7 @@ on:
- nonebot_bison/**
- tests/**
- pyproject.toml
- poetry.lock
- uv.lock
- docker.env.prod
- .github/**
pull_request:
@@ -21,7 +21,7 @@ on:
- nonebot_bison/**
- tests/**
- pyproject.toml
- poetry.lock
- uv.lock
- docker.env.prod
- .github/**
types:
@@ -55,7 +55,10 @@ jobs:
python-version: ${{ matrix.python-version }}
- name: Install prerequisites
run: poetry add pydantic@^1.10 && poetry install
run: uv add pydantic==1.10 && uv sync --frozen
- name: Install playwright
run: uv run playwright install
- name: Run Pytest
run: poetry run pytest -k 'not compare and not render' -n auto
run: uv run pytest -k 'not compare and not render' -n auto
+2 -2
View File
@@ -16,7 +16,7 @@ jobs:
- name: Setup Python environment
uses: ./.github/actions/setup-python
- run: echo "TAG_NAME=v$(poetry version -s)" >> $GITHUB_ENV
- run: echo "TAG_NAME=v$(uvx --from=toml-cli toml get --toml-path=pyproject.toml project.version)" >> $GITHUB_ENV
- name: Archive Changelog
uses: docker://ghcr.io/nonebot/auto-changelog:master
@@ -30,6 +30,6 @@ jobs:
git config user.name github-actions[bot]
git config user.email github-actions[bot]@users.noreply.github.com
git add .
git commit -m ":bookmark: Release $(poetry version -s)"
git commit -m ":bookmark: Release ${{ env.TAG_NAME }}"
git tag ${{ env.TAG_NAME }}
git push && git push --tags
+5 -12
View File
@@ -54,24 +54,17 @@ jobs:
- name: Publish PyPI and Github
run: |
poetry publish --build -u ${{ secrets.PYPI_USERNAME }} -p ${{ secrets.PYPI_PASSWORD }} || echo "Already pushed to pypi"
uv publish --build -u ${{ secrets.PYPI_USERNAME }} -p ${{ secrets.PYPI_PASSWORD }} || echo "Already pushed to pypi"
gh release upload --clobber ${{ env.TAG_NAME }} dist/*
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
publish-docker:
runs-on: ubuntu-latest
needs: build-frontend
steps:
- uses: actions/checkout@v4
- name: Download frontend files
uses: actions/download-artifact@v4
with:
name: frontend
path: ./nonebot_bison/admin_page/dist
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
@@ -92,7 +85,7 @@ jobs:
uses: docker/build-push-action@v6
with:
context: .
file: ./docker/Dockerfile_with_frontend
file: ./docker/Dockerfile
push: true
tags: |
felinae98/nonebot-bison:latest
@@ -104,10 +97,10 @@ jobs:
uses: docker/build-push-action@v6
with:
context: .
file: ./docker/Dockerfile_with_frontend_sentry
file: ./docker/Dockerfile_without_browser
push: true
tags: |
felinae98/nonebot-bison:${{ env.TAG_NAME }}-sentry
felinae98/nonebot-bison:sentry
felinae98/nonebot-bison:${{ env.TAG_NAME }}-nobrowser
felinae98/nonebot-bison:nobrowser
cache-from: type=gha
cache-to: type=gha,mode=max
+6 -1
View File
@@ -19,4 +19,9 @@ jobs:
- uses: actions/checkout@v4
- name: Run Ruff Lint
uses: chartboost/ruff-action@v1
uses: astral-sh/ruff-action@v2
with:
src: >-
nonebot_bison/
extra_plugins/
tests/
+3 -13
View File
@@ -7,22 +7,12 @@ ci:
autoupdate_commit_msg: ":arrow_up: auto update by pre-commit hooks"
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.8.1
rev: v0.8.2
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
args: [--fix]
stages: [pre-commit]
- repo: https://github.com/pycqa/isort
rev: 5.13.2
hooks:
- id: isort
stages: [pre-commit]
- repo: https://github.com/psf/black
rev: 24.10.0
hooks:
- id: black
- id: ruff-format
stages: [pre-commit]
- repo: https://github.com/pre-commit/mirrors-prettier
-3
View File
@@ -17,9 +17,6 @@ _✨ 通用订阅推送插件 ✨_
<img src="https://img.shields.io/pypi/v/nonebot-bison?logo=python&logoColor=edb641" alt="pypi">
</a>
<img src="https://img.shields.io/badge/python-3.10+-blue?logo=python&logoColor=edb641" alt="python">
<a href="https://github.com/psf/black">
<img src="https://img.shields.io/badge/code%20style-black-000000.svg?logo=python&logoColor=edb641" alt="black">
</a>
<a href="https://github.com/astral-sh/ruff">
<img src="https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v2.json" alt="ruff">
</a>
+190 -14
View File
@@ -1,16 +1,192 @@
FROM node:22.11.0 as frontend
ADD . /app
WORKDIR /app/admin-frontend
RUN npm install -g pnpm
RUN pnpm install && pnpm build
# syntax=docker/dockerfile:1.10
FROM python:3.12-slim AS metadata-stage
WORKDIR /tmp
ENV DEBIAN_FRONTEND=noninteractive \
PYTHONUNBUFFERED=1 \
TZ=Asia/Shanghai
RUN apt-get update && \
apt-get install -y --no-install-recommends \
curl \
python3-pip \
git \
&& pip install toml \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
COPY ./pyproject.toml ./
RUN echo "📋 Collecting metadata..." && \
export CREATED_DATE="$(date -u '+%Y-%m-%d %H:%M:%S UTC')" && \
export PYPROJECT_NAME=$(python3 -c 'import toml; print(toml.load("pyproject.toml")["project"]["name"])') && \
export DESCRIPTION=$(python3 -c 'import toml; print(toml.load("pyproject.toml")["project"]["description"])') && \
export AUTHOR=$(python3 -c 'import toml; print(toml.load("pyproject.toml")["project"]["authors"][0]["name"])') && \
echo "⏰ Created Date: $CREATED_DATE" && \
echo "📝 Project Name: $PYPROJECT_NAME" && \
echo "📄 Description: $DESCRIPTION" && \
echo "👤 Author: $AUTHOR" && \
echo "$CREATED_DATE" > /tmp/CREATED_DATE && \
echo "$PYPROJECT_NAME" > /tmp/PYPROJECT_NAME && \
echo "$DESCRIPTION" > /tmp/DESCRIPTION && \
echo "$AUTHOR" > /tmp/AUTHOR
RUN --mount=type=bind,source=./.git/,target=/tmp/.git/ \
echo "🏷️ Checking version information..." && \
VERSION=$(git describe --tags --exact-match 2>/dev/null || git rev-parse --short HEAD || echo "unknown") && \
echo "📌 Version: $VERSION" && \
echo "$VERSION" > /tmp/VERSION
LABEL org.opencontainers.image.created="$(cat /tmp/CREATED_DATE)" \
org.opencontainers.image.authors="$(cat /tmp/AUTHOR)" \
org.opencontainers.image.description="$(cat /tmp/DESCRIPTION)" \
org.opencontainers.image.name="$(cat /tmp/PYPROJECT_NAME)" \
org.opencontainers.image.version="$(cat /tmp/VERSION)"
FROM node:20-slim AS frontend-stage
WORKDIR /tmp/admin-frontend
ENV TZ=Asia/Shanghai \
PNPM_HOME="/pnpm" \
PATH="$PNPM_HOME:$PATH"
COPY ./admin-frontend/package.json ./admin-frontend/pnpm-lock.yaml ./
RUN corepack enable && \
corepack prepare pnpm@9.15.1 --activate && \
echo "🔔 Verifying Node.js installation..." && \
if NODE_VERSION=$(node --version 2>&1); then \
echo "✅ Node.js version ${NODE_VERSION} installed successfully"; \
else \
echo "❌ Node.js installation failed" && exit 1; \
fi && \
echo "🔔 Verifying pnpm installation..." && \
if PNPM_VERSION=$(pnpm --version 2>&1); then \
echo "✅ pnpm version ${PNPM_VERSION} installed successfully"; \
else \
echo "❌ pnpm installation failed" && exit 1; \
fi
RUN pnpm install --frozen-lockfile
COPY ./admin-frontend .
RUN mkdir -p ../nonebot_bison/admin_page/dist
RUN echo "🏗️ Starting frontend build..." && \
pnpm run build && \
echo "✅ Frontend build completed"
FROM python:3.12-slim AS playwright-stage
WORKDIR /tmp
ENV DEBIAN_FRONTEND=noninteractive \
PYTHONUNBUFFERED=1 \
TZ=Asia/Shanghai
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
RUN echo "📦 Installing playwright..." && \
uvx playwright install --with-deps chromium && \
echo "✅ Playwright installed successfully"
FROM python:3.12-slim AS production
FROM python:3.11
RUN python3 -m pip install poetry && poetry config virtualenvs.create false
WORKDIR /app
COPY ./README.md ./pyproject.toml ./poetry.lock* /app/
RUN poetry install --only=main,docker
ADD src /app/src
ADD bot.py /app/
COPY --from=frontend /app/nonebot_bison/admin_page/dist /app/nonebot_bison/admin_page/dist
ENV HOST=0.0.0.0
CMD ["nb", "run"]
#ENV UV_INDEX_URL=https://mirrors.aliyun.com/pypi/simple/
ENV DEBIAN_FRONTEND=noninteractive \
UV_COMPILE_BYTECODE=1 \
UV_SYSTEM_PYTHON=true \
PYTHONUNBUFFERED=1 \
UV_LINK_MODE=copy \
TZ=Asia/Shanghai \
SHELL="/bin/bash"
RUN echo "📦 Installing system dependencies..." && \
apt-get update && \
apt-get install -y --no-install-recommends \
curl \
xvfb \
fonts-noto-color-emoji \
fonts-unifont \
libfontconfig1 \
libfreetype6 \
xfonts-scalable \
fonts-liberation \
fonts-ipafont-gothic \
fonts-wqy-zenhei \
fonts-tlwg-loma-otf \
at-spi2-common \
fonts-freefont-ttf \
libasound2 \
libasound2-data \
libatk-bridge2.0-0 \
libatk1.0-0 \
libatspi2.0-0 \
libavahi-client3 \
libavahi-common-data \
libavahi-common3 \
libcairo2 \
libcups2 \
libdatrie1 \
libdbus-1-3 \
libfribidi0 \
libgbm1 \
libglib2.0-0 \
libgraphite2-3 \
libharfbuzz0b \
libnspr4 \
libnss3 \
libpango-1.0-0 \
libthai-data \
libthai0 \
libwayland-server0 \
libxcb-render0 \
libxcomposite1 \
libxdamage1 \
libxi6 \
libxkbcommon0 && \
fc-cache -fv && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* && \
echo "✅ System dependencies installed successfully"
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
RUN echo "🔔 Verifying uv installation..." && \
if UV_VERSION=$(uv --version 2>&1); then \
echo "✅ uv version ${UV_VERSION} installed successfully"; \
else \
echo "❌ uv installation failed" && exit 1; \
fi
COPY ./pyproject.toml ./uv.lock ./
RUN --mount=type=cache,target=/root/.cache/ \
--mount=type=cache,target=/root/.uv \
echo "📦 Installing bison dependencies..." && \
uv pip install --extra=docker --requirement pyproject.toml --no-cache && \
echo "✅ Dependencies installed successfully" && \
echo "🔔 Cleaning up uv..." && \
rm -rf /bin/uv /bin/uvx && \
echo "✅ UV cleanup completed"
COPY --from=metadata-stage /tmp/VERSION /app/VERSION
COPY nonebot_bison/ /app/nonebot_bison/
COPY --from=frontend-stage /tmp/nonebot_bison/admin_page/dist ./nonebot_bison/admin_page/dist
COPY --from=playwright-stage /root/.cache/ms-playwright /root/.cache/ms-playwright
EXPOSE 8080
RUN echo '#!/bin/bash' > /app/start.sh && \
echo 'echo "📌 Current bison Version: $(cat /app/VERSION)"' >> /app/start.sh && \
echo 'echo "🚀 Starting service..."' >> /app/start.sh && \
echo 'nb run' >> /app/start.sh && \
chmod +x /app/start.sh
CMD ["/app/start.sh"]
-10
View File
@@ -1,10 +0,0 @@
FROM python:3.11
RUN apt-get update && apt-get install -y fonts-wqy-microhei chromium nano
RUN python3 -m pip config set global.index-url https://mirrors.aliyun.com/pypi/simple
RUN python3 -m pip install poetry && poetry config virtualenvs.create false
WORKDIR /app
COPY ./README.md ./pyproject.toml ./poetry.lock* /app/
RUN poetry install --only=main,docker
ENV BISON_BROWSER=local:/usr/bin/chromium
ADD src /app/src
ENV HOST=0.0.0.0
-9
View File
@@ -1,9 +0,0 @@
FROM python:3.11
RUN python3 -m pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
RUN python3 -m pip install poetry && poetry config virtualenvs.create false
WORKDIR /app
COPY ./README.md ./pyproject.toml ./poetry.lock* /app/
RUN poetry install --only=main,docker
ADD src /app/src
ENV HOST=0.0.0.0
CMD ["nb", "run"]
-50
View File
@@ -1,50 +0,0 @@
# syntax=docker/dockerfile:1.10
FROM python:3.11-slim-bullseye as base
FROM base as builder
ENV PYTHONFAULTHANDLER=1 \
PYTHONUNBUFFERED=1 \
PYTHONHASHSEED=random \
PIP_NO_CACHE_DIR=off \
PIP_DISABLE_PIP_VERSION_CHECK=on \
PIP_DEFAULT_TIMEOUT=100 \
POETRY_NO_INTERACTION=1 \
POETRY_VIRTUALENVS_CREATE=false \
PATH="$PATH:/runtime/bin" \
PYTHONPATH="$PYTHONPATH:/runtime/lib/python3.10/site-packages" \
# Versions:
POETRY_VERSION=1.7.1
RUN apt-get update && apt-get install -y build-essential unzip wget python-dev git
RUN pip install "poetry==$POETRY_VERSION"
WORKDIR /src
COPY README.md pyproject.toml poetry.lock /src/
RUN poetry export --only=main,docker --without-hashes --no-interaction --no-ansi -f requirements.txt -o requirements.txt
RUN pip install --prefix=/runtime --force-reinstall -r requirements.txt
FROM base as runtime
WORKDIR /app
RUN --mount=type=cache,target=/var/cache/apt \
--mount=type=cache,target=/var/lib/apt \
apt-get update && apt-get install -y xvfb fonts-noto-color-emoji ttf-unifont \
libfontconfig1 libfreetype6 xfonts-cyrillic xfonts-scalable fonts-liberation \
fonts-ipafont-gothic fonts-wqy-zenhei fonts-tlwg-loma-otf \
fonts-liberation libasound2 libatk-bridge2.0-0 libatk1.0-0 libatspi2.0-0 \
libcairo2 libcups2 libdbus-1-3 libdrm2 libegl1 libgbm1 libglib2.0-0 libgtk-3-0 \
libnspr4 libnss3 libpango-1.0-0 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 \
libxdamage1 libxext6 libxfixes3 libxrandr2 libxshmfence1 nano
COPY --from=builder /runtime /usr/local
ADD . /app/
RUN pip install -e . && playwright install chromium
RUN mv docker.env.prod .env.prod && \
nb adapter install nonebot-adapter-red && \
nb adapter install nonebot-adapter-qq
ENV HOST=0.0.0.0
CMD ["nb", "run"]
# vim: ft=dockerfile
-51
View File
@@ -1,51 +0,0 @@
# syntax=docker/dockerfile:1.10
FROM python:3.11-slim-bullseye as base
FROM base as builder
ENV PYTHONFAULTHANDLER=1 \
PYTHONUNBUFFERED=1 \
PYTHONHASHSEED=random \
PIP_NO_CACHE_DIR=off \
PIP_DISABLE_PIP_VERSION_CHECK=on \
PIP_DEFAULT_TIMEOUT=100 \
POETRY_NO_INTERACTION=1 \
POETRY_VIRTUALENVS_CREATE=false \
PATH="$PATH:/runtime/bin" \
PYTHONPATH="$PYTHONPATH:/runtime/lib/python3.10/site-packages" \
# Versions:
POETRY_VERSION=1.7.1
RUN apt-get update && apt-get install -y build-essential unzip wget python3-dev git
RUN pip install "poetry==$POETRY_VERSION"
WORKDIR /src
COPY pyproject.toml poetry.lock /src/
RUN poetry export --only=main,docker --without-hashes --no-interaction --no-ansi -f requirements.txt -o requirements.txt
RUN pip install --prefix=/runtime --force-reinstall -r requirements.txt
FROM base as runtime
WORKDIR /app
RUN --mount=type=cache,target=/var/cache/apt \
--mount=type=cache,target=/var/lib/apt \
apt-get update && apt-get install -y xvfb fonts-noto-color-emoji ttf-unifont \
libfontconfig1 libfreetype6 xfonts-cyrillic xfonts-scalable fonts-liberation \
fonts-ipafont-gothic fonts-wqy-zenhei fonts-tlwg-loma-otf \
fonts-liberation libasound2 libatk-bridge2.0-0 libatk1.0-0 libatspi2.0-0 \
libcairo2 libcups2 libdbus-1-3 libdrm2 libegl1 libgbm1 libglib2.0-0 libgtk-3-0 \
libnspr4 libnss3 libpango-1.0-0 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 \
libxdamage1 libxext6 libxfixes3 libxrandr2 libxshmfence1 nano
COPY --from=builder /runtime /usr/local
ADD . /app/
RUN pip install -e . && playwright install chromium
RUN mv docker.env.prod .env.prod && \
nb adapter install nonebot-adapter-red && \
nb adapter install nonebot-adapter-qq && \
nb plugin install nonebot-plugin-sentry
ENV HOST=0.0.0.0
CMD ["nb", "run"]
# vim: ft=dockerfile
+145
View File
@@ -0,0 +1,145 @@
# syntax=docker/dockerfile:1.10
FROM python:3.12-slim AS metadata-stage
WORKDIR /tmp
ENV DEBIAN_FRONTEND=noninteractive \
PYTHONUNBUFFERED=1 \
TZ=Asia/Shanghai
RUN apt-get update && \
apt-get install -y --no-install-recommends \
curl \
python3-pip \
git \
&& pip install toml \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
COPY ./pyproject.toml ./
RUN echo "📋 Collecting metadata..." && \
export CREATED_DATE="$(date -u '+%Y-%m-%d %H:%M:%S UTC')" && \
export PYPROJECT_NAME=$(python3 -c 'import toml; print(toml.load("pyproject.toml")["project"]["name"])') && \
export DESCRIPTION=$(python3 -c 'import toml; print(toml.load("pyproject.toml")["project"]["description"])') && \
export AUTHOR=$(python3 -c 'import toml; print(toml.load("pyproject.toml")["project"]["authors"][0]["name"])') && \
echo "⏰ Created Date: $CREATED_DATE" && \
echo "📝 Project Name: $PYPROJECT_NAME" && \
echo "📄 Description: $DESCRIPTION" && \
echo "👤 Author: $AUTHOR" && \
echo "$CREATED_DATE" > /tmp/CREATED_DATE && \
echo "$PYPROJECT_NAME" > /tmp/PYPROJECT_NAME && \
echo "$DESCRIPTION" > /tmp/DESCRIPTION && \
echo "$AUTHOR" > /tmp/AUTHOR
RUN --mount=type=bind,source=./.git/,target=/tmp/.git/ \
echo "🏷️ Checking version information..." && \
VERSION=$(git describe --tags --exact-match 2>/dev/null || git rev-parse --short HEAD || echo "unknown") && \
echo "📌 Version: $VERSION" && \
echo "$VERSION" > /tmp/VERSION
LABEL org.opencontainers.image.created="$(cat /tmp/CREATED_DATE)" \
org.opencontainers.image.authors="$(cat /tmp/AUTHOR)" \
org.opencontainers.image.description="$(cat /tmp/DESCRIPTION)" \
org.opencontainers.image.name="$(cat /tmp/PYPROJECT_NAME)" \
org.opencontainers.image.version="$(cat /tmp/VERSION)"
FROM node:20-slim AS frontend-stage
WORKDIR /tmp/admin-frontend
ENV TZ=Asia/Shanghai \
PNPM_HOME="/pnpm" \
PATH="$PNPM_HOME:$PATH"
COPY ./admin-frontend/package.json ./admin-frontend/pnpm-lock.yaml ./
RUN corepack enable && \
corepack prepare pnpm@9.15.1 --activate && \
echo "🔔 Verifying Node.js installation..." && \
if NODE_VERSION=$(node --version 2>&1); then \
echo "✅ Node.js version ${NODE_VERSION} installed successfully"; \
else \
echo "❌ Node.js installation failed" && exit 1; \
fi && \
echo "🔔 Verifying pnpm installation..." && \
if PNPM_VERSION=$(pnpm --version 2>&1); then \
echo "✅ pnpm version ${PNPM_VERSION} installed successfully"; \
else \
echo "❌ pnpm installation failed" && exit 1; \
fi
RUN pnpm install --frozen-lockfile
COPY ./admin-frontend .
RUN mkdir -p ../nonebot_bison/admin_page/dist
RUN echo "🏗️ Starting frontend build..." && \
pnpm run build && \
echo "✅ Frontend build completed"
FROM python:3.12-slim AS playwright-stage
WORKDIR /tmp
ENV DEBIAN_FRONTEND=noninteractive \
PYTHONUNBUFFERED=1 \
TZ=Asia/Shanghai
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
RUN echo "📦 Installing playwright..." && \
uvx playwright install --with-deps chromium && \
echo "✅ Playwright installed successfully"
FROM python:3.12-slim AS production
WORKDIR /app
ENV DEBIAN_FRONTEND=noninteractive \
UV_COMPILE_BYTECODE=1 \
UV_SYSTEM_PYTHON=true \
PYTHONUNBUFFERED=1 \
UV_LINK_MODE=copy \
TZ=Asia/Shanghai \
SHELL="/bin/bash"
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
RUN echo "🔔 Verifying uv installation..." && \
if UV_VERSION=$(uv --version 2>&1); then \
echo "✅ uv version ${UV_VERSION} installed successfully"; \
else \
echo "❌ uv installation failed" && exit 1; \
fi
COPY ./pyproject.toml ./uv.lock ./
COPY ./docker.env.prod ./.env
RUN --mount=type=cache,target=/root/.cache/ \
--mount=type=cache,target=/root/.uv \
echo "📦 Installing bison dependencies..." && \
uv pip install --extra=docker --requirement pyproject.toml --no-cache && \
echo "✅ Dependencies installed successfully" && \
echo "🔔 Cleaning up uv..." && \
rm -rf /bin/uv /bin/uvx && \
echo "✅ UV cleanup completed"
COPY --from=metadata-stage /tmp/VERSION /app/VERSION
COPY nonebot_bison/ /app/nonebot_bison/
COPY --from=frontend-stage /tmp/nonebot_bison/admin_page/dist ./nonebot_bison/admin_page/dist
EXPOSE 8080
ENV BISON_SKIP_BROWSER_CHECK=True \
BISON_USE_BROWSER=False
RUN echo '#!/bin/bash' > /app/start.sh && \
echo 'echo "📌 Current bison Version: $(cat /app/VERSION)"' >> /app/start.sh && \
echo 'echo "🚀 Starting service..."' >> /app/start.sh && \
echo 'nb run' >> /app/start.sh && \
chmod +x /app/start.sh
CMD ["/app/start.sh"]
+2 -2
View File
@@ -1,7 +1,7 @@
from nonebot import on_request
from nonebot.log import logger
from nonebot.adapters.onebot.v11 import Bot
from nonebot.adapters.onebot.v11.event import GroupRequestEvent, FriendRequestEvent
from nonebot.adapters.onebot.v11.event import FriendRequestEvent, GroupRequestEvent
from nonebot.log import logger
friend_req = on_request(priority=5)
+4 -4
View File
@@ -6,8 +6,8 @@ require("nonebot_plugin_saa")
import nonebot_plugin_saa
from . import admin_page, bootstrap, config, platform, post, scheduler, send, sub_manager, theme, types, utils
from .plugin_config import PlugConfig, plugin_config
from . import post, send, theme, types, utils, config, platform, bootstrap, scheduler, admin_page, sub_manager
__help__version__ = "0.8.2"
nonebot_plugin_saa.enable_auto_select_bot()
@@ -36,12 +36,12 @@ __all__ = [
"admin_page",
"bootstrap",
"config",
"sub_manager",
"platform",
"post",
"scheduler",
"send",
"platform",
"sub_manager",
"theme",
"types",
"utils",
"theme",
]
+6 -7
View File
@@ -2,17 +2,17 @@ import os
from pathlib import Path
from typing import TYPE_CHECKING
from nonebot.log import logger
from nonebot.rule import to_me
from nonebot.typing import T_State
from nonebot import get_driver, on_command
from nonebot.adapters.onebot.v11 import Bot
from nonebot.adapters.onebot.v11.event import PrivateMessageEvent
from nonebot.log import logger
from nonebot.rule import to_me
from nonebot.typing import T_State
from nonebot_bison.plugin_config import plugin_config
from .api import router as api_router
from ..plugin_config import plugin_config
from .token_manager import token_manager as tm
from ..metrics import metrics_router as metrics_router
if TYPE_CHECKING:
from nonebot.drivers.fastapi import Driver
@@ -22,9 +22,9 @@ STATIC_PATH = (Path(__file__).parent / "dist").resolve()
def init_fastapi(driver: "Driver"):
import socketio
from fastapi.applications import FastAPI
from fastapi.staticfiles import StaticFiles
import socketio
sio = socketio.AsyncServer(async_mode="asgi", cors_allowed_origins="*")
socket_app = socketio.ASGIApp(sio, socketio_path="socket")
@@ -47,7 +47,6 @@ def init_fastapi(driver: "Driver"):
description="nonebot-bison webui and api",
)
nonebot_app.include_router(api_router)
nonebot_app.include_router(metrics_router)
nonebot_app.mount("/", SinglePageApplication(directory=static_path), name="bison-frontend")
app = driver.server_app
+20 -19
View File
@@ -1,38 +1,39 @@
from typing import cast
import nonebot
from fastapi import status
from fastapi.routing import APIRouter
from fastapi.param_functions import Depends
from fastapi.exceptions import HTTPException
from fastapi.param_functions import Depends
from fastapi.routing import APIRouter
from fastapi.security.oauth2 import OAuth2PasswordBearer
import nonebot
from nonebot_plugin_saa import TargetQQGroup
from nonebot_plugin_saa.auto_select_bot import get_bot
from fastapi.security.oauth2 import OAuth2PasswordBearer
from ..types import WeightConfig
from ..apis import check_sub_target
from nonebot_bison.apis import check_sub_target
from nonebot_bison.config import NoSuchSubscribeException, NoSuchTargetException, NoSuchUserException, config
from nonebot_bison.config.db_config import SubscribeDupException
from nonebot_bison.platform import platform_manager
from nonebot_bison.scheduler import scheduler_dict
from nonebot_bison.types import Target as T_Target
from nonebot_bison.types import WeightConfig
from nonebot_bison.utils.get_bot import get_groups
from nonebot_bison.utils.site import CookieClientManager, is_cookie_client_manager, site_manager
from .jwt import load_jwt, pack_jwt
from ..scheduler import scheduler_dict
from ..types import Target as T_Target
from ..utils.get_bot import get_groups
from ..platform import platform_manager
from .token_manager import token_manager
from ..config.db_config import SubscribeDupException
from ..utils.site import CookieClientManager, site_manager, is_cookie_client_manager
from ..config import NoSuchUserException, NoSuchTargetException, NoSuchSubscribeException, config
from .types import (
AddSubscribeReq,
Cookie,
Target,
TokenResp,
CookieTarget,
GlobalConf,
PlatformConfig,
SiteConfig,
StatusResp,
CookieTarget,
SubscribeResp,
PlatformConfig,
AddSubscribeReq,
SubscribeConfig,
SubscribeGroupDetail,
SubscribeResp,
Target,
TokenResp,
)
router = APIRouter(prefix="/api", tags=["api"])
+1 -1
View File
@@ -1,6 +1,6 @@
import datetime
import random
import string
import datetime
import jwt
+1 -1
View File
@@ -1,6 +1,6 @@
from datetime import timedelta
import random
import string
from datetime import timedelta
from expiringdictx import ExpiringDict
+1 -1
View File
@@ -60,8 +60,8 @@ class StatusResp(BaseModel):
msg: str
from typing import Any
from datetime import datetime
from typing import Any
from pydantic import BaseModel
+2 -2
View File
@@ -1,6 +1,6 @@
from .types import Target
from .scheduler import scheduler_dict
from .platform import platform_manager
from .scheduler import scheduler_dict
from .types import Target
async def check_sub_target(platform_name: str, target: Target):
+3 -3
View File
@@ -1,10 +1,10 @@
from nonebot.log import logger
from sqlalchemy import text, inspect
from nonebot_plugin_datastore.db import get_engine, pre_db_init, post_db_init
from nonebot_plugin_datastore.db import get_engine, post_db_init, pre_db_init
from sqlalchemy import inspect, text
from .config.config_legacy import start_up as legacy_db_startup
from .config.db_migration import data_migrate
from .scheduler.manager import init_scheduler
from .config.config_legacy import start_up as legacy_db_startup
@pre_db_init
+2 -2
View File
@@ -1,9 +1,9 @@
from typing import Literal, overload
from pydantic import BaseModel
from nonebot.compat import PYDANTIC_V2
from pydantic import BaseModel
__all__ = ("model_validator", "model_rebuild")
__all__ = ("model_rebuild", "model_validator")
if PYDANTIC_V2:
+2 -2
View File
@@ -1,4 +1,4 @@
from .db_config import config as config
from .utils import NoSuchUserException as NoSuchUserException
from .utils import NoSuchTargetException as NoSuchTargetException
from .utils import NoSuchSubscribeException as NoSuchSubscribeException
from .utils import NoSuchTargetException as NoSuchTargetException
from .utils import NoSuchUserException as NoSuchUserException
+9 -8
View File
@@ -1,19 +1,20 @@
import os
from collections import defaultdict
from datetime import datetime
import json
import os
from os import path
from pathlib import Path
from datetime import datetime
from collections import defaultdict
from typing import Literal, TypedDict
from nonebot.log import logger
from tinydb import Query, TinyDB
from ..utils import Singleton
from ..types import User, Target
from ..platform import platform_manager
from ..plugin_config import plugin_config
from .utils import NoSuchUserException, NoSuchSubscribeException
from nonebot_bison.platform import platform_manager
from nonebot_bison.plugin_config import plugin_config
from nonebot_bison.types import Target, User
from nonebot_bison.utils import Singleton
from .utils import NoSuchSubscribeException, NoSuchUserException
supported_target_type = platform_manager.keys()
+11 -11
View File
@@ -1,20 +1,20 @@
import asyncio
from collections import defaultdict
from datetime import time, datetime
from collections.abc import Callable, Sequence, Awaitable
from collections.abc import Awaitable, Callable, Sequence
from datetime import datetime, time
from nonebot.compat import model_dump
from sqlalchemy.orm import selectinload
from sqlalchemy.exc import IntegrityError
from sqlalchemy import func, delete, select
from nonebot_plugin_saa import PlatformTarget
from nonebot_plugin_datastore import create_session
from nonebot_plugin_saa import PlatformTarget
from sqlalchemy import delete, func, select
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import selectinload
from ..types import Tag
from ..types import Target as T_Target
from .utils import NoSuchTargetException, DuplicateCookieTargetException
from .db_model import User, Cookie, Target, Subscribe, CookieTarget, ScheduleTimeWeight
from ..types import Category, UserSubInfo, WeightConfig, TimeWeightConfig, PlatformWeightConfigResp
from nonebot_bison.types import Category, PlatformWeightConfigResp, Tag, TimeWeightConfig, UserSubInfo, WeightConfig
from nonebot_bison.types import Target as T_Target
from .db_model import Cookie, CookieTarget, ScheduleTimeWeight, Subscribe, Target, User
from .utils import DuplicateCookieTargetException, NoSuchTargetException
def _get_time():
+3 -3
View File
@@ -1,11 +1,11 @@
from nonebot.log import logger
from nonebot.compat import model_dump
from nonebot.log import logger
from nonebot_plugin_datastore.db import get_engine
from sqlalchemy.ext.asyncio.session import AsyncSession
from nonebot_plugin_saa import TargetQQGroup, TargetQQPrivate
from sqlalchemy.ext.asyncio.session import AsyncSession
from .db_model import User, Target, Subscribe
from .config_legacy import Config, ConfigContent, drop
from .db_model import Subscribe, Target, User
async def data_migrate():
+6 -6
View File
@@ -1,15 +1,15 @@
import datetime
from typing import Any
from pathlib import Path
from typing import Any
from nonebot_plugin_saa import PlatformTarget
from sqlalchemy.dialects.postgresql import JSONB
from nonebot.compat import PYDANTIC_V2, ConfigDict
from nonebot_plugin_datastore import get_plugin_data
from sqlalchemy.orm import Mapped, relationship, mapped_column
from sqlalchemy import JSON, String, DateTime, ForeignKey, UniqueConstraint
from nonebot_plugin_saa import PlatformTarget
from sqlalchemy import JSON, DateTime, ForeignKey, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column, relationship
from ..types import Tag, Category
from nonebot_bison.types import Category, Tag
Model = get_plugin_data().Model
get_plugin_data().set_migration_dir(Path(__file__).parent / "migrations")
@@ -6,8 +6,8 @@ Create Date: 2022-03-21 19:18:13.762626
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision = "0571870f5222"
@@ -6,8 +6,8 @@ Create Date: 2022-03-27 21:50:10.911649
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision = "4a46ba54a3f3"
@@ -6,8 +6,8 @@ Create Date: 2022-05-31 22:05:13.235981
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision = "5f3370328e44"
@@ -6,8 +6,8 @@ Create Date: 2023-03-20 00:39:30.199915
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects.postgresql import JSONB
# revision identifiers, used by Alembic.
@@ -6,8 +6,8 @@ Create Date: 2023-03-20 11:08:42.883556
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects.postgresql import JSONB
# revision identifiers, used by Alembic.
@@ -6,8 +6,8 @@ Create Date: 2023-03-20 15:38:20.220599
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision = "8d3863e9d74b"
@@ -6,8 +6,8 @@ Create Date: 2022-03-29 21:01:38.213153
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision = "a333d6224193"
@@ -6,10 +6,10 @@ Create Date: 2023-03-20 01:14:42.623789
"""
import sqlalchemy as sa
from alembic import op
from sqlalchemy.orm import Session
import sqlalchemy as sa
from sqlalchemy.ext.automap import automap_base
from sqlalchemy.orm import Session
# revision identifiers, used by Alembic.
revision = "a5466912fad0"
@@ -6,8 +6,8 @@ Create Date: 2023-03-09 19:10:42.168133
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision = "aceef470d69c"
@@ -6,11 +6,11 @@ Create Date: 2023-03-02 14:04:16.492133
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
from sqlalchemy import select
from sqlalchemy.orm import Session
from sqlalchemy.ext.automap import automap_base
from sqlalchemy.orm import Session
# revision identifiers, used by Alembic.
revision = "bd92923c218f"
@@ -6,8 +6,8 @@ Create Date: 2024-09-23 10:03:30.593263
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
from sqlalchemy import Text
from sqlalchemy.dialects import postgresql
@@ -6,10 +6,10 @@ Create Date: 2023-08-25 00:20:51.511329
"""
import sqlalchemy as sa
from alembic import op
from sqlalchemy.orm import Session
import sqlalchemy as sa
from sqlalchemy.ext.automap import automap_base
from sqlalchemy.orm import Session
# revision identifiers, used by Alembic.
revision = "f9baef347cc8"
@@ -3,4 +3,4 @@
from . import v1, v2, v3
from .base import NBESFBase
__all__ = ["v1", "v2", "v3", "NBESFBase"]
__all__ = ["NBESFBase", "v1", "v2", "v3"]
@@ -1,10 +1,10 @@
from abc import ABC
from pydantic import BaseModel
from nonebot.compat import PYDANTIC_V2, ConfigDict
from nonebot_plugin_saa.registries import AllSupportedPlatformTarget as UserInfo
from pydantic import BaseModel
from ....types import Tag, Category
from nonebot_bison.types import Category, Tag
class NBESFBase(BaseModel, ABC):
+12 -11
View File
@@ -1,17 +1,18 @@
"""nbesf is Nonebot Bison Enchangable Subscribes File! ver.1"""
from typing import Any
from functools import partial
from typing import Any
from nonebot.log import logger
from pydantic import BaseModel
from nonebot_plugin_saa import TargetQQGroup, TargetQQPrivate
from nonebot.compat import PYDANTIC_V2, ConfigDict, model_dump, type_validate_json, type_validate_python
from nonebot.log import logger
from nonebot_plugin_saa import TargetQQGroup, TargetQQPrivate
from pydantic import BaseModel, Field
from nonebot_bison.config.db_config import SubscribeDupException, config
from nonebot_bison.config.subs_io.utils import NBESFParseErr
from nonebot_bison.types import Category, Tag
from ..utils import NBESFParseErr
from ....types import Tag, Category
from .base import NBESFBase, SubReceipt
from ...db_config import SubscribeDupException, config
# ===== nbesf 定义格式 ====== #
NBESF_VERSION = 1
@@ -79,7 +80,7 @@ class SubGroup(
"""
version: int = NBESF_VERSION
groups: list[SubPack] = []
groups: list[SubPack] = Field(default_factory=list)
# ======================= #
@@ -108,11 +109,11 @@ async def subs_receipt_gen(nbesf_data: SubGroup):
try:
await config.add_subscribe(receipt.user, **model_dump(receipt, exclude={"user"}))
except SubscribeDupException:
logger.warning(f"!添加订阅条目 {repr(receipt)} 失败: 相同的订阅已存在")
logger.warning(f"!添加订阅条目 {receipt!r} 失败: 相同的订阅已存在")
except Exception as e:
logger.error(f"!添加订阅条目 {repr(receipt)} 失败: {repr(e)}")
logger.error(f"!添加订阅条目 {receipt!r} 失败: {e!r}")
else:
logger.success(f"添加订阅条目 {repr(receipt)} 成功!")
logger.success(f"添加订阅条目 {receipt!r} 成功!")
def nbesf_parser(raw_data: Any) -> SubGroup:
+12 -11
View File
@@ -1,17 +1,18 @@
"""nbesf is Nonebot Bison Enchangable Subscribes File! ver.2"""
from typing import Any
from functools import partial
from typing import Any
from nonebot.log import logger
from pydantic import BaseModel
from nonebot_plugin_saa.registries import AllSupportedPlatformTarget
from nonebot.compat import PYDANTIC_V2, ConfigDict, model_dump, type_validate_json, type_validate_python
from nonebot.log import logger
from nonebot_plugin_saa.registries import AllSupportedPlatformTarget
from pydantic import BaseModel, Field
from nonebot_bison.config.db_config import SubscribeDupException, config
from nonebot_bison.config.subs_io.utils import NBESFParseErr
from nonebot_bison.types import Category, Tag
from ..utils import NBESFParseErr
from ....types import Tag, Category
from .base import NBESFBase, SubReceipt
from ...db_config import SubscribeDupException, config
# ===== nbesf 定义格式 ====== #
NBESF_VERSION = 2
@@ -64,7 +65,7 @@ class SubGroup(NBESFBase):
"""
version: int = NBESF_VERSION
groups: list[SubPack] = []
groups: list[SubPack] = Field(default_factory=list)
# ======================= #
@@ -85,11 +86,11 @@ async def subs_receipt_gen(nbesf_data: SubGroup):
try:
await config.add_subscribe(receipt.user, **model_dump(receipt, exclude={"user"}))
except SubscribeDupException:
logger.warning(f"!添加订阅条目 {repr(receipt)} 失败: 相同的订阅已存在")
logger.warning(f"!添加订阅条目 {receipt!r} 失败: 相同的订阅已存在")
except Exception as e:
logger.error(f"!添加订阅条目 {repr(receipt)} 失败: {repr(e)}")
logger.error(f"!添加订阅条目 {receipt!r} 失败: {e!r}")
else:
logger.success(f"添加订阅条目 {repr(receipt)} 成功!")
logger.success(f"添加订阅条目 {receipt!r} 成功!")
def nbesf_parser(raw_data: Any) -> SubGroup:
+15 -16
View File
@@ -1,21 +1,20 @@
"""nbesf is Nonebot Bison Enchangable Subscribes File! ver.2"""
from typing import Any
from functools import partial
from typing import Any
from nonebot.log import logger
from pydantic import BaseModel
from nonebot_plugin_saa.registries import AllSupportedPlatformTarget
from nonebot.compat import PYDANTIC_V2, ConfigDict, model_dump, type_validate_json, type_validate_python
from nonebot.log import logger
from nonebot_plugin_saa.registries import AllSupportedPlatformTarget
from pydantic import BaseModel, Field
from nonebot_bison.types import Tag
from nonebot_bison.types import Category
from nonebot_bison.config.db_config import SubscribeDupException, config
from nonebot_bison.config.db_model import Cookie as DBCookie
from nonebot_bison.config.subs_io.utils import NBESFParseErr
from nonebot_bison.types import Category, Tag
from nonebot_bison.types import Target as T_Target
from ..utils import NBESFParseErr
from .base import NBESFBase, SubReceipt
from ...db_model import Cookie as DBCookie
from ...db_config import SubscribeDupException, config
# ===== nbesf 定义格式 ====== #
NBESF_VERSION = 3
@@ -80,8 +79,8 @@ class SubGroup(NBESFBase):
"""
version: int = NBESF_VERSION
groups: list[SubPack] = []
cookies: list[Cookie] = []
groups: list[SubPack] = Field(default_factory=list)
cookies: list[Cookie] = Field(default_factory=list)
# ======================= #
@@ -103,11 +102,11 @@ async def subs_receipt_gen(nbesf_data: SubGroup):
try:
await config.add_subscribe(receipt.user, **model_dump(receipt, exclude={"user"}))
except SubscribeDupException:
logger.warning(f"!添加订阅条目 {repr(receipt)} 失败: 相同的订阅已存在")
logger.warning(f"!添加订阅条目 {receipt!r} 失败: 相同的订阅已存在")
except Exception as e:
logger.error(f"!添加订阅条目 {repr(receipt)} 失败: {repr(e)}")
logger.error(f"!添加订阅条目 {receipt!r} 失败: {e!r}")
else:
logger.success(f"添加订阅条目 {repr(receipt)} 成功!")
logger.success(f"添加订阅条目 {receipt!r} 成功!")
async def magic_cookie_gen(nbesf_data: SubGroup):
@@ -119,9 +118,9 @@ async def magic_cookie_gen(nbesf_data: SubGroup):
for target in cookie.targets:
await config.add_cookie_target(T_Target(target.target), target.platform_name, cookie_id)
except Exception as e:
logger.error(f"!添加 Cookie 条目 {repr(cookie)} 失败: {repr(e)}")
logger.error(f"!添加 Cookie 条目 {cookie!r} 失败: {e!r}")
else:
logger.success(f"添加 Cookie 条目 {repr(cookie)} 成功!")
logger.success(f"添加 Cookie 条目 {cookie!r} 成功!")
def nbesf_parser(raw_data: Any) -> SubGroup:
+9 -8
View File
@@ -1,19 +1,20 @@
from typing import cast
from collections import defaultdict
from collections.abc import Callable
from typing import cast
from sqlalchemy import select
from nonebot.log import logger
from sqlalchemy.sql.selectable import Select
from nonebot_plugin_saa import PlatformTarget
from nonebot.compat import type_validate_python
from nonebot.log import logger
from nonebot_plugin_datastore.db import create_session
from nonebot_plugin_saa import PlatformTarget
from sqlalchemy import select
from sqlalchemy.orm.strategy_options import selectinload
from sqlalchemy.sql.selectable import Select
from nonebot_bison.config import config
from nonebot_bison.config.db_model import Cookie, CookieTarget, Subscribe, Target, User
from .. import config
from .utils import NBESFVerMatchErr, row2dict
from .nbesf_model import NBESFBase, v1, v2, v3
from ..db_model import User, Cookie, Target, Subscribe, CookieTarget
from .utils import NBESFVerMatchErr, row2dict
async def subscribes_export(selector: Callable[[Select], Select]) -> v3.SubGroup:
-42
View File
@@ -1,42 +0,0 @@
import time
from fastapi import APIRouter
from starlette.responses import Response
from prometheus_client import CONTENT_TYPE_LATEST, Gauge, Counter, Histogram, generate_latest
# Request counter
request_counter = Counter(
"bison_request_counter", "The number of requests", ["site_name", "platform_name", "target", "success"]
)
# Sent counter
sent_counter = Counter("bison_sent_counter", "The number of sent messages", ["site_name", "platform_name", "target"])
cookie_choose_counter = Counter(
"bison_cookie_choose_counter", "The number of cookie choose", ["site_name", "target", "cookie_id"]
)
request_histogram = Histogram(
"bison_request_histogram",
"The time of platform used to request the source",
["site_name", "platform_name"],
buckets=[0.1, 0.5, 1, 2, 5, 10, 30, 60],
)
render_histogram = Histogram(
"bison_render_histogram",
"The time of theme used to render",
["site_name", "platform_name"],
buckets=[0.1, 0.5, 1, 2, 5, 10, 30, 60],
)
start_time = Gauge("bison_start_time", "The start time of the program")
start_time.set(time.time())
metrics_router = APIRouter(prefix="/api/metrics", tags=["metrics"])
@metrics_router.get("")
async def metrics():
return Response(media_type=CONTENT_TYPE_LATEST, content=generate_latest())
+4 -3
View File
@@ -1,9 +1,10 @@
from pathlib import Path
from pkgutil import iter_modules
from collections import defaultdict
from importlib import import_module
from pathlib import Path
from pkgutil import iter_modules
from nonebot_bison.plugin_config import plugin_config
from ..plugin_config import plugin_config
from .platform import Platform, make_no_target_group
_package_dir = str(Path(__file__).resolve().parent)
+16 -15
View File
@@ -1,18 +1,19 @@
import re
import html
from typing import Any
from functools import partial
import html
import re
from typing import Any, ClassVar
from yarl import URL
from httpx import AsyncClient
from bs4 import BeautifulSoup as bs
from pydantic import Field, BaseModel
from httpx import AsyncClient
from nonebot.compat import type_validate_python
from pydantic import BaseModel, Field
from yarl import URL
from nonebot_bison.post import Post
from nonebot_bison.post.protocol import HTMLContentSupport
from nonebot_bison.types import Category, RawPost, Target
from nonebot_bison.utils import Site
from ..post import Post
from ..utils import Site
from ..types import Target, RawPost, Category
from ..post.protocol import HTMLContentSupport
from .platform import NewMessage, StatusChange
@@ -58,7 +59,7 @@ class ArkBulletinResponse(ArkResponseBase):
class ArknightsSite(Site):
name = "arknights"
schedule_type = "interval"
schedule_setting = {"seconds": 30}
schedule_setting: ClassVar[dict] = {"seconds": 30}
class ArknightsPost(Post, HTMLContentSupport):
@@ -95,7 +96,7 @@ class ArknightsPost(Post, HTMLContentSupport):
class Arknights(NewMessage):
categories = {1: "游戏公告"}
categories: ClassVar[dict[Category, str]] = {1: "游戏公告"}
platform_name = "arknights"
name = "明日方舟游戏信息"
enable_tag = False
@@ -157,7 +158,7 @@ class Arknights(NewMessage):
class AkVersion(StatusChange):
categories = {2: "更新信息"}
categories: ClassVar[dict[Category, str]] = {2: "更新信息"}
platform_name = "arknights"
name = "明日方舟游戏信息"
enable_tag = False
@@ -202,7 +203,7 @@ class AkVersion(StatusChange):
class MonsterSiren(NewMessage):
categories = {3: "塞壬唱片新闻"}
categories: ClassVar[dict[Category, str]] = {3: "塞壬唱片新闻"}
platform_name = "arknights"
name = "明日方舟游戏信息"
enable_tag = False
@@ -250,7 +251,7 @@ class MonsterSiren(NewMessage):
class TerraHistoricusComic(NewMessage):
categories = {4: "泰拉记事社漫画"}
categories: ClassVar[dict[Category, str]] = {4: "泰拉记事社漫画"}
platform_name = "arknights"
name = "明日方舟游戏信息"
enable_tag = False
+3 -3
View File
@@ -1,7 +1,7 @@
from .platforms import Bilibili as Bilibili
from .platforms import Bilibililive as Bilibililive
from .scheduler import BilibiliSite as BilibiliSite
from .scheduler import BililiveSite as BililiveSite
from .platforms import BilibiliBangumi as BilibiliBangumi
from .platforms import Bilibililive as Bilibililive
from .scheduler import BiliBangumiSite as BiliBangumiSite
from .scheduler import BilibiliClientManager as BilibiliClientManager
from .scheduler import BilibiliSite as BilibiliSite
from .scheduler import BililiveSite as BililiveSite
+9 -9
View File
@@ -1,22 +1,22 @@
import sys
import asyncio
import inspect
from collections.abc import AsyncGenerator, Awaitable, Callable, Sequence
from collections.abc import Set as AbstractSet
from dataclasses import dataclass
from enum import Enum
from functools import wraps
from dataclasses import dataclass
from collections.abc import Set as AbstractSet
from collections.abc import Callable, Sequence, Awaitable, AsyncGenerator
import inspect
import sys
from typing import (
TYPE_CHECKING,
Any,
Concatenate,
Generic,
TypeVar,
Protocol,
NamedTuple,
ParamSpec,
Protocol,
TypeAlias,
TypedDict,
NamedTuple,
Concatenate,
TypeVar,
overload,
runtime_checkable,
)
+3 -3
View File
@@ -1,7 +1,7 @@
from typing import Any, Literal, TypeVar, TypeAlias
from typing import Any, Literal, TypeAlias, TypeVar
from pydantic import BaseModel
from nonebot.compat import PYDANTIC_V2, ConfigDict
from pydantic import BaseModel
from nonebot_bison.compat import model_rebuild
@@ -13,7 +13,7 @@ TBaseModel = TypeVar("TBaseModel", bound=type[BaseModel])
def model_rebuild_recurse(cls: TBaseModel) -> TBaseModel:
"""Recursively rebuild all BaseModel subclasses in the class."""
if not PYDANTIC_V2:
from inspect import isclass, getmembers
from inspect import getmembers, isclass
for _, sub_cls in getmembers(cls, lambda x: isclass(x) and issubclass(x, BaseModel)):
model_rebuild_recurse(sub_cls)
+26 -27
View File
@@ -1,41 +1,41 @@
import re
import json
from copy import deepcopy
from enum import Enum, unique
from typing import NamedTuple
import json
import re
from typing import ClassVar, NamedTuple
from typing_extensions import Self
from yarl import URL
from nonebot import logger
from httpx import AsyncClient
from pydantic import Field, BaseModel, ValidationError
from nonebot import logger
from nonebot.compat import type_validate_json, type_validate_python
from pydantic import BaseModel, Field, ValidationError
from yarl import URL
from nonebot_bison.post.post import Post
from nonebot_bison.compat import model_rebuild
from nonebot_bison.utils import text_similarity, decode_unicode_escapes
from nonebot_bison.types import Tag, Target, RawPost, ApiError, Category
from nonebot_bison.platform.platform import CategoryNotRecognize, CategoryNotSupport, NewMessage, StatusChange
from nonebot_bison.post.post import Post
from nonebot_bison.types import ApiError, Category, RawPost, Tag, Target
from nonebot_bison.utils import decode_unicode_escapes, text_similarity
from .retry import ApiCode352Error, retry_for_352
from .scheduler import BilibiliSite, BililiveSite, BiliBangumiSite
from ..platform import NewMessage, StatusChange, CategoryNotSupport, CategoryNotRecognize
from .models import (
PostAPI,
UserAPI,
PGCMajor,
DrawMajor,
LiveMajor,
OPUSMajor,
DynRawPost,
VideoMajor,
CommonMajor,
DynamicType,
ArticleMajor,
CommonMajor,
CoursesMajor,
DeletedMajor,
UnknownMajor,
DrawMajor,
DynamicType,
DynRawPost,
LiveMajor,
LiveRecommendMajor,
OPUSMajor,
PGCMajor,
PostAPI,
UnknownMajor,
UserAPI,
VideoMajor,
)
from .retry import ApiCode352Error, retry_for_352
from .scheduler import BiliBangumiSite, BilibiliSite, BililiveSite
class _ProcessedText(NamedTuple):
@@ -51,7 +51,7 @@ class _ParsedMojarPost(NamedTuple):
class Bilibili(NewMessage):
categories = {
categories: ClassVar[dict[Category, str]] = {
1: "一般动态",
2: "专栏文章",
3: "视频",
@@ -162,7 +162,6 @@ class Bilibili(NewMessage):
return tags
def _text_process(self, dynamic: str, desc: str, title: str) -> _ProcessedText:
# 计算视频标题和视频描述相似度
title_similarity = 0.0 if len(title) == 0 or len(desc) == 0 else text_similarity(title, desc[: len(title)])
if title_similarity > 0.9:
@@ -308,7 +307,7 @@ class Bilibili(NewMessage):
class Bilibililive(StatusChange):
categories = {1: "开播提醒", 2: "标题更新提醒", 3: "下播提醒"}
categories: ClassVar[dict[Category, str]] = {1: "开播提醒", 2: "标题更新提醒", 3: "下播提醒"}
platform_name = "bilibili-live"
enable_tag = False
enabled = True
@@ -458,7 +457,7 @@ class Bilibililive(StatusChange):
class BilibiliBangumi(StatusChange):
categories = {}
categories: ClassVar[dict[Category, str]] = {}
platform_name = "bilibili-bangumi"
enable_tag = False
enabled = True
+9 -10
View File
@@ -1,20 +1,20 @@
import random
from enum import Enum
from functools import wraps
from collections.abc import Awaitable, Callable
from dataclasses import dataclass
from datetime import datetime, timedelta
from collections.abc import Callable, Awaitable
from typing_extensions import override, assert_never
from enum import Enum
from functools import wraps
import random
from typing import TYPE_CHECKING, Generic, Literal, TypeVar
from typing_extensions import assert_never, override
from strenum import StrEnum
from nonebot.log import logger
from httpx import URL as HttpxURL
from nonebot.log import logger
from strenum import StrEnum
from nonebot_bison.types import Target
from .fsm import FSM, ActionReturn, Condition, StateGraph, Transition, reset_on_exception
from .models import DynRawPost
from .fsm import FSM, Condition, StateGraph, Transition, ActionReturn, reset_on_exception
if TYPE_CHECKING:
from .platforms import Bilibili
@@ -115,8 +115,7 @@ class RetryAddon(Generic[TBilibili]):
def record_backoff_finish_time(self):
self.backoff_finish_time = (
datetime.now()
+ self.backoff_timedelta * self.backoff_count**2
datetime.now() + self.backoff_timedelta * self.backoff_count**2
# + timedelta(seconds=random.randint(1, 60)) # jitter
)
logger.trace(f"set backoff finish time: {self.backoff_finish_time}")
+11 -11
View File
@@ -1,29 +1,29 @@
from datetime import datetime, timedelta
import json
import random
from typing import TYPE_CHECKING, ClassVar, TypeVar
from typing_extensions import override
from datetime import datetime, timedelta
from typing import TYPE_CHECKING, TypeVar
from httpx import AsyncClient
from nonebot import logger, require
from playwright.async_api import Cookie
from nonebot_bison.config.db_model import Cookie as CookieModel
from nonebot_bison.plugin_config import plugin_config
from nonebot_bison.utils import Site, http_client
from ...utils.site import CookieClientManager
from ...config.db_model import Cookie as CookieModel
from nonebot_bison.utils.site import CookieClientManager
if TYPE_CHECKING:
from .platforms import Bilibili
require("nonebot_plugin_htmlrender")
from nonebot_plugin_htmlrender import get_browser
if plugin_config.bison_use_browser:
require("nonebot_plugin_htmlrender")
from nonebot_plugin_htmlrender import get_browser
B = TypeVar("B", bound="Bilibili")
class BilibiliClientManager(CookieClientManager):
_default_cookie_cd = timedelta(seconds=120)
async def _get_cookies(self) -> list[Cookie]:
@@ -75,7 +75,7 @@ class BilibiliClientManager(CookieClientManager):
class BilibiliSite(Site):
name = "bilibili.com"
schedule_setting = {"seconds": 60}
schedule_setting: ClassVar[dict] = {"seconds": 60}
schedule_type = "interval"
client_mgr = BilibiliClientManager
require_browser = True
@@ -83,11 +83,11 @@ class BilibiliSite(Site):
class BililiveSite(Site):
name = "live.bilibili.com"
schedule_setting = {"seconds": 5}
schedule_setting: ClassVar[dict] = {"seconds": 5}
schedule_type = "interval"
class BiliBangumiSite(Site):
name = "bilibili.com/bangumi"
schedule_setting = {"seconds": 30}
schedule_setting: ClassVar[dict] = {"seconds": 30}
schedule_type = "interval"
+7 -7
View File
@@ -1,16 +1,16 @@
from typing import TypeAlias
from functools import partial
from datetime import timedelta
from types import MappingProxyType
from collections.abc import Callable
from datetime import timedelta
from functools import partial
from types import MappingProxyType
from typing import TypeAlias
from expiringdictx import ExpiringDict, SimpleCache
from hishel import AsyncCacheTransport, AsyncInMemoryStorage, Controller
from httpx import AsyncClient, AsyncHTTPTransport
from expiringdictx import SimpleCache, ExpiringDict
from hishel import Controller, AsyncCacheTransport, AsyncInMemoryStorage
from .const import DATASOURCE_URL
from .utils import process_response
from .models import CeobeSource, CeobeTarget, DataSourceResponse
from .utils import process_response
cache_transport = AsyncCacheTransport(
AsyncHTTPTransport(),
@@ -1,4 +1,4 @@
from typing import Literal, TypeVar, NamedTuple
from typing import Literal, NamedTuple, TypeVar
from pydantic import BaseModel
+16 -14
View File
@@ -1,23 +1,23 @@
from typing import ParamSpec
from functools import partial
from datetime import timedelta
from collections import defaultdict
from datetime import timedelta
from functools import partial
from typing import ClassVar, ParamSpec
from httpx import AsyncClient
from nonebot import logger, require
from rapidfuzz import fuzz, process
from nonebot_bison.post import Post
from nonebot_bison.platform.platform import NewMessage
from nonebot_bison.plugin_config import plugin_config
from nonebot_bison.types import Target, RawPost, Category
from nonebot_bison.utils import Site, ClientManager, capture_html
from nonebot_bison.post import Post
from nonebot_bison.types import Category, RawPost, Target
from nonebot_bison.utils import ClientManager, Site, capture_html
from ..platform import NewMessage
from .utils import process_response
from .const import COMB_ID_URL, COOKIES_URL, COOKIE_ID_URL
from .exception import CeobeSnapshotSkip, CeobeSnapshotFailed
from .cache import CeobeCache, CeobeClient, CeobeDataSourceCache
from .models import CeobeImage, CeobeCookie, CeobeTextPic, CombIdResponse, CookiesResponse, CookieIdResponse
from .const import COMB_ID_URL, COOKIE_ID_URL, COOKIES_URL
from .exception import CeobeSnapshotFailed, CeobeSnapshotSkip
from .models import CeobeCookie, CeobeImage, CeobeTextPic, CombIdResponse, CookieIdResponse, CookiesResponse
from .utils import process_response
P = ParamSpec("P")
@@ -49,7 +49,7 @@ class CeobeCanteenSite(Site):
name = "ceobe_canteen"
schedule_type = "interval"
# lwt の 推荐间隔
schedule_setting = {"seconds": 15}
schedule_setting: ClassVar[dict] = {"seconds": 15}
client_mgr = CeobeCanteenClientManager
@@ -64,7 +64,7 @@ class CeobeCanteen(NewMessage):
use_batch: bool = True
default_theme: str = "ceobecanteen"
categories: dict[Category, str] = {1: "普通", 2: "转发"}
categories: ClassVar[dict[Category, str]] = {1: "普通", 2: "转发"}
data_source_cache = CeobeDataSourceCache()
@@ -213,7 +213,9 @@ class CeobeCanteen(NewMessage):
logger.debug(f"snapshot official website url: {url}")
# /html/body/div[1]/div[1]/div/div[1]/div[1]/div
snapshot_selector = "html > body > div:nth-child(1) > div:nth-child(1) > div > div:nth-child(1) > div:nth-child(1) > div" # noqa: E501
snapshot_selector = (
"html > body > div:nth-child(1) > div:nth-child(1) > div > div:nth-child(1) > div:nth-child(1) > div"
)
# /html/body/div[1]/div[1]/div/div[1]/div[1]/div/div[4]/div/div/div
calculate_selector = "html > body > div:nth-child(1) > div:nth-child(1) > div > div:nth-child(1) > div:nth-child(1) > div > div:nth-child(4) > div > div > div" # noqa: E501
viewport = {"width": 1024, "height": 19990}
+1 -1
View File
@@ -3,7 +3,7 @@ from nonebot import logger
from nonebot.compat import type_validate_python
from .exception import CeobeResponseError
from .models import ResponseModel, CookieIdResponse
from .models import CookieIdResponse, ResponseModel
def process_response(response: Response, parse_model: type[ResponseModel]) -> ResponseModel:
+6 -5
View File
@@ -1,15 +1,16 @@
from typing import Any
from typing import Any, ClassVar
from httpx import AsyncClient
from ..post import Post
from nonebot_bison.post import Post
from nonebot_bison.types import RawPost, Target
from nonebot_bison.utils import anonymous_site
from .platform import NewMessage
from ..utils import anonymous_site
from ..types import Target, RawPost
class FF14(NewMessage):
categories = {}
categories: ClassVar[dict] = {}
platform_name = "ff14"
name = "最终幻想XIV官方公告"
enable_tag = False
+8 -7
View File
@@ -1,22 +1,23 @@
import re
from typing import Any
from typing import Any, ClassVar
from httpx import AsyncClient
from ..post import Post
from ..utils import Site
from nonebot_bison.post import Post
from nonebot_bison.types import ApiError, Category, RawPost, Target
from nonebot_bison.utils import Site
from .platform import NewMessage
from ..types import Target, RawPost, ApiError
class NcmSite(Site):
name = "music.163.com"
schedule_type = "interval"
schedule_setting = {"minutes": 1}
schedule_setting: ClassVar[dict] = {"minutes": 1}
class NcmArtist(NewMessage):
categories = {}
categories: ClassVar[dict[Category, str]] = {}
platform_name = "ncm-artist"
enable_tag = False
enabled = True
@@ -73,7 +74,7 @@ class NcmArtist(NewMessage):
class NcmRadio(NewMessage):
categories = {}
categories: ClassVar[dict[Category, str]] = {}
platform_name = "ncm-radio"
enable_tag = False
enabled = True
+11 -12
View File
@@ -1,22 +1,22 @@
import ssl
import json
import time
import typing
from dataclasses import dataclass
from abc import ABC, abstractmethod
from collections import defaultdict
from typing import Any, TypeVar, ParamSpec
from collections.abc import Callable, Awaitable, Collection
from collections.abc import Awaitable, Callable, Collection
from dataclasses import dataclass
import json
import ssl
import time
import typing
from typing import Any, ParamSpec, TypeVar
import httpx
from httpx import AsyncClient
from nonebot.log import logger
from nonebot_plugin_saa import PlatformTarget
from ..post import Post
from ..utils import Site, ProcessContext
from ..plugin_config import plugin_config
from ..types import Tag, Target, RawPost, SubUnit, Category
from nonebot_bison.plugin_config import plugin_config
from nonebot_bison.post import Post
from nonebot_bison.types import Category, RawPost, SubUnit, Tag, Target
from nonebot_bison.utils import ProcessContext, Site
class CategoryNotSupport(Exception):
@@ -315,7 +315,6 @@ class NewMessage(MessageProcess, abstract=True):
res.append(raw_post)
store.exists_posts.add(post_id)
self.set_stored_data(target, store)
logger.trace(f"本次抓取 {len(raw_post_list)} 条,过滤后 {len(filtered_post)} 条,新消息 {len(res)}")
return res
async def _handle_new_post(
+10 -10
View File
@@ -1,27 +1,27 @@
import time
import calendar
from typing import Any
import time
from typing import Any, ClassVar
from bs4 import BeautifulSoup as bs
import feedparser
from httpx import AsyncClient
from bs4 import BeautifulSoup as bs
from ..post import Post
from nonebot_bison.post import Post
from nonebot_bison.types import Category, RawPost, Target
from nonebot_bison.utils import text_similarity
from nonebot_bison.utils.site import CookieClientManager, Site
from .platform import NewMessage
from ..types import Target, RawPost
from ..utils import text_similarity
from ..utils.site import Site, CookieClientManager
class RssSite(Site):
name = "rss"
schedule_type = "interval"
schedule_setting = {"seconds": 30}
schedule_setting: ClassVar[dict] = {"seconds": 30}
client_mgr = CookieClientManager.from_name(name)
class RssPost(Post):
async def get_plain_content(self) -> str:
soup = bs(self.content, "html.parser")
@@ -38,7 +38,7 @@ class RssPost(Post):
class Rss(NewMessage):
categories = {}
categories: ClassVar[dict[Category, str]] = {}
enable_tag = False
platform_name = "rss"
name = "Rss"
+15 -14
View File
@@ -1,21 +1,22 @@
import re
import json
from typing import Any
from datetime import datetime
from urllib.parse import unquote
import json
import re
from typing import Any, ClassVar
from typing_extensions import override
from urllib.parse import unquote
from yarl import URL
from lxml.etree import HTML
from httpx import AsyncClient
from nonebot.log import logger
from bs4 import BeautifulSoup as bs
from httpx import AsyncClient
from lxml.etree import HTML
from nonebot.log import logger
from yarl import URL
from nonebot_bison.post import Post
from nonebot_bison.types import ApiError, Category, RawPost, Tag, Target
from nonebot_bison.utils import http_client, text_fletten
from nonebot_bison.utils.site import CookieClientManager, Site
from ..post import Post
from .platform import NewMessage
from ..utils import http_client, text_fletten
from ..utils.site import Site, CookieClientManager
from ..types import Tag, Target, RawPost, ApiError, Category
_HEADER = {
"accept": (
@@ -59,12 +60,12 @@ class WeiboClientManager(CookieClientManager):
class WeiboSite(Site):
name = "weibo.com"
schedule_type = "interval"
schedule_setting = {"seconds": 3}
schedule_setting: ClassVar[dict] = {"seconds": 3}
client_mgr = WeiboClientManager
class Weibo(NewMessage):
categories = {
categories: ClassVar[dict[Category, str]] = {
1: "转发",
2: "视频",
3: "图文",
+2 -2
View File
@@ -1,8 +1,8 @@
import nonebot
from yarl import URL
from nonebot import get_plugin_config
from pydantic import Field, BaseModel
from nonebot.compat import PYDANTIC_V2, ConfigDict
from pydantic import BaseModel, Field
from yarl import URL
global_config = nonebot.get_driver().config
PlatformName = str
+4 -4
View File
@@ -1,10 +1,10 @@
from dataclasses import dataclass
from abc import ABC, abstractmethod
from dataclasses import dataclass
from nonebot_plugin_saa import Text, MessageFactory, MessageSegmentFactory
from nonebot_plugin_saa import MessageFactory, MessageSegmentFactory, Text
from ..utils import text_to_image
from ..plugin_config import plugin_config
from nonebot_bison.plugin_config import plugin_config
from nonebot_bison.utils import text_to_image
@dataclass(kw_only=True)
+8 -7
View File
@@ -1,21 +1,22 @@
import reprlib
from collections.abc import Sequence
from dataclasses import dataclass, fields
from io import BytesIO
from pathlib import Path
import reprlib
from typing import TYPE_CHECKING
from collections.abc import Sequence
from dataclasses import fields, dataclass
from nonebot.log import logger
from nonebot_plugin_saa import MessageSegmentFactory
from ..theme import theme_manager
from nonebot_bison.plugin_config import plugin_config
from nonebot_bison.theme import theme_manager
from nonebot_bison.theme.types import ThemeRenderError, ThemeRenderUnsupportError
from .abstract_post import AbstractPost
from ..plugin_config import plugin_config
from .protocol import PlainContentSupport
from ..theme.types import ThemeRenderError, ThemeRenderUnsupportError
if TYPE_CHECKING:
from ..platform import Platform
from nonebot_bison.platform import Platform
@dataclass
+2 -2
View File
@@ -1,3 +1,3 @@
from .manager import init_scheduler, scheduler_dict, handle_delete_target, handle_insert_new_target
from .manager import handle_delete_target, handle_insert_new_target, init_scheduler, scheduler_dict
__all__ = ["init_scheduler", "handle_delete_target", "handle_insert_new_target", "scheduler_dict"]
__all__ = ["handle_delete_target", "handle_insert_new_target", "init_scheduler", "scheduler_dict"]
+8 -7
View File
@@ -2,14 +2,15 @@ from typing import cast
from nonebot.log import logger
from ..utils import Site
from ..config import config
from nonebot_bison.config import config
from nonebot_bison.config.db_model import Target
from nonebot_bison.platform import platform_manager
from nonebot_bison.plugin_config import plugin_config
from nonebot_bison.types import Target as T_Target
from nonebot_bison.utils import Site
from nonebot_bison.utils.site import CookieClientManager, is_cookie_client_manager
from .scheduler import Scheduler
from ..config.db_model import Target
from ..types import Target as T_Target
from ..platform import platform_manager
from ..plugin_config import plugin_config
from ..utils.site import CookieClientManager, is_cookie_client_manager
scheduler_dict: dict[type[Site], Scheduler] = {}
+31 -62
View File
@@ -1,21 +1,16 @@
from dataclasses import dataclass
from collections import defaultdict
from collections.abc import Callable
from dataclasses import dataclass
from nonebot.log import logger
from nonebot_plugin_apscheduler import scheduler
from apscheduler.events import EVENT_JOB_MAX_INSTANCES
from nonebot_plugin_saa.utils.exceptions import NoBotFound
from nonebot_bison.utils import ClientManager
from nonebot_bison.metrics import sent_counter, request_counter, render_histogram, request_histogram
from ..config import config
from ..send import send_msgs
from ..types import Target, SubUnit
from ..platform import platform_manager
from ..utils import Site, ProcessContext
from ..utils.site import SkipRequestException
from nonebot_bison.config import config
from nonebot_bison.platform import platform_manager
from nonebot_bison.send import send_msgs
from nonebot_bison.types import SubUnit, Target
from nonebot_bison.utils import ClientManager, ProcessContext, Site
from nonebot_bison.utils.site import SkipRequestException
@dataclass
@@ -26,15 +21,6 @@ class Schedulable:
use_batch: bool = False
def handle_time_exceeded(event):
# event.job_id 是该任务在 apscheduler 的 id, 进而可以获得该任务的函数,再获取该函数绑定的对象
logger.warning(f"{scheduler.get_job(event.job_id).func.__self__.name} 抓取执行超时")
scheduler.get_job(event.job_id).func.__self__.metrics_report(False)
scheduler.add_listener(handle_time_exceeded, EVENT_JOB_MAX_INSTANCES)
class Scheduler:
schedulable_list: list[Schedulable] # for load weigth from db
batch_api_target_cache: dict[str, dict[Target, list[Target]]] # platform_name -> (target -> [target])
@@ -67,7 +53,6 @@ class Scheduler:
self.platform_name_list = platform_name_list
self.pre_weight_val = 0 # 轮调度中“本轮”增加权重和的初值
self.metrics_report: Callable[[bool], None] | None = None # 作为函数变量,允许外部调用来上报此次抓取是否成功
logger.info(
f"register scheduler for {self.name} with "
f"{self.scheduler_config.schedule_type} {self.scheduler_config.schedule_setting}"
@@ -107,32 +92,20 @@ class Scheduler:
context = ProcessContext(self.client_mgr)
success_flag = False
platform_obj = platform_manager[schedulable.platform_name](context)
# 通过闭包的形式,将此次抓取任务的信息保存为函数变量,允许在该任务无法正常结束时由外部上报
self.metrics_report = lambda x: request_counter.labels(
platform_name=schedulable.platform_name,
site_name=platform_obj.site.name,
target=schedulable.target,
success=x,
).inc()
try:
with request_histogram.labels(
platform_name=schedulable.platform_name, site_name=platform_obj.site.name
).time():
if schedulable.use_batch:
batch_targets = self.batch_api_target_cache[schedulable.platform_name][schedulable.target]
sub_units = []
for batch_target in batch_targets:
userinfo = await config.get_platform_target_subscribers(schedulable.platform_name, batch_target)
sub_units.append(SubUnit(batch_target, userinfo))
to_send = await platform_obj.do_batch_fetch_new_post(sub_units)
else:
send_userinfo_list = await config.get_platform_target_subscribers(
schedulable.platform_name, schedulable.target
)
to_send = await platform_obj.do_fetch_new_post(SubUnit(schedulable.target, send_userinfo_list))
success_flag = True
platform_obj = platform_manager[schedulable.platform_name](context)
if schedulable.use_batch:
batch_targets = self.batch_api_target_cache[schedulable.platform_name][schedulable.target]
sub_units = []
for batch_target in batch_targets:
userinfo = await config.get_platform_target_subscribers(schedulable.platform_name, batch_target)
sub_units.append(SubUnit(batch_target, userinfo))
to_send = await platform_obj.do_batch_fetch_new_post(sub_units)
else:
send_userinfo_list = await config.get_platform_target_subscribers(
schedulable.platform_name, schedulable.target
)
to_send = await platform_obj.do_fetch_new_post(SubUnit(schedulable.target, send_userinfo_list))
except SkipRequestException as err:
logger.debug(f"skip request: {err}")
except Exception as err:
@@ -142,23 +115,19 @@ class Scheduler:
err.args += (records,)
raise
self.metrics_report(success_flag)
if not to_send:
return
sent_counter.labels(
platform_name=schedulable.platform_name, site_name=platform_obj.site.name, target=schedulable.target
).inc()
with render_histogram.labels(platform_name=schedulable.platform_name, site_name=platform_obj.site.name).time():
for user, send_list in to_send:
for send_post in send_list:
logger.info(f"send to {user}: {send_post}")
try:
await send_msgs(
user,
await send_post.generate_messages(),
)
except NoBotFound:
logger.warning("no bot connected")
for user, send_list in to_send:
for send_post in send_list:
logger.info(f"send to {user}: {send_post}")
try:
await send_msgs(
user,
await send_post.generate_messages(),
)
except NoBotFound:
logger.warning("no bot connected")
def insert_new_schedulable(self, platform_name: str, target: Target):
self.pre_weight_val += 1000
+13 -12
View File
@@ -1,25 +1,26 @@
import json
import time
from collections.abc import Callable, Coroutine
from functools import partial, wraps
import importlib
import json
from pathlib import Path
import time
from types import ModuleType
from typing import Any, TypeVar
from functools import wraps, partial
from collections.abc import Callable, Coroutine
from nonebot.log import logger
from anyio import open_file
from nonebot.compat import model_dump
from nonebot.log import logger
from ..scheduler.manager import init_scheduler
from ..config.subs_io.nbesf_model import v1, v2, v3
from ..config.subs_io import subscribes_export, subscribes_import
from nonebot_bison.config.subs_io import subscribes_export, subscribes_import
from nonebot_bison.config.subs_io.nbesf_model import v1, v2, v3
from nonebot_bison.scheduler.manager import init_scheduler
try:
from typing_extensions import ParamSpec
import anyio
from anyio import from_thread, to_thread
import click
from anyio import to_thread, from_thread
except ImportError as e: # pragma: no cover
raise ImportError("请使用 `pip install nonebot-bison[cli]` 安装所需依赖") from e
@@ -127,18 +128,18 @@ async def subs_import(path: str, format: str):
import_file_path = Path(path)
assert import_file_path.is_file(), "该路径不是文件!"
with import_file_path.open("r", encoding="utf-8") as f:
async with await open_file(import_file_path, "r", encoding="utf-8") as f:
match format:
case "yaml" | "yml":
logger.info("正在从yaml导入...")
pyyaml = import_yaml_module()
import_items = pyyaml.safe_load(f)
import_items = pyyaml.safe_load(await f.read())
case "json":
logger.info("正在从json导入...")
import_items = json.load(f)
import_items = json.loads(await f.read())
case _:
raise click.BadParameter(message=f"不支持的导入格式: {format}")
+8 -4
View File
@@ -1,10 +1,10 @@
import asyncio
from collections import deque
from nonebot.log import logger
from nonebot_plugin_saa.auto_select_bot import refresh_bots
from nonebot.adapters.onebot.v11.exception import ActionFailed
from nonebot_plugin_saa import MessageFactory, PlatformTarget, AggregatedMessageFactory
from nonebot.log import logger
from nonebot_plugin_saa import AggregatedMessageFactory, MessageFactory, PlatformTarget
from nonebot_plugin_saa.auto_select_bot import refresh_bots
from .plugin_config import plugin_config
@@ -14,6 +14,8 @@ QUEUE: deque[tuple[PlatformTarget, Sendable, int]] = deque()
MESSGE_SEND_INTERVAL = 1.5
_MESSAGE_DISPATCH_TASKS: set[asyncio.Task] = set()
async def _do_send(send_target: PlatformTarget, msg: Sendable):
try:
@@ -59,7 +61,9 @@ async def _send_msgs_dispatch(send_target: PlatformTarget, msg: Sendable):
QUEUE.append((send_target, msg, plugin_config.bison_resend_times))
# len(QUEUE) before append was 0
if len(QUEUE) == 1:
asyncio.create_task(do_send_msgs())
task = asyncio.create_task(do_send_msgs())
_MESSAGE_DISPATCH_TASKS.add(task)
task.add_done_callback(_MESSAGE_DISPATCH_TASKS.discard)
else:
await _do_send(send_target, msg)
+23 -18
View File
@@ -2,23 +2,25 @@ import asyncio
from datetime import datetime
from nonebot import on_command
from nonebot.typing import T_State
from nonebot.matcher import Matcher
from nonebot.rule import Rule, to_me
from nonebot.permission import SUPERUSER
from nonebot_plugin_saa import TargetQQGroup
from nonebot.params import ArgStr, ArgPlainText
from nonebot.adapters import Bot, MessageTemplate
from nonebot.adapters.onebot.v11.event import PrivateMessageEvent
from nonebot.matcher import Matcher
from nonebot.params import ArgPlainText, ArgStr
from nonebot.permission import SUPERUSER
from nonebot.rule import Rule, to_me
from nonebot.typing import T_State
from nonebot_plugin_saa import TargetQQGroup
from .add_cookie import do_add_cookie
from .add_cookie_target import do_add_cookie_target
from .add_sub import do_add_sub
from .del_cookie import do_del_cookie
from .del_cookie_target import do_del_cookie_target
from .del_sub import do_del_sub
from .query_sub import do_query_sub
from .add_cookie import do_add_cookie
from .del_cookie import do_del_cookie
from .add_cookie_target import do_add_cookie_target
from .del_cookie_target import do_del_cookie_target
from .utils import common_platform, admin_permission, gen_handle_cancel, configurable_to_me, set_target_user_info
from .utils import admin_permission, common_platform, configurable_to_me, gen_handle_cancel, set_target_user_info
_COMMAND_DISPATCH_TASKS: set[asyncio.Task] = set()
add_sub_matcher = on_command(
"添加订阅",
@@ -149,7 +151,10 @@ async def do_dispatch_command(
else:
do_del_sub(new_matcher)
new_matcher_ins = new_matcher()
asyncio.create_task(new_matcher_ins.run(bot, event, state))
task = asyncio.create_task(new_matcher_ins.run(bot, event, state))
_COMMAND_DISPATCH_TASKS.add(task)
task.add_done_callback(_COMMAND_DISPATCH_TASKS.discard)
no_permission_matcher = on_command(
@@ -167,14 +172,14 @@ async def send_no_permission():
__all__ = [
"common_platform",
"add_cookie_matcher",
"add_cookie_target_matcher",
"add_sub_matcher",
"query_sub_matcher",
"common_platform",
"del_cookie_matcher",
"del_cookie_target_matcher",
"del_sub_matcher",
"group_manage_matcher",
"no_permission_matcher",
"add_cookie_matcher",
"add_cookie_target_matcher",
"del_cookie_target_matcher",
"del_cookie_matcher",
"query_sub_matcher",
]
+8 -7
View File
@@ -1,16 +1,17 @@
from typing import cast
from json import JSONDecodeError
from typing import cast
from nonebot.adapters import Message, MessageTemplate
from nonebot.adapters.onebot.v11 import MessageEvent
from nonebot.log import logger
from nonebot.typing import T_State
from nonebot.matcher import Matcher
from nonebot.params import Arg, ArgPlainText
from nonebot.adapters.onebot.v11 import MessageEvent
from nonebot.adapters import Message, MessageTemplate
from nonebot.typing import T_State
from nonebot_bison.platform import platform_manager
from nonebot_bison.scheduler import scheduler_dict
from nonebot_bison.utils.site import CookieClientManager, is_cookie_client_manager
from ..scheduler import scheduler_dict
from ..platform import platform_manager
from ..utils.site import CookieClientManager, is_cookie_client_manager
from .utils import common_platform, gen_handle_cancel, only_allow_private
@@ -1,14 +1,15 @@
from nonebot.typing import T_State
from nonebot.matcher import Matcher
from nonebot.params import ArgPlainText
from nonebot_plugin_saa import MessageFactory
from nonebot.adapters.onebot.v11 import MessageEvent
from nonebot.internal.adapter import MessageTemplate
from nonebot.matcher import Matcher
from nonebot.params import ArgPlainText
from nonebot.typing import T_State
from nonebot_plugin_saa import MessageFactory
from ..config import config
from ..utils import parse_text
from ..platform import platform_manager
from .utils import gen_handle_cancel, only_allow_private, generate_sub_list_text
from nonebot_bison.config import config
from nonebot_bison.platform import platform_manager
from nonebot_bison.utils import parse_text
from .utils import gen_handle_cancel, generate_sub_list_text, only_allow_private
def do_add_cookie_target(add_cookie_target_matcher: type[Matcher]):
@@ -33,7 +34,6 @@ def do_add_cookie_target(add_cookie_target_matcher: type[Matcher]):
@add_cookie_target_matcher.handle()
async def init_promote_cookie(state: T_State):
# 获取 site 的所有用户 cookie,再排除掉已经关联的 cookie,剩下的就是可以关联的 cookie
cookies = await config.get_cookie(site_name=state["site"].name, is_anonymous=False)
associated_cookies = await config.get_cookie(
+9 -8
View File
@@ -1,17 +1,18 @@
import contextlib
from nonebot.typing import T_State
from nonebot.adapters import Message, MessageTemplate
from nonebot.matcher import Matcher
from nonebot.params import Arg, ArgPlainText
from nonebot.adapters import Message, MessageTemplate
from nonebot_plugin_saa import Text, PlatformTarget, SupportedAdapters
from nonebot.typing import T_State
from nonebot_plugin_saa import PlatformTarget, SupportedAdapters, Text
from nonebot_bison.apis import check_sub_target
from nonebot_bison.config import config
from nonebot_bison.config.db_config import SubscribeDupException
from nonebot_bison.platform import Platform, platform_manager, unavailable_paltforms
from nonebot_bison.types import Target
from ..types import Target
from ..config import config
from ..apis import check_sub_target
from ..config.db_config import SubscribeDupException
from .utils import common_platform, ensure_user_info, gen_handle_cancel
from ..platform import Platform, platform_manager, unavailable_paltforms
def do_add_sub(add_sub: type[Matcher]):
+5 -4
View File
@@ -1,11 +1,12 @@
from nonebot.typing import T_State
from nonebot.adapters.onebot.v11 import MessageEvent
from nonebot.matcher import Matcher
from nonebot.params import EventPlainText
from nonebot.typing import T_State
from nonebot_plugin_saa import MessageFactory
from nonebot.adapters.onebot.v11 import MessageEvent
from ..config import config
from ..utils import parse_text
from nonebot_bison.config import config
from nonebot_bison.utils import parse_text
from .utils import gen_handle_cancel, only_allow_private
@@ -1,11 +1,12 @@
from nonebot.typing import T_State
from nonebot.adapters.onebot.v11 import MessageEvent
from nonebot.matcher import Matcher
from nonebot.params import EventPlainText
from nonebot.typing import T_State
from nonebot_plugin_saa import MessageFactory
from nonebot.adapters.onebot.v11 import MessageEvent
from ..config import config
from ..utils import parse_text
from nonebot_bison.config import config
from nonebot_bison.utils import parse_text
from .utils import gen_handle_cancel, only_allow_private
+6 -5
View File
@@ -1,12 +1,13 @@
from nonebot.typing import T_State
from nonebot.matcher import Matcher
from nonebot.params import Arg, EventPlainText
from nonebot.typing import T_State
from nonebot_plugin_saa import MessageFactory, PlatformTarget
from ..config import config
from ..types import Category
from ..utils import parse_text
from ..platform import platform_manager
from nonebot_bison.config import config
from nonebot_bison.platform import platform_manager
from nonebot_bison.types import Category
from nonebot_bison.utils import parse_text
from .utils import ensure_user_info, gen_handle_cancel
+6 -5
View File
@@ -1,12 +1,13 @@
from nonebot.params import Arg
from nonebot.matcher import Matcher
from nonebot.params import Arg
from nonebot_plugin_saa import MessageFactory, PlatformTarget
from ..config import config
from ..types import Category
from ..utils import parse_text
from nonebot_bison.config import config
from nonebot_bison.platform import platform_manager
from nonebot_bison.types import Category
from nonebot_bison.utils import parse_text
from .utils import ensure_user_info
from ..platform import platform_manager
def do_query_sub(query_sub: type[Matcher]):
+10 -10
View File
@@ -1,22 +1,22 @@
import contextlib
from typing import Annotated
from itertools import groupby
from operator import attrgetter
from typing import Annotated
from nonebot.rule import Rule
from nonebot.adapters import Event
from nonebot.typing import T_State
from nonebot.matcher import Matcher
from nonebot.params import Depends, EventPlainText, EventToMe
from nonebot.permission import SUPERUSER
from nonebot.params import Depends, EventToMe, EventPlainText
from nonebot.rule import Rule
from nonebot.typing import T_State
from nonebot_plugin_saa import PlatformTarget, extract_target
from ..config import config
from ..types import Category
from ..types import Target as T_Target
from ..platform import platform_manager
from ..plugin_config import plugin_config
from ..utils.site import is_cookie_client_manager
from nonebot_bison.config import config
from nonebot_bison.platform import platform_manager
from nonebot_bison.plugin_config import plugin_config
from nonebot_bison.types import Category
from nonebot_bison.types import Target as T_Target
from nonebot_bison.utils.site import is_cookie_client_manager
def _configurable_to_me(to_me: bool = EventToMe()):
+2 -3
View File
@@ -1,10 +1,9 @@
from importlib import import_module
from pathlib import Path
from pkgutil import iter_modules
from importlib import import_module
from .types import Theme
from .registry import theme_manager
from .types import ThemeRegistrationError
from .types import Theme, ThemeRegistrationError
from .types import ThemeRenderError as ThemeRenderError
from .types import ThemeRenderUnsupportError as ThemeRenderUnsupportError
+5 -2
View File
@@ -1,11 +1,14 @@
from typing import ClassVar
from nonebot import logger
from ..plugin_config import plugin_config
from nonebot_bison.plugin_config import plugin_config
from .types import Theme, ThemeRegistrationError
class ThemeManager:
__themes: dict[str, Theme] = {}
__themes: ClassVar[dict[str, Theme]] = {}
def register(self, theme: Theme):
logger.trace(f"Registering theme: {theme}")
@@ -1,13 +1,13 @@
from dataclasses import dataclass
from io import BytesIO
from pathlib import Path
from dataclasses import dataclass
from typing import TYPE_CHECKING, Literal
from nonebot_plugin_saa import Text, Image, MessageSegmentFactory
from nonebot_plugin_saa import Image, MessageSegmentFactory, Text
from nonebot_bison.utils import text_fletten
from nonebot_bison.theme.utils import web_embed_image
from nonebot_bison.theme import Theme, ThemeRenderError, ThemeRenderUnsupportError
from nonebot_bison.theme.utils import web_embed_image
from nonebot_bison.utils import text_fletten
if TYPE_CHECKING:
from nonebot_bison.platform.arknights import ArknightsPost
+4 -3
View File
@@ -1,11 +1,12 @@
from collections.abc import Sequence
from io import BytesIO
from pathlib import Path
from typing import TYPE_CHECKING, Literal
from nonebot_plugin_saa import Text, Image, MessageSegmentFactory
from nonebot_plugin_saa import Image, MessageSegmentFactory, Text
from nonebot_bison.theme import Theme
from nonebot_bison.utils import pic_merge, is_pics_mergable
from nonebot_bison.utils import is_pics_mergable, pic_merge
if TYPE_CHECKING:
from nonebot_bison.post import Post
@@ -50,7 +51,7 @@ class BasicTheme(Theme):
client = await post.platform.ctx.get_client_for_static()
msgs: list[MessageSegmentFactory] = [Text(text)]
pics_group: list[list[str | bytes | Path | BytesIO]] = []
pics_group: list[Sequence[str | bytes | Path | BytesIO]] = []
if post.images:
pics_group.append(post.images)
if rp and rp.images:
+2 -2
View File
@@ -1,9 +1,9 @@
from typing import TYPE_CHECKING, Literal
from nonebot_plugin_saa import Text, Image, MessageSegmentFactory
from nonebot_plugin_saa import Image, MessageSegmentFactory, Text
from nonebot_bison.utils import pic_merge, is_pics_mergable
from nonebot_bison.theme import Theme, ThemeRenderUnsupportError
from nonebot_bison.utils import is_pics_mergable, pic_merge
if TYPE_CHECKING:
from nonebot_bison.post import Post
@@ -1,19 +1,20 @@
from collections.abc import Sequence
from datetime import datetime
from io import BytesIO
from pathlib import Path
from datetime import datetime
from typing import TYPE_CHECKING, Literal
import jinja2
from yarl import URL
from httpx import AsyncClient
from pydantic import BaseModel
import jinja2
from nonebot_plugin_saa import Image, MessageSegmentFactory, Text
from PIL import Image as PILImage
from nonebot_plugin_saa import Text, Image, MessageSegmentFactory
from pydantic import BaseModel
from yarl import URL
from nonebot_bison.compat import model_validator
from nonebot_bison.utils import pic_merge, is_pics_mergable
from nonebot_bison.theme.utils import convert_to_qr, web_embed_image
from nonebot_bison.theme import Theme, ThemeRenderError, ThemeRenderUnsupportError
from nonebot_bison.theme.utils import convert_to_qr, web_embed_image
from nonebot_bison.utils import is_pics_mergable, pic_merge
if TYPE_CHECKING:
from nonebot_bison.post import Post
@@ -122,7 +123,7 @@ class CeobeCanteenTheme(Theme):
@staticmethod
async def merge_pics(
images: list[str | bytes | Path | BytesIO],
images: Sequence[str | bytes | Path | BytesIO],
client: AsyncClient,
) -> list[str | bytes | Path | BytesIO]:
if is_pics_mergable(images):
@@ -224,7 +225,7 @@ class CeobeCanteenTheme(Theme):
text += f"详情: {post.url}"
msgs.append(Text(text))
pics_group: list[list[str | bytes | Path | BytesIO]] = []
pics_group: list[Sequence[str | bytes | Path | BytesIO]] = []
if post.images:
pics_group.append(post.images)
if post.repost and post.repost.images:
+6 -8
View File
@@ -1,12 +1,13 @@
from collections.abc import Sequence
from io import BytesIO
from pathlib import Path
from typing import TYPE_CHECKING, Literal
from nonebot_plugin_saa import Text, Image, MessageSegmentFactory
from nonebot_plugin_saa import Image, MessageSegmentFactory, Text
from nonebot_bison.theme import Theme, ThemeRenderError
from nonebot_bison.post.protocol import HTMLContentSupport
from nonebot_bison.utils import pic_merge, is_pics_mergable
from nonebot_bison.theme import Theme, ThemeRenderError
from nonebot_bison.utils import is_pics_mergable, pic_merge
if TYPE_CHECKING:
from nonebot_bison.post import Post
@@ -31,7 +32,6 @@ class Ht2iTheme(Theme):
raise ThemeRenderError(f"渲染文本失败: {e}")
async def render(self, post: "Post"):
md_text = ""
md_text += f"## {post.title}\n\n" if post.title else ""
@@ -50,9 +50,7 @@ class Ht2iTheme(Theme):
else:
rp_content = await rp.get_content()
md_text += (
"> \n> " + rp_content if len(rp_content) < 500 else f"{rp_content[:500]}..." + " \n" # noqa: E501
) # noqa: E501
md_text += "> \n> " + rp_content if len(rp_content) < 500 else f"{rp_content[:500]}..." + " \n"
md_text += "\n\n"
md_text += f"###### 来源: {post.platform.name} {post.nickname or ''}\n"
@@ -68,7 +66,7 @@ class Ht2iTheme(Theme):
if urls:
msgs.append(Text("\n".join(urls)))
pics_group: list[list[str | bytes | Path | BytesIO]] = []
pics_group: list[Sequence[str | bytes | Path | BytesIO]] = []
if post.images:
pics_group.append(post.images)
if rp and rp.images:
+4 -4
View File
@@ -1,14 +1,14 @@
from typing import TYPE_CHECKING
from abc import ABC, abstractmethod
from typing import TYPE_CHECKING
from nonebot import logger, require
from pydantic import BaseModel, PrivateAttr
from nonebot_plugin_saa import MessageSegmentFactory
from pydantic import BaseModel, PrivateAttr
from ..plugin_config import plugin_config
from nonebot_bison.plugin_config import plugin_config
if TYPE_CHECKING:
from ..post.abstract_post import AbstractPost
from nonebot_bison.post.abstract_post import AbstractPost
class Theme(ABC, BaseModel):
+2 -2
View File
@@ -1,10 +1,10 @@
from base64 import b64encode
from io import BytesIO
from pathlib import Path
from base64 import b64encode
from qrcode import constants
from qrcode.main import QRCode
from qrcode.image.pil import PilImage
from qrcode.main import QRCode
def convert_to_qr(data: str, **kwarg) -> bytes:
+3 -3
View File
@@ -1,10 +1,10 @@
from datetime import time
from dataclasses import dataclass
from typing import Any, Literal, NewType, NamedTuple
from datetime import time
from typing import Any, Literal, NamedTuple, NewType
from httpx import URL
from pydantic import BaseModel
from nonebot_plugin_saa import PlatformTarget as SendTarget
from pydantic import BaseModel
RawPost = Any
Target = NewType("Target", str)
+15 -13
View File
@@ -1,29 +1,31 @@
import difflib
import re
import sys
import difflib
from typing import Any, ClassVar
import nonebot
from nonebot.plugin import require
from bs4 import BeautifulSoup as bs
from nonebot.log import logger, default_format
from nonebot_plugin_saa import Text, Image, MessageSegmentFactory
import nonebot
from nonebot.log import default_format, logger
from nonebot.plugin import require
from nonebot_plugin_saa import Image, MessageSegmentFactory, Text
from .site import Site as Site
from ..plugin_config import plugin_config
from .image import pic_merge as pic_merge
from nonebot_bison.plugin_config import plugin_config
from .context import ProcessContext as ProcessContext
from .http import http_client as http_client
from .image import capture_html as capture_html
from .site import ClientManager as ClientManager
from .image import text_to_image as text_to_image
from .site import anonymous_site as anonymous_site
from .context import ProcessContext as ProcessContext
from .image import is_pics_mergable as is_pics_mergable
from .image import pic_merge as pic_merge
from .image import pic_url_to_image as pic_url_to_image
from .image import text_to_image as text_to_image
from .site import ClientManager as ClientManager
from .site import DefaultClientManager as DefaultClientManager
from .site import Site as Site
from .site import anonymous_site as anonymous_site
class Singleton(type):
_instances = {}
_instances: ClassVar[dict[Any, Any]] = {}
def __call__(cls, *args, **kwargs):
if cls not in cls._instances:
+1 -1
View File
@@ -1,6 +1,6 @@
from base64 import b64encode
from httpx import Response, AsyncClient
from httpx import AsyncClient, Response
from nonebot_bison.types import Target
+2 -2
View File
@@ -1,12 +1,12 @@
"""提供获取 Bot 的方法"""
from typing import Any
from collections import defaultdict
from typing import Any
import nonebot
from nonebot.adapters import Bot
from nonebot_plugin_saa import PlatformTarget
from nonebot.adapters.onebot.v11 import Bot as Ob11Bot
from nonebot_plugin_saa import PlatformTarget
GROUP: dict[int, list[Bot]] = {}
USER: dict[int, list[Bot]] = {}
+2 -2
View File
@@ -1,9 +1,9 @@
import httpx
from ..plugin_config import plugin_config
from nonebot_bison.plugin_config import plugin_config
http_args = {
"proxies": plugin_config.bison_proxy or None,
"proxy": plugin_config.bison_proxy or None,
}
http_headers = {"user-agent": plugin_config.bison_ua}
+8 -7
View File
@@ -1,16 +1,17 @@
from io import BytesIO
from collections.abc import Sequence
from functools import partial
from io import BytesIO
from typing import Literal, TypeGuard
from yarl import URL
from PIL import Image
from httpx import AsyncClient
from nonebot import logger, require
from PIL.Image import Image as PILImage
from nonebot_plugin_saa import Text as SaaText
from nonebot_plugin_saa import Image as SaaImage
from nonebot_plugin_saa import Text as SaaText
from PIL import Image
from PIL.Image import Image as PILImage
from yarl import URL
from ..plugin_config import plugin_config
from nonebot_bison.plugin_config import plugin_config
async def pic_url_to_image(data: str | bytes, http_client: AsyncClient) -> PILImage:
@@ -96,7 +97,7 @@ async def pic_merge(pics: list[str | bytes], http_client: AsyncClient) -> list[s
return pics
def is_pics_mergable(imgs: list) -> TypeGuard[list[str | bytes]]:
def is_pics_mergable(imgs: Sequence) -> TypeGuard[list[str | bytes]]:
if any(not isinstance(img, str | bytes) for img in imgs):
return False
+8 -9
View File
@@ -1,19 +1,19 @@
import json
from typing import Literal
from json import JSONDecodeError
from abc import ABC, abstractmethod
from collections.abc import Callable
from datetime import datetime, timedelta
import json
from json import JSONDecodeError
from typing import Literal
import httpx
from httpx import AsyncClient
from nonebot.log import logger
from ..types import Target
from ..config import config
from nonebot_bison.config import config
from nonebot_bison.config.db_model import Cookie
from nonebot_bison.types import Target
from .http import http_client
from ..config.db_model import Cookie
from ..metrics import cookie_choose_counter
class ClientManager(ABC):
@@ -132,7 +132,6 @@ class CookieClientManager(ClientManager):
"""获取 client,根据 target 选择 cookie"""
client = http_client()
cookie = await self._choose_cookie(target)
cookie_choose_counter.labels(site_name=self._site_name, target=target, cookie_id=cookie.id).inc()
if cookie.is_universal:
logger.trace(f"平台 {self._site_name} 未获取到用户cookie, 使用匿名cookie")
else:
@@ -186,7 +185,7 @@ class SiteMeta(type):
elif not kwargs.get("abstract"):
# this is the subclass
if "name" in namespace:
site_manager[namespace["name"]] = cls
site_manager[namespace["name"]] = cls # type: ignore[reportArgumentType]
super().__init__(name, bases, namespace, **kwargs)

Some files were not shown because too many files have changed in this diff Show More