Compare commits
10 Commits
a9104eb6d5
...
9f4ff8fdc6
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9f4ff8fdc6 | ||
|
|
ca252c2981 | ||
|
|
4a4fd0ae60 | ||
|
|
e987d0876a | ||
|
|
fa4653f028 | ||
|
|
6e78f1a13e | ||
|
|
5307913a01 | ||
|
|
8ba2887f46 | ||
|
|
3d16f81c88 | ||
|
|
4127a4c618 |
31
.gitignore
vendored
31
.gitignore
vendored
@@ -1,7 +1,30 @@
|
|||||||
|
# 运行产物
|
||||||
results/
|
results/
|
||||||
__pycache__/
|
|
||||||
*.pyc
|
|
||||||
*.log
|
|
||||||
downloaded_files/
|
downloaded_files/
|
||||||
latest_logs/
|
latest_logs/
|
||||||
batchmortal_bench_*/
|
batchmortal_bench_*/
|
||||||
|
|
||||||
|
# Python
|
||||||
|
__pycache__/
|
||||||
|
*.pyc
|
||||||
|
*.pyo
|
||||||
|
.venv/
|
||||||
|
venv/
|
||||||
|
env/
|
||||||
|
.pytest_cache/
|
||||||
|
*.egg-info/
|
||||||
|
build/
|
||||||
|
dist/
|
||||||
|
|
||||||
|
# 个人配置(含玩家昵称等私有信息,不提交)
|
||||||
|
config.yaml
|
||||||
|
config.yml
|
||||||
|
config.toml
|
||||||
|
sstpp.yaml
|
||||||
|
|
||||||
|
# 编辑器 / 系统
|
||||||
|
.DS_Store
|
||||||
|
.idea/
|
||||||
|
.vscode/
|
||||||
|
*.swp
|
||||||
|
*.swo
|
||||||
|
|||||||
235
README.md
235
README.md
@@ -1,14 +1,16 @@
|
|||||||
# Batch Mortal Analysis
|
# Batch Mortal Analysis
|
||||||
|
|
||||||
`batchmortal` 是一个基于 Python 和 SeleniumBase 的批量牌谱分析脚本。它会从 `amae-koromo` 拉取雀魂对局记录,再自动打开 `mjai.ekyu.moe` 提交牌谱、等待分析完成,并把结果导出为 `xlsx` 或 `csv`。
|
`batchmortal` 是一个批量牌谱分析脚本:从 `amae-koromo` 获取雀魂对局,或从 `nodocchi.moe` 获取天凤四人半庄对局,再通过 SeleniumBase 将牌谱提交到 `mjai.ekyu.moe`,最后导出 CSV/XLSX 结果与可选图表。
|
||||||
|
|
||||||
## 环境要求
|
## 环境要求
|
||||||
|
|
||||||
- Python 3.8+
|
- Python 3.10+
|
||||||
- Google Chrome
|
- Google Chrome
|
||||||
- 可访问目标站点的网络环境
|
- 能够访问所选数据源及 `mjai.ekyu.moe` 的网络环境
|
||||||
|
|
||||||
安装依赖:
|
支持 macOS、Linux(含 Fedora)和 Windows。无头模式下不需要图形界面,也不需要任何系统辅助功能/屏幕录制权限——Turnstile 通过浏览器指纹自动放行,不再有模拟点击。Fedora 安装 Chrome:`sudo dnf install google-chrome-stable`(需先启用第三方仓库)或从官网下载 rpm。
|
||||||
|
|
||||||
|
安装:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/myouo/batchmortal.git
|
git clone https://github.com/myouo/batchmortal.git
|
||||||
@@ -16,154 +18,149 @@ cd batchmortal
|
|||||||
pip install -r requirements.txt
|
pip install -r requirements.txt
|
||||||
```
|
```
|
||||||
|
|
||||||
## 基本用法
|
## 推荐:使用配置文件
|
||||||
|
|
||||||
|
项目提供完整示例配置 [`config.example.yaml`](config.example.yaml)。先编辑其中的 `mode` 和对应玩家 ID,再运行:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python main.py -p <玩家昵称> [选项]
|
python main.py --config config.example.yaml
|
||||||
# 或者
|
|
||||||
python main.py -a <数字账号ID> [选项]
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### 使用配置文件(推荐)
|
建议先开启 `dry_run: true`,确认玩家、模式和提取出的牌谱链接正确;确认后改回 `false` 进行 Mortal 分析。
|
||||||
|
|
||||||
你可以通过配置文件来简化命令行输入。在项目根目录下创建一个 `config.yaml` 或 `config.toml` 文件(或参考 [`config.default.yaml`](config.default.yaml))。
|
### 数据源配置
|
||||||
|
|
||||||
示例 `config.yaml`:
|
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
nickname: "言乾"
|
# 同时只能选择一个数据源:mj/0 为雀魂,th/1 为天凤
|
||||||
limit: 10
|
mode: "th"
|
||||||
modes: "12"
|
|
||||||
review_language: "zh-CN"
|
# mode: mj 时只读取这一段
|
||||||
headless: true
|
mj:
|
||||||
save_screenshot: true
|
nickname: ""
|
||||||
save_local_paipu: false
|
# account_id: 12345678
|
||||||
output: "xlsx"
|
limit: 10
|
||||||
|
modes: "12"
|
||||||
|
|
||||||
|
# mode: th 时只读取这一段
|
||||||
|
th:
|
||||||
|
nickname: ""
|
||||||
|
limit: 10
|
||||||
|
modes: "4p-south"
|
||||||
```
|
```
|
||||||
|
|
||||||
带配置文件的运行方式:
|
`mj:` 与 `th:` 可以同时保存两套参数,但每次运行只读取顶层 `mode` 选中的一段:
|
||||||
```bash
|
|
||||||
python main.py
|
|
||||||
```
|
|
||||||
*(你也可以通过 `python main.py --config my_config.yaml` 手动指定配置文件位置。命令行传入的参数会覆盖配置文件中的同名设置。)*
|
|
||||||
|
|
||||||
常见示例(不使用配置文件):
|
- `mode`:数据源开关,推荐填写 `mj` 或 `th`;也兼容 `0` 或 `1`。
|
||||||
|
- `modes`:当前数据源内部的对局类型筛选,与顶层 `mode` 含义不同;天凤固定使用 `4p-south`。
|
||||||
|
- 雀魂可使用 `nickname`,也可使用数字 `account_id`;天凤必须使用 `nickname`。
|
||||||
|
- `limit` 按实际对局模式限制获取数量。
|
||||||
|
|
||||||
|
### 对局模式
|
||||||
|
|
||||||
|
| 数据源 | `modes` 示例 | 说明 |
|
||||||
|
| :--- | :--- | :--- |
|
||||||
|
| 雀魂 `mj` | `9,12,16` | amae-koromo 的数字模式 ID,例如 9 四人金南、12 四人玉南、16 四人王座南 |
|
||||||
|
| 天凤 `th` | `4p-south` | 仅接受四人半庄(四麻南场)牌谱 |
|
||||||
|
|
||||||
|
天凤的 `4p-south` 是本项目根据 Nodocchi 返回的 `playernum` 和 `playlength` 生成的统一筛选名,并非 Nodocchi API 直接返回的字符串。三麻和东风牌谱不在本项目的天凤分析范围内。
|
||||||
|
|
||||||
|
### 常用公共配置
|
||||||
|
|
||||||
|
完整字段和注释请直接查看 [`config.example.yaml`](config.example.yaml),常用字段包括:
|
||||||
|
|
||||||
|
| 配置项 | 作用 |
|
||||||
|
| :--- | :--- |
|
||||||
|
| `review_language` | 分析页面语言:`zh-CN`, `en`, `ja`, `ko` |
|
||||||
|
| `review_ui` | 结果页样式:`classic` 或 `killerducky`;默认保持 `classic` 兼容性 |
|
||||||
|
| `model_tag` | Mortal 模型版本 |
|
||||||
|
| `headless` | 是否无界面运行浏览器,默认 `true`;调试时可设 `false` |
|
||||||
|
| `proxy` | 代理 URL;设为 `"direct"` 强制浏览器直连、绕过系统代理(mjai.ekyu.moe 的 Turnstile 在机房代理 IP 下会拒绝自动放行,需直连) |
|
||||||
|
| `dry_run` | 只提取并打印牌谱链接,不启动浏览器 |
|
||||||
|
| `retry` | 单条牌谱失败后的重试次数 |
|
||||||
|
| `analyze_bad_move_rate` | 是否统计 5%/10% 两档恶手率 |
|
||||||
|
| `save_screenshot` | 是否保存分析结果截图 |
|
||||||
|
| `save_local_paipu` | 是否保存 Mortal 结果页 HTML |
|
||||||
|
| `output` | `csv` 或 `xlsx` |
|
||||||
|
| `plot` | `none`, `html`, `png`, `both` |
|
||||||
|
| `plot_limit` | 图表只使用最近 N 条结果;不填表示全部 |
|
||||||
|
|
||||||
|
## 命令行覆盖
|
||||||
|
|
||||||
|
配置文件是推荐入口;临时参数可以在命令行中覆盖:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python main.py -p 言乾 --modes 12 --limit 1 --headless --save-screenshot --output xlsx
|
# 使用配置文件,但临时切换到天凤并只提取链接
|
||||||
|
python main.py --config config.example.yaml --mode th -p ププリン --modes 4p-south --limit 10 --dry-run
|
||||||
|
|
||||||
|
# 使用配置文件,但临时分析指定雀魂玩家
|
||||||
|
python main.py --config config.example.yaml --mode mj -p 言乾 --modes 12 --limit 10
|
||||||
```
|
```
|
||||||
|
|
||||||
## 参数说明
|
主要参数:
|
||||||
|
|
||||||
### Target Options (目标参数)
|
| 参数 | 说明 |
|
||||||
| 参数 | 默认值 | 说明 |
|
| :--- | :--- |
|
||||||
| :--- | :--- | :--- |
|
| `--config` | 指定 YAML/TOML 配置文件 |
|
||||||
| `-p`, `--player` | 无 | 目标雀魂玩家昵称(必须提供 `-p` 或 `-a`) |
|
| `--mode` | 唯一数据源:`mj`/`0` 或 `th`/`1` |
|
||||||
| `-a`, `--account-id`| 无 | 直接指定玩家数字账号 ID。当与 `-p` 同时提供时,实际拉取以 `-a` 为准,但程序的目标名称仍沿用 `-p` 提供的昵称。若仅提供 `-a`,程序会通过 API 尝试获取真实昵称。 |
|
| `-p`, `--player` | 当前数据源的玩家昵称 |
|
||||||
|
| `-a`, `--account-id` | 雀魂数字账号 ID;天凤不支持 |
|
||||||
|
| `--modes` | 逗号分隔的对局模式 |
|
||||||
|
| `--limit` | 每个实际模式最多获取的记录数 |
|
||||||
|
| `--review-ui` | 结果页样式:`classic` 或 `killerducky` |
|
||||||
|
| `--dry-run` | 只打印牌谱 URL |
|
||||||
|
| `--headless` / `--no-headless` | 强制开启/关闭无头模式(默认开) |
|
||||||
|
| `--proxy` | 代理 URL,或 `direct` 强制直连绕过系统代理 |
|
||||||
|
| `--badmove` | 开启恶手率统计 |
|
||||||
|
| `--save-local` | 保存 Mortal 结果页 HTML |
|
||||||
|
| `--save-screenshot` | 保存结果截图 |
|
||||||
|
| `--plot` | 生成 HTML/PNG 图表 |
|
||||||
|
|
||||||
### Analysis Options (分析参数)
|
KillerDucky 页面将 Rating 和 AI 一致率显示在 About 中。项目实际从该页面引用的
|
||||||
| 参数 | 默认值 | 说明 |
|
`/report/*.json` 结构化数据读取这些字段;开启 `analyze_bad_move_rate` 后,也会根据每个
|
||||||
| :--- | :--- | :--- |
|
决策的 `actual_index` 与实际选择概率计算 5%/10% 恶手率。两种 UI 的恶手率口径一致。
|
||||||
| `--limit` | `10` | 每个 mode 最多拉取多少条记录 |
|
|
||||||
| `--modes` | `9` | 逗号分隔的 mode 列表,例如 `9(四人金南),12(四人玉南),16(四人王座南)` |
|
|
||||||
| `--model-tag` | `4.1b` | Mortal 分析模型版本 |
|
|
||||||
| `--review-language`, `--lang` | `zh-CN` | 跑谱页面语言,会写入 `mjai.ekyu.moe` 表单字段 `select[name="lang"]`。可选 `zh-CN`, `en`, `ja`, `ko` |
|
|
||||||
| `--retry` | `3` | 失败条目的重试次数。每次重试都会重新打开分析页并重新提交 |
|
|
||||||
| `--badmove` | `False` | 额外统计恶手率。开启后会在日志和结果文件中写入 5% 与 10% 两档恶手率 |
|
|
||||||
|
|
||||||
### Browser & Network Options (浏览器与网络)
|
旧参数 `--source majsoul|tenhou` 仍可兼容使用,但不能和 `--mode` 同时出现;新配置统一推荐 `mode: mj|th`。
|
||||||
| 参数 | 默认值 | 说明 |
|
|
||||||
| :--- | :--- | :--- |
|
|
||||||
| `--headless` | `False` | 后台无界面运行浏览器(强烈推荐) |
|
|
||||||
| `--proxy` | 系统代理 | 指定浏览器代理;不传时尝试自动读取系统代理 |
|
|
||||||
|
|
||||||
### Output Options (输出与绘图)
|
## 浏览器提交模式
|
||||||
| 参数 | 默认值 | 说明 |
|
|
||||||
| :--- | :--- | :--- |
|
|
||||||
| `--output` | `xlsx` | 导出格式,可选 `xlsx` 或 `csv`(默认xlsx) |
|
|
||||||
| `--plot` | `none` | 生成折线图:`none`, `html`, `png`, 或 `both`。默认不生成 |
|
|
||||||
| `--save-screenshot` | `False` | 保存分析结果页面截图(举报时可用) |
|
|
||||||
| `--save-local` | `False` | 保存每条 Mortal 分析结果页的本地 HTML,并在结果文件中记录路径 |
|
|
||||||
|
|
||||||
### Advanced Submission Options (高级配置)
|
|
||||||
| 参数 | 默认值 | 说明 |
|
|
||||||
| :--- | :--- | :--- |
|
|
||||||
| `--unsafe-parallel-review` | `False` | 允许并发提交 review。理论上更快,但在单代理环境下通常易失败(已弃用,不推荐) |
|
|
||||||
| `--submit-interval` | `6` | 受控模式下,两次提交之间的最小间隔秒数(已弃用,不推荐) |
|
|
||||||
| `--submit-cooldown` | `30` | 受控模式下,连续失败后的冷却秒数(已弃用,不推荐) |
|
|
||||||
| `--prewarm-standby` | `False` | 实验功能。使用两个持久窗口轮流接力,每次只让当前焦点窗口完整处理一条任务(推荐尝试) |
|
|
||||||
|
|
||||||
### General / Legacy Options (通用及历史参数)
|
|
||||||
| 参数 | 默认值 | 说明 |
|
|
||||||
| :--- | :--- | :--- |
|
|
||||||
| `--config` | 无 | 指定配置文件路径(支持 yaml/toml),命令行参数会覆盖配置项 |
|
|
||||||
| `--dry-run` | `False` | 只拉取并打印牌谱 URL,不启动浏览器 |
|
|
||||||
| `--no-manual-verification` | `False` | 兼容旧脚本保留参数,当前无作用(已弃用) |
|
|
||||||
| `--flare-url` | 无 | 兼容旧脚本保留参数,当前无作用(已弃用) |
|
|
||||||
|
|
||||||
## 运行模式建议
|
|
||||||
|
|
||||||
- 默认模式:单窗口串行,当前最稳。
|
|
||||||
- `--prewarm-standby`:实验功能。现在表示“双窗口轮转”,不是后台抢跑预热;在单代理环境下不保证比默认串行更快,但值得一试。
|
|
||||||
- `--unsafe-parallel-review`:不推荐在单系统代理环境下使用,通常会增加 Cloudflare/Turnstile 等待和失败率。
|
|
||||||
|
|
||||||
推荐先从默认模式开始:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python main.py -p 言乾 --limit 10 --modes 16 --headless
|
|
||||||
```
|
|
||||||
|
|
||||||
如果你要测试实验性的双窗口轮转:
|
|
||||||
|
|
||||||
注意:这是实验路径,建议只在你已经拿到默认串行基线后再做对比测试。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python main.py -p 言乾 --limit 10 --modes 16 --headless --prewarm-standby
|
|
||||||
```
|
|
||||||
|
|
||||||
## 绘图模块 (--plot)
|
|
||||||
|
|
||||||
使用 `--plot html` 或 `both` 将会在产出数据后调用底层的可视化引擎生成 `report_<nickname>.html`。
|
|
||||||
|
|
||||||
示例:
|
|
||||||
```bash
|
|
||||||
python main.py -p "main()" --limit 20 --modes 16 --headless --retry 3 --badmove --prewarm-standby --save-screenshot --plot both
|
|
||||||
```
|
|
||||||
|
|
||||||

|
|
||||||
|
|
||||||
|
- 默认:单个持久浏览器串行处理,使用提交间隔和失败冷却,稳定性最好。
|
||||||
|
- `prewarm_standby: true`:使用两个持久窗口轮流处理任务;仍是受控提交,不保证更快。
|
||||||
|
- `unsafe_parallel_review: true`:绕过受控提交协调,不代表真正的多线程并发,可能更容易触发 Turnstile 或限流。
|
||||||
|
- `submit_interval`:受控模式下两次提交的最小间隔秒数。
|
||||||
|
- `submit_cooldown`:连续提交失败后的冷却秒数。
|
||||||
|
|
||||||
## 输出目录
|
## 输出目录
|
||||||
|
|
||||||
结果默认写入:
|
雀魂与天凤结果按来源并列保存:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
results/<nickname>/
|
results/
|
||||||
|
├── majsoul/<nickname>/
|
||||||
|
└── tenhou/<nickname>/
|
||||||
```
|
```
|
||||||
|
|
||||||
常见产物包括:
|
常见文件:
|
||||||
|
|
||||||
- `results/<nickname>/results.xlsx` 或 `results/<nickname>/results.csv`
|
- `results.xlsx` 或 `results.csv`
|
||||||
- `results/<nickname>/mode_<id>/<uuid>.png`
|
- `mode_<id>/<uuid>.png`
|
||||||
- `results/<nickname>/mode_<id>/<uuid>_error.png`
|
- `mode_<id>/<uuid>_error.png`
|
||||||
- `results/<nickname>/mode_<id>/<uuid>.html`
|
- `mode_<id>/<uuid>.html`
|
||||||
|
- `report_<nickname>.html` / `report_<nickname>.png`
|
||||||
|
|
||||||
## 日志
|
导出结果包含 `source` 字段,用于标识 `majsoul` 或 `tenhou`。天凤模式目录示例为 `mode_4p-south`。
|
||||||
|
|
||||||
运行日志会在每条输出前附带当前系统时间,便于定位慢点和错误发生时刻。
|
可视化报告包含关键指标卡、Rating 单半庄值与半庄移动平均、按决策数加权的 AI 一致率、Rating 分布和低 Rating 牌谱检讨入口。缺失指标显示为 `—`,不会按 0 计入图表或汇总。
|
||||||
|
|
||||||
## 说明
|
从旧版本升级时,请将原有的 `results/<nickname>/` 雀魂目录移动到 `results/majsoul/<nickname>/`;否则程序无法从新目录识别以前已经处理的牌谱。
|
||||||
|
|
||||||
- `xlsx` 写入逻辑已做批量化优化,但整体耗时通常主要由浏览器提交、Cloudflare Turnstile 和远端分析生成决定。
|
## 注意事项
|
||||||
- `--badmove` 只会在新分析的牌谱中统计恶手率;已经成功写入结果文件的牌谱会继续跳过,不会为了补恶手率自动重跑。
|
|
||||||
- `--save-local` 只会保存新分析的结果页;已经成功写入结果文件的牌谱会继续跳过,不会为了补本地 HTML 自动重跑。
|
|
||||||
- 在只有一个系统代理的情况下,多窗口或多线程通常不会线性提速。
|
|
||||||
|
|
||||||
## 下一步
|
- Nodocchi 返回的 `tw` 是压缩座位排列;脚本会解码目标玩家视角,并且只接受 `tenhou.net` 正式牌谱 URL。
|
||||||
|
- Nodocchi 中没有 `url` 或 `tw` 的历史统计记录不会进入 Mortal 分析队列。
|
||||||
- 计划增加常见 id 与`match_id`的对应词典,以满足大家喜爱的主播/牌手改名而自己不愿意去翻的情景,如:{"火龙果说电影" : "8888621"}(主播名称与`account_id`的对应) 以便大家使用。
|
- 已成功写入结果文件的牌谱会跳过;失败记录仍可在后续运行中重试。
|
||||||
- anything in issue...?
|
- `--badmove`、本地 HTML 和截图只会应用于新执行的分析,不会自动重跑已经成功的牌谱。
|
||||||
|
- 总耗时通常取决于浏览器提交、Cloudflare Turnstile 和远端分析生成速度。
|
||||||
|
- Turnstile 通过无头浏览器指纹自动放行,不做模拟点击、不需要系统辅助权限;少数情况会卡住,脚本会先重置控件再重开页面,仍失败才走 `retry`。机房代理出口 IP 会被 Turnstile 触发交互挑战,需用 `proxy: "direct"` 走直连。
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
|
|||||||
@@ -9,6 +9,7 @@ from seleniumbase import SB
|
|||||||
|
|
||||||
REVIEW_BASE_URL = "https://mjai.ekyu.moe"
|
REVIEW_BASE_URL = "https://mjai.ekyu.moe"
|
||||||
DEFAULT_REVIEW_LANGUAGE = "zh-CN"
|
DEFAULT_REVIEW_LANGUAGE = "zh-CN"
|
||||||
|
DEFAULT_REVIEW_UI = "classic"
|
||||||
|
|
||||||
# Mirrors the reviewer form field `select[name="lang"]`.
|
# Mirrors the reviewer form field `select[name="lang"]`.
|
||||||
# Supported values on mjai.ekyu.moe are:
|
# Supported values on mjai.ekyu.moe are:
|
||||||
@@ -45,6 +46,153 @@ REPORT_URL_FRAGMENT = "/report/"
|
|||||||
BAD_MOVE_STRICT_LIMIT = 5
|
BAD_MOVE_STRICT_LIMIT = 5
|
||||||
BAD_MOVE_LOOSE_LIMIT = 10
|
BAD_MOVE_LOOSE_LIMIT = 10
|
||||||
|
|
||||||
|
REVIEW_UI_ALIASES = {
|
||||||
|
"classic": "classic",
|
||||||
|
"killerducky": "killerducky",
|
||||||
|
"killer-ducky": "killerducky",
|
||||||
|
"kd": "killerducky",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_review_ui(review_ui):
|
||||||
|
if review_ui is None:
|
||||||
|
return DEFAULT_REVIEW_UI
|
||||||
|
|
||||||
|
key = str(review_ui).strip().lower().replace("_", "-")
|
||||||
|
normalized = REVIEW_UI_ALIASES.get(key)
|
||||||
|
if normalized:
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
raise ValueError(
|
||||||
|
f"Unsupported review UI '{review_ui}'. Supported values: classic, killerducky"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
PROXY_DIRECT = "direct"
|
||||||
|
|
||||||
|
|
||||||
|
def build_browser_kwargs(headless: bool, proxy: str | None) -> dict:
|
||||||
|
"""Translate headless/proxy into ``SB()`` kwargs.
|
||||||
|
|
||||||
|
A proxy value of ``"direct"`` (case-insensitive) forces Chrome to bypass
|
||||||
|
every proxy — including the macOS system proxy that Clash/etc. configure —
|
||||||
|
by passing ``--proxy-server=direct://``. This is required for mjai.ekyu.moe:
|
||||||
|
Cloudflare Turnstile flags datacenter proxy exit IPs and refuses to
|
||||||
|
auto-issue a token, so the browser must reach the site from the residential
|
||||||
|
IP. SeleniumBase's ``proxy=`` param cannot express this because its
|
||||||
|
``validate_proxy_string`` rejects ``direct://`` and falls back to None,
|
||||||
|
which leaves Chrome on the system proxy.
|
||||||
|
"""
|
||||||
|
kwargs = {"uc": True, "headless": headless}
|
||||||
|
if proxy and str(proxy).strip().lower() == PROXY_DIRECT:
|
||||||
|
kwargs["chromium_arg"] = "--proxy-server=direct://"
|
||||||
|
elif proxy:
|
||||||
|
kwargs["proxy"] = proxy
|
||||||
|
return kwargs
|
||||||
|
|
||||||
|
|
||||||
|
def parse_killerducky_metadata(data):
|
||||||
|
"""Convert KillerDucky's report JSON into the metadata shape used by results.py."""
|
||||||
|
data = data if isinstance(data, dict) else {}
|
||||||
|
review = data.get("review")
|
||||||
|
review = review if isinstance(review, dict) else {}
|
||||||
|
|
||||||
|
total_matches = review.get("total_matches")
|
||||||
|
total_reviewed = review.get("total_reviewed")
|
||||||
|
matches_total = ""
|
||||||
|
if (
|
||||||
|
isinstance(total_matches, (int, float))
|
||||||
|
and isinstance(total_reviewed, (int, float))
|
||||||
|
and total_reviewed > 0
|
||||||
|
):
|
||||||
|
rate = 100 * total_matches / total_reviewed
|
||||||
|
matches_total = f"{int(total_matches)}/{int(total_reviewed)} = {rate:.3f}%"
|
||||||
|
|
||||||
|
rating = review.get("rating")
|
||||||
|
formatted_rating = ""
|
||||||
|
if isinstance(rating, (int, float)):
|
||||||
|
formatted_rating = f"{rating * 100:.3f}"
|
||||||
|
|
||||||
|
def text(value):
|
||||||
|
return "" if value is None else str(value)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"engine": text(data.get("engine")),
|
||||||
|
"model tag": text(review.get("model_tag")),
|
||||||
|
"rating": formatted_rating,
|
||||||
|
"matches/total": matches_total,
|
||||||
|
"temperature": text(review.get("temperature")),
|
||||||
|
"game length": text(data.get("game_length")),
|
||||||
|
"player id": text(data.get("player_id")),
|
||||||
|
"review duration": text(data.get("review_time")),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def parse_killerducky_bad_move_stats(
|
||||||
|
data,
|
||||||
|
strict_limit=BAD_MOVE_STRICT_LIMIT,
|
||||||
|
loose_limit=BAD_MOVE_LOOSE_LIMIT,
|
||||||
|
):
|
||||||
|
"""Calculate bad-move rates from KillerDucky's structured Mortal decisions."""
|
||||||
|
data = data if isinstance(data, dict) else {}
|
||||||
|
review = data.get("review")
|
||||||
|
review = review if isinstance(review, dict) else {}
|
||||||
|
total_reviewed = review.get("total_reviewed")
|
||||||
|
denominator = total_reviewed if isinstance(total_reviewed, int) and total_reviewed >= 0 else None
|
||||||
|
|
||||||
|
strict_count = 0
|
||||||
|
loose_count = 0
|
||||||
|
mismatch_count = 0
|
||||||
|
unparsed_count = 0
|
||||||
|
|
||||||
|
kyokus = review.get("kyokus")
|
||||||
|
for kyoku in kyokus if isinstance(kyokus, list) else []:
|
||||||
|
if not isinstance(kyoku, dict):
|
||||||
|
continue
|
||||||
|
entries = kyoku.get("entries")
|
||||||
|
for entry in entries if isinstance(entries, list) else []:
|
||||||
|
if not isinstance(entry, dict) or entry.get("is_equal") is not False:
|
||||||
|
continue
|
||||||
|
|
||||||
|
mismatch_count += 1
|
||||||
|
actual_index = entry.get("actual_index")
|
||||||
|
details = entry.get("details")
|
||||||
|
if (
|
||||||
|
not isinstance(actual_index, int)
|
||||||
|
or not isinstance(details, list)
|
||||||
|
or actual_index < 0
|
||||||
|
or actual_index >= len(details)
|
||||||
|
or not isinstance(details[actual_index], dict)
|
||||||
|
):
|
||||||
|
unparsed_count += 1
|
||||||
|
continue
|
||||||
|
|
||||||
|
probability = details[actual_index].get("prob")
|
||||||
|
if not isinstance(probability, (int, float)):
|
||||||
|
unparsed_count += 1
|
||||||
|
continue
|
||||||
|
|
||||||
|
probability_percent = probability * 100
|
||||||
|
if probability_percent <= strict_limit:
|
||||||
|
strict_count += 1
|
||||||
|
if probability_percent <= loose_limit:
|
||||||
|
loose_count += 1
|
||||||
|
|
||||||
|
def format_rate(count):
|
||||||
|
if denominator is None or denominator <= 0:
|
||||||
|
return ""
|
||||||
|
return f"{100 * count / denominator:.3f}%"
|
||||||
|
|
||||||
|
return {
|
||||||
|
"badMoveRate5": format_rate(strict_count),
|
||||||
|
"badMoveCount5": str(strict_count),
|
||||||
|
"badMoveRate10": format_rate(loose_count),
|
||||||
|
"badMoveCount10": str(loose_count),
|
||||||
|
"badMoveDenominator": "" if denominator is None else str(denominator),
|
||||||
|
"badMoveOrderLossCount": str(mismatch_count),
|
||||||
|
"badMoveUnparsedCount": str(unparsed_count),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def normalize_review_language(language):
|
def normalize_review_language(language):
|
||||||
if language is None:
|
if language is None:
|
||||||
@@ -178,10 +326,12 @@ class BrowserAutomator:
|
|||||||
submission_coordinator=None,
|
submission_coordinator=None,
|
||||||
controlled_submission=True,
|
controlled_submission=True,
|
||||||
review_language=DEFAULT_REVIEW_LANGUAGE,
|
review_language=DEFAULT_REVIEW_LANGUAGE,
|
||||||
|
review_ui=DEFAULT_REVIEW_UI,
|
||||||
):
|
):
|
||||||
self.headless = headless
|
self.headless = headless
|
||||||
self.proxy = proxy
|
self.proxy = proxy
|
||||||
self.review_language = normalize_review_language(review_language)
|
self.review_language = normalize_review_language(review_language)
|
||||||
|
self.review_ui = normalize_review_ui(review_ui)
|
||||||
self.review_url = build_review_url(self.review_language)
|
self.review_url = build_review_url(self.review_language)
|
||||||
self.controlled_submission = controlled_submission
|
self.controlled_submission = controlled_submission
|
||||||
if controlled_submission:
|
if controlled_submission:
|
||||||
@@ -195,7 +345,7 @@ class BrowserAutomator:
|
|||||||
break
|
break
|
||||||
|
|
||||||
try:
|
try:
|
||||||
with SB(uc=True, headless=self.headless, proxy=self.proxy) as sb:
|
with SB(**build_browser_kwargs(self.headless, self.proxy)) as sb:
|
||||||
tasks_processed = 0
|
tasks_processed = 0
|
||||||
|
|
||||||
while True:
|
while True:
|
||||||
@@ -259,7 +409,7 @@ class BrowserAutomator:
|
|||||||
def iter_alternating_windows(self, tasks, max_retries=3):
|
def iter_alternating_windows(self, tasks, max_retries=3):
|
||||||
pending = deque(tasks)
|
pending = deque(tasks)
|
||||||
|
|
||||||
with SB(uc=True, headless=self.headless, proxy=self.proxy) as sb:
|
with SB(**build_browser_kwargs(self.headless, self.proxy)) as sb:
|
||||||
slots = [
|
slots = [
|
||||||
{
|
{
|
||||||
"name": "window-a",
|
"name": "window-a",
|
||||||
@@ -306,7 +456,7 @@ class BrowserAutomator:
|
|||||||
def iter_dual_window_pipeline(self, tasks, max_retries=3):
|
def iter_dual_window_pipeline(self, tasks, max_retries=3):
|
||||||
pending = deque(tasks)
|
pending = deque(tasks)
|
||||||
|
|
||||||
with SB(uc=True, headless=self.headless, proxy=self.proxy) as sb:
|
with SB(**build_browser_kwargs(self.headless, self.proxy)) as sb:
|
||||||
active_slot = {
|
active_slot = {
|
||||||
"name": "active",
|
"name": "active",
|
||||||
"handle": sb.driver.current_window_handle,
|
"handle": sb.driver.current_window_handle,
|
||||||
@@ -419,10 +569,9 @@ class BrowserAutomator:
|
|||||||
else:
|
else:
|
||||||
logging.info(f"{log_prefix} Unthrottled mode, starting Turnstile")
|
logging.info(f"{log_prefix} Unthrottled mode, starting Turnstile")
|
||||||
self._prepare_review_form(sb)
|
self._prepare_review_form(sb)
|
||||||
self._poke_captcha(sb)
|
|
||||||
|
|
||||||
token_started_at = time.perf_counter()
|
token_started_at = time.perf_counter()
|
||||||
self._wait_for_turnstile_token(sb, log_prefix, timeout=35)
|
self._acquire_turnstile_token(sb, task, log_prefix)
|
||||||
token_wait_seconds = time.perf_counter() - token_started_at
|
token_wait_seconds = time.perf_counter() - token_started_at
|
||||||
logging.info(f"{log_prefix} Turnstile token ready in {token_wait_seconds:.1f}s")
|
logging.info(f"{log_prefix} Turnstile token ready in {token_wait_seconds:.1f}s")
|
||||||
|
|
||||||
@@ -568,7 +717,7 @@ class BrowserAutomator:
|
|||||||
|
|
||||||
logging.info(f"{log_prefix} {slot['name']} tab entering Turnstile")
|
logging.info(f"{log_prefix} {slot['name']} tab entering Turnstile")
|
||||||
token_started_at = time.perf_counter()
|
token_started_at = time.perf_counter()
|
||||||
self._wait_for_turnstile_token(sb, log_prefix, timeout=35)
|
self._acquire_turnstile_token(sb, task, log_prefix)
|
||||||
token_wait_seconds = time.perf_counter() - token_started_at
|
token_wait_seconds = time.perf_counter() - token_started_at
|
||||||
logging.info(f"{log_prefix} Turnstile token ready in {token_wait_seconds:.1f}s")
|
logging.info(f"{log_prefix} Turnstile token ready in {token_wait_seconds:.1f}s")
|
||||||
|
|
||||||
@@ -693,7 +842,6 @@ class BrowserAutomator:
|
|||||||
self._open_fresh_review_page(sb, log_prefix)
|
self._open_fresh_review_page(sb, log_prefix)
|
||||||
self._populate_form(sb, task["paipu_url"], task["model_tag"])
|
self._populate_form(sb, task["paipu_url"], task["model_tag"])
|
||||||
self._prepare_review_form(sb)
|
self._prepare_review_form(sb)
|
||||||
self._poke_captcha(sb)
|
|
||||||
|
|
||||||
def _prime_rotation_slot(self, sb, slot, label):
|
def _prime_rotation_slot(self, sb, slot, label):
|
||||||
if not slot.get("handle"):
|
if not slot.get("handle"):
|
||||||
@@ -774,7 +922,7 @@ class BrowserAutomator:
|
|||||||
if "mjai.ekyu.moe" in current_url:
|
if "mjai.ekyu.moe" in current_url:
|
||||||
sb.execute_script("window.location.replace(arguments[0]);", self.review_url)
|
sb.execute_script("window.location.replace(arguments[0]);", self.review_url)
|
||||||
else:
|
else:
|
||||||
sb.uc_open_with_reconnect(self.review_url, reconnect_time=2)
|
sb.uc_open_with_reconnect(self.review_url, reconnect_time=4)
|
||||||
except Exception:
|
except Exception:
|
||||||
sb.open(self.review_url)
|
sb.open(self.review_url)
|
||||||
|
|
||||||
@@ -826,6 +974,7 @@ class BrowserAutomator:
|
|||||||
const paipuUrl = arguments[0];
|
const paipuUrl = arguments[0];
|
||||||
const modelTag = arguments[1];
|
const modelTag = arguments[1];
|
||||||
const reviewLanguage = arguments[2];
|
const reviewLanguage = arguments[2];
|
||||||
|
const reviewUi = arguments[3];
|
||||||
|
|
||||||
const dispatch = (el) => {
|
const dispatch = (el) => {
|
||||||
el.dispatchEvent(new Event('input', { bubbles: true }));
|
el.dispatchEvent(new Event('input', { bubbles: true }));
|
||||||
@@ -857,7 +1006,7 @@ class BrowserAutomator:
|
|||||||
|
|
||||||
setSelect('select[name="engine"]', 'mortal');
|
setSelect('select[name="engine"]', 'mortal');
|
||||||
setSelect('select[name="mortal-model-tag"]', modelTag);
|
setSelect('select[name="mortal-model-tag"]', modelTag);
|
||||||
setSelect('select[name="ui"]', 'classic');
|
setSelect('select[name="ui"]', reviewUi);
|
||||||
setSelect('select[name="lang"]', reviewLanguage);
|
setSelect('select[name="lang"]', reviewLanguage);
|
||||||
|
|
||||||
const details = document.querySelector('details.details.mb-3');
|
const details = document.querySelector('details.details.mb-3');
|
||||||
@@ -870,7 +1019,7 @@ class BrowserAutomator:
|
|||||||
showRating.click();
|
showRating.click();
|
||||||
}
|
}
|
||||||
|
|
||||||
const form = document.querySelector(arguments[3]);
|
const form = document.querySelector(arguments[4]);
|
||||||
if (form) {
|
if (form) {
|
||||||
form.target = '_self';
|
form.target = '_self';
|
||||||
}
|
}
|
||||||
@@ -880,6 +1029,7 @@ class BrowserAutomator:
|
|||||||
paipu_url,
|
paipu_url,
|
||||||
model_tag,
|
model_tag,
|
||||||
self.review_language,
|
self.review_language,
|
||||||
|
self.review_ui,
|
||||||
FORM_SELECTOR,
|
FORM_SELECTOR,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -887,10 +1037,13 @@ class BrowserAutomator:
|
|||||||
raise RuntimeError("Could not populate review form")
|
raise RuntimeError("Could not populate review form")
|
||||||
|
|
||||||
def _wait_for_turnstile_token(self, sb, log_prefix, timeout):
|
def _wait_for_turnstile_token(self, sb, log_prefix, timeout):
|
||||||
deadline = time.time() + timeout
|
"""Pure poller: returns when cf-turnstile-response has a token.
|
||||||
next_poke_at = time.time() + 8
|
|
||||||
recoveries = 0
|
|
||||||
|
|
||||||
|
No GUI clicks, no widget reset, no page reload. Recovery is handled by
|
||||||
|
the caller (_acquire_turnstile_token) so this method stays a simple
|
||||||
|
timeout-bound poll.
|
||||||
|
"""
|
||||||
|
deadline = time.time() + timeout
|
||||||
while time.time() < deadline:
|
while time.time() < deadline:
|
||||||
state = self._read_review_state(sb)
|
state = self._read_review_state(sb)
|
||||||
if state["token_length"] > 0:
|
if state["token_length"] > 0:
|
||||||
@@ -902,19 +1055,65 @@ class BrowserAutomator:
|
|||||||
):
|
):
|
||||||
raise RuntimeError(f"{log_prefix} Turnstile token was rejected before submission")
|
raise RuntimeError(f"{log_prefix} Turnstile token was rejected before submission")
|
||||||
|
|
||||||
if time.time() >= next_poke_at:
|
|
||||||
recoveries += 1
|
|
||||||
logging.info(f"{log_prefix} Turnstile token still missing, retrying captcha click")
|
|
||||||
self._recover_turnstile_widget(sb)
|
|
||||||
self._poke_captcha(sb)
|
|
||||||
if recoveries >= 2:
|
|
||||||
raise RuntimeError(f"{log_prefix} Turnstile widget stalled before token issuance")
|
|
||||||
next_poke_at = time.time() + 8
|
|
||||||
|
|
||||||
time.sleep(0.5)
|
time.sleep(0.5)
|
||||||
|
|
||||||
raise RuntimeError(f"{log_prefix} Timed out waiting for Turnstile token")
|
raise RuntimeError(f"{log_prefix} Timed out waiting for Turnstile token")
|
||||||
|
|
||||||
|
def _acquire_turnstile_token(
|
||||||
|
self,
|
||||||
|
sb,
|
||||||
|
task,
|
||||||
|
log_prefix,
|
||||||
|
per_attempt_timeout: float = 12.0,
|
||||||
|
max_reloads: int = 2,
|
||||||
|
):
|
||||||
|
"""Obtain a Turnstile token via headless UC-mode auto-pass.
|
||||||
|
|
||||||
|
No GUI clicks and no OS permissions are needed. On stall we first try a
|
||||||
|
cheap in-page JS reset of the Turnstile widget, then fall back to fully
|
||||||
|
reloading the review form (up to ``max_reloads`` times) to give
|
||||||
|
Cloudflare a fresh chance to auto-issue a token. Server-side rejections
|
||||||
|
(invalid/timeout-or-duplicate) are propagated without retry.
|
||||||
|
"""
|
||||||
|
js_reset_done = False
|
||||||
|
reloads = 0
|
||||||
|
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
self._wait_for_turnstile_token(sb, log_prefix, per_attempt_timeout)
|
||||||
|
return
|
||||||
|
except RuntimeError as exc:
|
||||||
|
msg = str(exc).lower()
|
||||||
|
if "rejected" in msg or "rate" in msg:
|
||||||
|
raise
|
||||||
|
|
||||||
|
if not js_reset_done:
|
||||||
|
js_reset_done = True
|
||||||
|
logging.info(f"{log_prefix} Turnstile token missing, resetting widget")
|
||||||
|
try:
|
||||||
|
self._recover_turnstile_widget(sb)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
continue
|
||||||
|
|
||||||
|
if reloads >= max_reloads:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"{log_prefix} Turnstile widget stalled before token issuance"
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
reloads += 1
|
||||||
|
logging.info(
|
||||||
|
f"{log_prefix} Turnstile token still missing, "
|
||||||
|
f"reloading form ({reloads}/{max_reloads})"
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
self._open_fresh_review_page(sb, log_prefix)
|
||||||
|
self._populate_form(sb, task["paipu_url"], task["model_tag"])
|
||||||
|
self._prepare_review_form(sb)
|
||||||
|
except Exception as reload_exc:
|
||||||
|
logging.warning(f"{log_prefix} Form reload failed: {reload_exc}")
|
||||||
|
|
||||||
|
|
||||||
def _submit_review(self, sb, log_prefix):
|
def _submit_review(self, sb, log_prefix):
|
||||||
submitted = sb.execute_script(
|
submitted = sb.execute_script(
|
||||||
"""
|
"""
|
||||||
@@ -971,6 +1170,12 @@ class BrowserAutomator:
|
|||||||
raise RuntimeError(f"{log_prefix} Review submission never left the form page")
|
raise RuntimeError(f"{log_prefix} Review submission never left the form page")
|
||||||
|
|
||||||
def _extract_metadata(self, sb):
|
def _extract_metadata(self, sb):
|
||||||
|
killerducky_data = self._extract_killerducky_data(sb)
|
||||||
|
if killerducky_data:
|
||||||
|
return parse_killerducky_metadata(killerducky_data)
|
||||||
|
if self.review_ui == "killerducky":
|
||||||
|
raise RuntimeError("Could not extract KillerDucky report JSON")
|
||||||
|
|
||||||
metadata = sb.execute_script(
|
metadata = sb.execute_script(
|
||||||
"""
|
"""
|
||||||
const data = {};
|
const data = {};
|
||||||
@@ -987,6 +1192,56 @@ class BrowserAutomator:
|
|||||||
)
|
)
|
||||||
return metadata or {}
|
return metadata or {}
|
||||||
|
|
||||||
|
def _extract_killerducky_data(self, sb, include_entries=False):
|
||||||
|
try:
|
||||||
|
data = sb.execute_script(
|
||||||
|
"""
|
||||||
|
if (!(window.MM && window.MM.GS && window.MM.GS.fullData)) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
const source = window.MM.GS.fullData;
|
||||||
|
const review = source.review || {};
|
||||||
|
const includeEntries = arguments[0];
|
||||||
|
return {
|
||||||
|
engine: source.engine,
|
||||||
|
game_length: source.game_length,
|
||||||
|
review_time: source.review_time,
|
||||||
|
player_id: source.player_id,
|
||||||
|
review: {
|
||||||
|
model_tag: review.model_tag,
|
||||||
|
rating: review.rating,
|
||||||
|
temperature: review.temperature,
|
||||||
|
total_matches: review.total_matches,
|
||||||
|
total_reviewed: review.total_reviewed,
|
||||||
|
kyokus: includeEntries && Array.isArray(review.kyokus)
|
||||||
|
? review.kyokus.map((kyoku) => ({
|
||||||
|
entries: Array.isArray(kyoku.entries) ? kyoku.entries.map((entry) => {
|
||||||
|
const actualIndex = entry.actual_index;
|
||||||
|
const details = Array.isArray(entry.details)
|
||||||
|
? entry.details.map((detail, index) => (
|
||||||
|
index === actualIndex ? {prob: detail.prob} : null
|
||||||
|
))
|
||||||
|
: null;
|
||||||
|
return {
|
||||||
|
is_equal: entry.is_equal,
|
||||||
|
actual_index: actualIndex,
|
||||||
|
details,
|
||||||
|
};
|
||||||
|
}) : [],
|
||||||
|
}))
|
||||||
|
: [],
|
||||||
|
},
|
||||||
|
};
|
||||||
|
""",
|
||||||
|
include_entries,
|
||||||
|
)
|
||||||
|
return data if isinstance(data, dict) else None
|
||||||
|
except Exception as exc:
|
||||||
|
if self.review_ui == "killerducky":
|
||||||
|
raise RuntimeError("Could not read KillerDucky report JSON") from exc
|
||||||
|
return None
|
||||||
|
|
||||||
def _save_local_paipu(self, sb, filepath, source_url, log_prefix):
|
def _save_local_paipu(self, sb, filepath, source_url, log_prefix):
|
||||||
os.makedirs(os.path.dirname(filepath), exist_ok=True)
|
os.makedirs(os.path.dirname(filepath), exist_ok=True)
|
||||||
html = sb.execute_script(
|
html = sb.execute_script(
|
||||||
@@ -1031,6 +1286,10 @@ class BrowserAutomator:
|
|||||||
|
|
||||||
def _extract_bad_move_stats(self, sb, log_prefix):
|
def _extract_bad_move_stats(self, sb, log_prefix):
|
||||||
try:
|
try:
|
||||||
|
killerducky_data = self._extract_killerducky_data(sb, include_entries=True)
|
||||||
|
if killerducky_data:
|
||||||
|
return parse_killerducky_bad_move_stats(killerducky_data)
|
||||||
|
|
||||||
stats = sb.execute_script(
|
stats = sb.execute_script(
|
||||||
"""
|
"""
|
||||||
const strictLimit = arguments[0];
|
const strictLimit = arguments[0];
|
||||||
@@ -1139,7 +1398,22 @@ class BrowserAutomator:
|
|||||||
def _wait_for_result_or_error(self, sb, log_prefix, timeout):
|
def _wait_for_result_or_error(self, sb, log_prefix, timeout):
|
||||||
deadline = time.time() + timeout
|
deadline = time.time() + timeout
|
||||||
while time.time() < deadline:
|
while time.time() < deadline:
|
||||||
if sb.is_element_present(RESULT_SELECTOR):
|
if self.review_ui == "killerducky":
|
||||||
|
try:
|
||||||
|
killerducky_ready = sb.execute_script(
|
||||||
|
"""
|
||||||
|
return !!(
|
||||||
|
window.MM && window.MM.GS && window.MM.GS.fullData
|
||||||
|
&& window.MM.GS.fullData.review
|
||||||
|
&& document.querySelector('.about-metadata table')
|
||||||
|
);
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
killerducky_ready = False
|
||||||
|
if killerducky_ready:
|
||||||
|
return
|
||||||
|
elif sb.is_element_present(RESULT_SELECTOR):
|
||||||
return
|
return
|
||||||
|
|
||||||
state = self._read_review_state(sb)
|
state = self._read_review_state(sb)
|
||||||
@@ -1174,6 +1448,18 @@ class BrowserAutomator:
|
|||||||
|
|
||||||
def _expand_metadata_panel(self, sb, log_prefix):
|
def _expand_metadata_panel(self, sb, log_prefix):
|
||||||
try:
|
try:
|
||||||
|
if self.review_ui == "killerducky":
|
||||||
|
sb.execute_script(
|
||||||
|
"""
|
||||||
|
const modal = document.getElementById('about-modal');
|
||||||
|
if (modal && !modal.open) {
|
||||||
|
modal.showModal();
|
||||||
|
}
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
time.sleep(0.5)
|
||||||
|
return
|
||||||
|
|
||||||
is_open = sb.execute_script(
|
is_open = sb.execute_script(
|
||||||
"""
|
"""
|
||||||
const details = document.querySelector('body > details:nth-child(6)');
|
const details = document.querySelector('body > details:nth-child(6)');
|
||||||
@@ -1186,12 +1472,6 @@ class BrowserAutomator:
|
|||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logging.warning(f"{log_prefix} Could not expand metadata menu: {exc}")
|
logging.warning(f"{log_prefix} Could not expand metadata menu: {exc}")
|
||||||
|
|
||||||
def _poke_captcha(self, sb):
|
|
||||||
try:
|
|
||||||
sb.uc_gui_click_captcha()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
def _recover_turnstile_widget(self, sb):
|
def _recover_turnstile_widget(self, sb):
|
||||||
try:
|
try:
|
||||||
sb.execute_script(
|
sb.execute_script(
|
||||||
|
|||||||
@@ -1,6 +1,79 @@
|
|||||||
import os
|
import os
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
|
|
||||||
|
MODE_TO_SOURCE = {
|
||||||
|
"mj": "majsoul",
|
||||||
|
"th": "tenhou",
|
||||||
|
}
|
||||||
|
SOURCE_MODE_ALIASES = {
|
||||||
|
"mj": "mj",
|
||||||
|
"majsoul": "mj",
|
||||||
|
"mahjong-soul": "mj",
|
||||||
|
"0": "mj",
|
||||||
|
"th": "th",
|
||||||
|
"tenhou": "th",
|
||||||
|
"1": "th",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_source_mode(value) -> str:
|
||||||
|
"""Normalize source selectors to the canonical config modes ``mj`` or ``th``."""
|
||||||
|
if value is None:
|
||||||
|
return "mj"
|
||||||
|
|
||||||
|
key = str(value).strip().lower().replace("_", "-")
|
||||||
|
mode = SOURCE_MODE_ALIASES.get(key)
|
||||||
|
if not mode:
|
||||||
|
raise ValueError(
|
||||||
|
f"Unsupported source mode '{value}'. Use 'mj' for Mahjong Soul or 'th' for Tenhou."
|
||||||
|
)
|
||||||
|
return mode
|
||||||
|
|
||||||
|
|
||||||
|
def source_for_mode(mode) -> str:
|
||||||
|
return MODE_TO_SOURCE[normalize_source_mode(mode)]
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_mode_config(config: dict, requested_mode=None) -> tuple[str, str, dict]:
|
||||||
|
"""
|
||||||
|
Select exactly one source-specific config section.
|
||||||
|
|
||||||
|
New configs use ``mode: mj|th`` with ``mj:`` and ``th:`` mappings. The old
|
||||||
|
top-level ``source: majsoul|tenhou`` format remains readable for backward
|
||||||
|
compatibility. Defining both selectors is rejected so the active source is
|
||||||
|
always unambiguous.
|
||||||
|
"""
|
||||||
|
config = config or {}
|
||||||
|
if not isinstance(config, dict):
|
||||||
|
raise ValueError("Config root must be a mapping/object.")
|
||||||
|
configured_mode = config.get("mode")
|
||||||
|
legacy_source = config.get("source")
|
||||||
|
|
||||||
|
if configured_mode is not None and legacy_source is not None:
|
||||||
|
raise ValueError(
|
||||||
|
"Config defines both 'mode' and legacy 'source'. "
|
||||||
|
"Keep exactly one selector: 'mode: mj' or 'mode: th'."
|
||||||
|
)
|
||||||
|
|
||||||
|
selector = requested_mode
|
||||||
|
if selector is None:
|
||||||
|
selector = configured_mode if configured_mode is not None else legacy_source
|
||||||
|
mode = normalize_source_mode(selector)
|
||||||
|
|
||||||
|
section = config.get(mode, {})
|
||||||
|
if section is None:
|
||||||
|
section = {}
|
||||||
|
if not isinstance(section, dict):
|
||||||
|
raise ValueError(f"Config section '{mode}' must be a mapping/object.")
|
||||||
|
|
||||||
|
effective_config = dict(config)
|
||||||
|
effective_config.update(section)
|
||||||
|
if mode == "th" and "modes" not in section and "tenhou_modes" in config:
|
||||||
|
effective_config["modes"] = config["tenhou_modes"]
|
||||||
|
return mode, MODE_TO_SOURCE[mode], effective_config
|
||||||
|
|
||||||
|
|
||||||
def load_config(config_path: str = None) -> dict:
|
def load_config(config_path: str = None) -> dict:
|
||||||
"""
|
"""
|
||||||
Load configuration from a YAML or TOML file.
|
Load configuration from a YAML or TOML file.
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import openpyxl
|
|||||||
|
|
||||||
CSV_COLUMNS = [
|
CSV_COLUMNS = [
|
||||||
"nickname",
|
"nickname",
|
||||||
|
"source",
|
||||||
"mode",
|
"mode",
|
||||||
"uuid",
|
"uuid",
|
||||||
"paipuUrl",
|
"paipuUrl",
|
||||||
@@ -46,7 +47,7 @@ def parse_metadata(metadata: dict) -> dict:
|
|||||||
return value
|
return value
|
||||||
return ""
|
return ""
|
||||||
|
|
||||||
ai_consistency = get(["\u4e00\u81f4\u7387", "Match Rate"])
|
ai_consistency = get(["\u4e00\u81f4\u7387", "Match Rate", "Matches/total"])
|
||||||
numerator, denominator, rate = "", "", ""
|
numerator, denominator, rate = "", "", ""
|
||||||
|
|
||||||
# Matches strings like "195/271 = 71.956%".
|
# Matches strings like "195/271 = 71.956%".
|
||||||
|
|||||||
306
batchmortal/tenhou.py
Normal file
306
batchmortal/tenhou.py
Normal file
@@ -0,0 +1,306 @@
|
|||||||
|
import logging
|
||||||
|
import re
|
||||||
|
import urllib.parse
|
||||||
|
from collections import defaultdict
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
|
||||||
|
import requests
|
||||||
|
|
||||||
|
|
||||||
|
NODOCCHI_API_URL = "https://nodocchi.moe/api/listuser.php"
|
||||||
|
REQUEST_HEADERS = {"Accept": "application/json"}
|
||||||
|
SESSION = requests.Session()
|
||||||
|
|
||||||
|
TENHOU_MODE_ALIASES = {
|
||||||
|
"all": "all",
|
||||||
|
"*": "all",
|
||||||
|
"4": "4p",
|
||||||
|
"4p": "4p",
|
||||||
|
"yonma": "4p",
|
||||||
|
"四麻": "4p",
|
||||||
|
"4e": "4p-east",
|
||||||
|
"4p-east": "4p-east",
|
||||||
|
"四东": "4p-east",
|
||||||
|
"四東": "4p-east",
|
||||||
|
"4s": "4p-south",
|
||||||
|
"4p-south": "4p-south",
|
||||||
|
"四南": "4p-south",
|
||||||
|
"3": "3p",
|
||||||
|
"3p": "3p",
|
||||||
|
"sanma": "3p",
|
||||||
|
"三麻": "3p",
|
||||||
|
"3e": "3p-east",
|
||||||
|
"3p-east": "3p-east",
|
||||||
|
"三东": "3p-east",
|
||||||
|
"三東": "3p-east",
|
||||||
|
"3s": "3p-south",
|
||||||
|
"3p-south": "3p-south",
|
||||||
|
"三南": "3p-south",
|
||||||
|
}
|
||||||
|
TENHOU_MODE_CHOICES = (
|
||||||
|
"all",
|
||||||
|
"4p",
|
||||||
|
"4p-east",
|
||||||
|
"4p-south",
|
||||||
|
"3p",
|
||||||
|
"3p-east",
|
||||||
|
"3p-south",
|
||||||
|
)
|
||||||
|
TENHOU_LOG_ID_RE = re.compile(
|
||||||
|
r"^\d{10}gm-[0-9a-z]{4}-[0-9a-z]+-[0-9a-z]{8}$",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def fetch_tenhou_player_records(player_name: str) -> tuple[str, list[dict]]:
|
||||||
|
"""Fetch a Tenhou player's records from nodocchi.moe."""
|
||||||
|
try:
|
||||||
|
response = SESSION.get(
|
||||||
|
NODOCCHI_API_URL,
|
||||||
|
params={"name": player_name},
|
||||||
|
timeout=30,
|
||||||
|
headers=REQUEST_HEADERS,
|
||||||
|
)
|
||||||
|
response.raise_for_status()
|
||||||
|
data = response.json()
|
||||||
|
except Exception as exc:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"Nodocchi API request failed while searching for '{player_name}': {exc}"
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
raise ValueError("Unexpected Nodocchi API response: expected an object.")
|
||||||
|
|
||||||
|
retry_after = data.get("retry")
|
||||||
|
if retry_after:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"Nodocchi API asked the client to retry after {retry_after} seconds."
|
||||||
|
)
|
||||||
|
|
||||||
|
records = data.get("list")
|
||||||
|
if not isinstance(records, list):
|
||||||
|
raise ValueError("Unexpected Nodocchi API response: missing 'list' field.")
|
||||||
|
if not records:
|
||||||
|
raise ValueError(f"Tenhou player not found or has no records: '{player_name}'.")
|
||||||
|
|
||||||
|
resolved_name = data.get("name")
|
||||||
|
if not isinstance(resolved_name, str) or not resolved_name:
|
||||||
|
resolved_name = player_name
|
||||||
|
|
||||||
|
logging.info(
|
||||||
|
"[Nodocchi] Found Tenhou player '%s' with %d records.",
|
||||||
|
resolved_name,
|
||||||
|
len(records),
|
||||||
|
)
|
||||||
|
return resolved_name, records
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_tenhou_modes(value: str | list[str] | tuple[str, ...]) -> tuple[str, ...]:
|
||||||
|
if isinstance(value, str):
|
||||||
|
raw_modes = value.split(",")
|
||||||
|
else:
|
||||||
|
raw_modes = value
|
||||||
|
|
||||||
|
normalized = []
|
||||||
|
for raw_mode in raw_modes:
|
||||||
|
key = str(raw_mode).strip().lower().replace("_", "-")
|
||||||
|
if not key:
|
||||||
|
continue
|
||||||
|
mode = TENHOU_MODE_ALIASES.get(key)
|
||||||
|
if not mode:
|
||||||
|
supported = ", ".join(TENHOU_MODE_CHOICES)
|
||||||
|
raise ValueError(
|
||||||
|
f"Unsupported Tenhou mode '{raw_mode}'. Supported values: {supported}"
|
||||||
|
)
|
||||||
|
if mode not in normalized:
|
||||||
|
normalized.append(mode)
|
||||||
|
|
||||||
|
if not normalized:
|
||||||
|
return ("all",)
|
||||||
|
if "all" in normalized:
|
||||||
|
return ("all",)
|
||||||
|
return tuple(normalized)
|
||||||
|
|
||||||
|
|
||||||
|
def tenhou_record_mode(record: dict) -> str:
|
||||||
|
try:
|
||||||
|
player_num = int(record.get("playernum"))
|
||||||
|
play_length = int(record.get("playlength"))
|
||||||
|
except (TypeError, ValueError) as exc:
|
||||||
|
raise ValueError("Tenhou record is missing a valid playernum/playlength.") from exc
|
||||||
|
|
||||||
|
if player_num not in (3, 4) or play_length not in (1, 2):
|
||||||
|
raise ValueError(
|
||||||
|
f"Unsupported Tenhou record mode: playernum={player_num}, playlength={play_length}."
|
||||||
|
)
|
||||||
|
|
||||||
|
length_name = {1: "east", 2: "south"}[play_length]
|
||||||
|
return f"{player_num}p-{length_name}"
|
||||||
|
|
||||||
|
|
||||||
|
def tenhou_mode_matches(record_mode: str, selected_modes: tuple[str, ...]) -> bool:
|
||||||
|
if "all" in selected_modes:
|
||||||
|
return True
|
||||||
|
return any(
|
||||||
|
record_mode == selected_mode
|
||||||
|
or (selected_mode in ("3p", "4p") and record_mode.startswith(f"{selected_mode}-"))
|
||||||
|
for selected_mode in selected_modes
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def find_player_order(record: dict, player_name: str) -> int | None:
|
||||||
|
try:
|
||||||
|
player_num = int(record.get("playernum"))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
for order in range(1, player_num + 1):
|
||||||
|
if record.get(f"player{order}") == player_name:
|
||||||
|
return order
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def decode_tenhou_viewpoint(encoded_tw, player_order: int | None) -> int | None:
|
||||||
|
"""
|
||||||
|
Decode Nodocchi's packed seat permutation.
|
||||||
|
|
||||||
|
This mirrors the site's frontend: each two-bit group stores the result
|
||||||
|
order for one Tenhou seat. The seat whose value equals order - 1 is the
|
||||||
|
requested player's ``tw`` value.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
packed = int(encoded_tw)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
if player_order not in (1, 2, 3, 4):
|
||||||
|
return None
|
||||||
|
|
||||||
|
expected_order = player_order - 1
|
||||||
|
for seat in range(4):
|
||||||
|
if (packed >> (seat * 2)) & 3 == expected_order:
|
||||||
|
return seat
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def parse_tenhou_log_url(url: str) -> tuple[str, str] | None:
|
||||||
|
"""Return ``(log_id, canonical_base_url)`` for an official Tenhou log URL."""
|
||||||
|
if not isinstance(url, str) or not url:
|
||||||
|
return None
|
||||||
|
|
||||||
|
parsed = urllib.parse.urlsplit(url)
|
||||||
|
if parsed.scheme not in ("http", "https"):
|
||||||
|
return None
|
||||||
|
if (parsed.hostname or "").lower() not in ("tenhou.net", "www.tenhou.net"):
|
||||||
|
return None
|
||||||
|
if parsed.path not in ("/0", "/0/"):
|
||||||
|
return None
|
||||||
|
|
||||||
|
log_values = urllib.parse.parse_qs(parsed.query).get("log", [])
|
||||||
|
if len(log_values) != 1 or not TENHOU_LOG_ID_RE.fullmatch(log_values[0]):
|
||||||
|
return None
|
||||||
|
|
||||||
|
log_id = log_values[0]
|
||||||
|
canonical_host = (parsed.hostname or "tenhou.net").lower()
|
||||||
|
canonical_url = urllib.parse.urlunsplit(
|
||||||
|
(
|
||||||
|
parsed.scheme,
|
||||||
|
canonical_host,
|
||||||
|
"/0/",
|
||||||
|
urllib.parse.urlencode({"log": log_id}),
|
||||||
|
"",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return log_id, canonical_url
|
||||||
|
|
||||||
|
|
||||||
|
def format_tenhou_timestamp(timestamp) -> str:
|
||||||
|
try:
|
||||||
|
timestamp = int(timestamp)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return ""
|
||||||
|
if timestamp <= 0:
|
||||||
|
return ""
|
||||||
|
return (
|
||||||
|
datetime.fromtimestamp(timestamp, timezone.utc)
|
||||||
|
.astimezone()
|
||||||
|
.strftime("%Y-%m-%d %H:%M:%S")
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def build_tenhou_paipu_urls(
|
||||||
|
records: list[dict],
|
||||||
|
player_name: str,
|
||||||
|
modes: str | list[str] | tuple[str, ...] = "all",
|
||||||
|
limit: int | None = None,
|
||||||
|
) -> list[dict]:
|
||||||
|
"""Build recent, player-focused Tenhou paipu URLs from Nodocchi records."""
|
||||||
|
selected_modes = normalize_tenhou_modes(modes)
|
||||||
|
per_mode_count: defaultdict[str, int] = defaultdict(int)
|
||||||
|
results = []
|
||||||
|
seen_log_ids = set()
|
||||||
|
skipped_without_url = 0
|
||||||
|
skipped_invalid = 0
|
||||||
|
|
||||||
|
def start_time(record):
|
||||||
|
try:
|
||||||
|
return int(record.get("starttime", 0))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return 0
|
||||||
|
|
||||||
|
for record in sorted(records, key=start_time, reverse=True):
|
||||||
|
try:
|
||||||
|
mode = tenhou_record_mode(record)
|
||||||
|
except ValueError:
|
||||||
|
skipped_invalid += 1
|
||||||
|
continue
|
||||||
|
if not tenhou_mode_matches(mode, selected_modes):
|
||||||
|
continue
|
||||||
|
if limit is not None and per_mode_count[mode] >= limit:
|
||||||
|
continue
|
||||||
|
|
||||||
|
raw_url = record.get("url")
|
||||||
|
if not raw_url:
|
||||||
|
skipped_without_url += 1
|
||||||
|
continue
|
||||||
|
parsed_url = parse_tenhou_log_url(raw_url)
|
||||||
|
if not parsed_url:
|
||||||
|
skipped_invalid += 1
|
||||||
|
continue
|
||||||
|
|
||||||
|
player_order = find_player_order(record, player_name)
|
||||||
|
viewpoint = decode_tenhou_viewpoint(record.get("tw"), player_order)
|
||||||
|
if viewpoint is None:
|
||||||
|
skipped_invalid += 1
|
||||||
|
continue
|
||||||
|
|
||||||
|
log_id, base_url = parsed_url
|
||||||
|
if log_id in seen_log_ids:
|
||||||
|
continue
|
||||||
|
|
||||||
|
start_timestamp = start_time(record)
|
||||||
|
try:
|
||||||
|
duration_seconds = max(0, int(record.get("during", 0))) * 60
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
duration_seconds = 0
|
||||||
|
|
||||||
|
results.append(
|
||||||
|
{
|
||||||
|
"source": "tenhou",
|
||||||
|
"mode": mode,
|
||||||
|
"uuid": log_id,
|
||||||
|
"paipuUrl": f"{base_url}&tw={viewpoint}",
|
||||||
|
"startTime": format_tenhou_timestamp(start_timestamp),
|
||||||
|
"endTime": format_tenhou_timestamp(start_timestamp + duration_seconds),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
seen_log_ids.add(log_id)
|
||||||
|
per_mode_count[mode] += 1
|
||||||
|
|
||||||
|
logging.info(
|
||||||
|
"[Nodocchi] Built %d Tenhou links (unavailable=%d, invalid=%d).",
|
||||||
|
len(results),
|
||||||
|
skipped_without_url,
|
||||||
|
skipped_invalid,
|
||||||
|
)
|
||||||
|
return results
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -3,15 +3,25 @@
|
|||||||
# 或者直接在主目录下放置名为 config.yaml 或 config.toml 的文件
|
# 或者直接在主目录下放置名为 config.yaml 或 config.toml 的文件
|
||||||
# 命令行传入的参数会覆盖这里的配置
|
# 命令行传入的参数会覆盖这里的配置
|
||||||
|
|
||||||
# 目标雀魂玩家昵称(必须提供,可以在命令行中通过位置参数指定,也可以在此处指定)
|
# 数据源模式,同时只能选择一个:
|
||||||
nickname: ""
|
# mj(或 0) -> 雀魂 / amae-koromo
|
||||||
|
# th(或 1) -> 天凤 / nodocchi.moe
|
||||||
|
mode: "th"
|
||||||
|
|
||||||
# 每个 mode 最多拉取多少条记录 (默认: 10)
|
# 雀魂配置;仅在 mode: "mj" 时读取
|
||||||
limit: 10
|
mj:
|
||||||
|
nickname: ""
|
||||||
|
# account_id: 12345678 # 可用数字账号 ID 替代 nickname
|
||||||
|
limit: 10
|
||||||
|
# 常见 modes: 9(四人金南), 12(四人玉南), 16(四人王座南)
|
||||||
|
modes: "12"
|
||||||
|
|
||||||
# 逗号分隔的 mode 列表 (默认: "9")
|
# 天凤配置;仅在 mode: "th" 时读取
|
||||||
# 常见 mode: 9(四人金南), 12(四人玉南), 16(四人王座南)
|
th:
|
||||||
modes: "12"
|
nickname: ""
|
||||||
|
limit: 10
|
||||||
|
# 天凤仅接受四人半庄(四麻南场),固定使用 4p-south
|
||||||
|
modes: "4p-south"
|
||||||
|
|
||||||
# Mortal 分析模型版本 (默认: "4.1b")
|
# Mortal 分析模型版本 (默认: "4.1b")
|
||||||
model_tag: "4.1b"
|
model_tag: "4.1b"
|
||||||
@@ -26,8 +36,13 @@ model_tag: "4.1b"
|
|||||||
# ko -> 한국어,页面 /ko.html
|
# ko -> 한국어,页面 /ko.html
|
||||||
review_language: "zh-CN"
|
review_language: "zh-CN"
|
||||||
|
|
||||||
|
# 分析结果页面样式:
|
||||||
|
# killerducky -> KillerDucky 交互界面;从页面引用的结构化 JSON 读取 Rating、AI 一致率和恶手率
|
||||||
|
# classic -> 经典长页面;保留原有 DOM 解析方式
|
||||||
|
review_ui: "killerducky"
|
||||||
|
|
||||||
# 是否额外分析恶手率。默认关闭,开启后会统计 5% 和 10% 两档恶手率
|
# 是否额外分析恶手率。默认关闭,开启后会统计 5% 和 10% 两档恶手率
|
||||||
analyze_bad_move_rate: false
|
analyze_bad_move_rate: true
|
||||||
|
|
||||||
# 是否在后台以无头模式运行浏览器 (强烈推荐 true)
|
# 是否在后台以无头模式运行浏览器 (强烈推荐 true)
|
||||||
headless: true
|
headless: true
|
||||||
@@ -36,10 +51,10 @@ headless: true
|
|||||||
dry_run: false
|
dry_run: false
|
||||||
|
|
||||||
# 是否保存分析结果页面的截图 (举报时有用)
|
# 是否保存分析结果页面的截图 (举报时有用)
|
||||||
save_screenshot: true
|
save_screenshot: false
|
||||||
|
|
||||||
# 是否保存 Mortal 分析结果页面到本地 HTML
|
# 是否保存 Mortal 分析结果页面到本地 HTML
|
||||||
save_local_paipu: false
|
save_local_paipu: true
|
||||||
|
|
||||||
# 导出格式,可选 "xlsx" 或 "csv" (默认: "xlsx")
|
# 导出格式,可选 "xlsx" 或 "csv" (默认: "xlsx")
|
||||||
output: "xlsx"
|
output: "xlsx"
|
||||||
@@ -48,17 +63,21 @@ output: "xlsx"
|
|||||||
retry: 3
|
retry: 3
|
||||||
|
|
||||||
# 指定浏览器的代理服务器,例如 "http://127.0.0.1:7890"
|
# 指定浏览器的代理服务器,例如 "http://127.0.0.1:7890"
|
||||||
|
# 设为 "direct" 可强制浏览器直连、绕过系统代理(含 Clash 设置的系统代理)。
|
||||||
|
# mjai.ekyu.moe 的 Cloudflare Turnstile 会对机房代理出口 IP 触发交互挑战、
|
||||||
|
# 拒绝自动放行;走直连(住宅 IP)才能在无头下自动拿到 token。
|
||||||
# 如果留空或注释掉,会自动尝试检测系统代理
|
# 如果留空或注释掉,会自动尝试检测系统代理
|
||||||
# proxy: "http://127.0.0.1:7890"
|
# proxy: "http://127.0.0.1:7890"
|
||||||
|
# proxy: "direct"
|
||||||
|
|
||||||
# 实验性功能:双窗口接力机制 (默认: false)
|
# 双窗口接力机制 (默认: false)
|
||||||
prewarm_standby: true
|
prewarm_standby: true
|
||||||
|
|
||||||
# 危险的高并发请求,容易导致Cloudflare卡盾拦截 (弃用,推荐置空或保留为false)
|
# 跳过受控提交协调;不代表真正并发,可能更容易触发 Cloudflare/Turnstile
|
||||||
unsafe_parallel_review: false
|
unsafe_parallel_review: false
|
||||||
|
|
||||||
# 绘图选项,支持 "none", "html", "png", "both" (默认: "none")
|
# 绘图选项,支持 "none", "html", "png", "both" (默认: "none")
|
||||||
plot: "none"
|
plot: "both"
|
||||||
|
|
||||||
# 画图时取最近多少条数据 (默认: 全部)
|
# 画图时取最近多少条数据 (默认: 全部)
|
||||||
# 设置后仅取最新的 N 条数据用于绘图
|
# 设置后仅取最新的 N 条数据用于绘图
|
||||||
127
docs/headless-turnstile-troubleshooting.md
Normal file
127
docs/headless-turnstile-troubleshooting.md
Normal file
@@ -0,0 +1,127 @@
|
|||||||
|
# 无头化与 Turnstile 排查记录
|
||||||
|
|
||||||
|
本文记录将 batchmortal 改造为无头模式、移除模拟点击、并解决 Cloudflare Turnstile 自动放行问题的完整排查过程。供后续维护或迁移到其他平台时参考。
|
||||||
|
|
||||||
|
## 背景
|
||||||
|
|
||||||
|
原实现通过 SeleniumBase 的 `sb.uc_gui_click_captcha()` 用 `pyautogui` 物理点击 Cloudflare Turnstile 复选框来获取 token。这条路径有三个硬伤:
|
||||||
|
|
||||||
|
1. 必须授予 macOS 辅助功能(Accessibility)+ 屏幕录制(Screen Recording)权限。
|
||||||
|
2. 无头模式下没有可点击的 GUI,根本无法工作。
|
||||||
|
3. `pyautogui` 会接管鼠标,运行期间不能并行使用电脑。
|
||||||
|
|
||||||
|
目标:无头、无点击、无系统权限依赖。
|
||||||
|
|
||||||
|
## 排查过程
|
||||||
|
|
||||||
|
### 第一步:确认服务器是否强制校验 Turnstile
|
||||||
|
|
||||||
|
直接 `curl -X POST https://mjai.ekyu.moe/review`(不带 token):
|
||||||
|
|
||||||
|
```
|
||||||
|
HTTP/2 403
|
||||||
|
invalid captcha response
|
||||||
|
```
|
||||||
|
|
||||||
|
结论:HTTP 层绕不开 Turnstile,必须拿到合法 token。表单 POST 到 `/review`,token 字段是 `cf-turnstile-response`,由 Cloudflare 的 `turnstile/v0/api.js` 在浏览器端生成。
|
||||||
|
|
||||||
|
### 第二步:无头 + 自动放行(初步方案)
|
||||||
|
|
||||||
|
删掉所有 `_poke_captcha` / `uc_gui_click_captcha` 调用,把 `_wait_for_turnstile_token` 改成纯轮询器,新增 `_acquire_turnstile_token` 做恢复(先 JS 重置 `window.turnstile.reset()`,再最多 2 次重开页面)。`_open_fresh_review_page` 的 `uc_open_with_reconnect` 的 `reconnect_time` 从 2 调到 4,给 Turnstile JS 更多时间在断开驱动期间自动下发 token。
|
||||||
|
|
||||||
|
实跑 `--limit 2`(无头 + 系统代理开着):**100% 失败**。每次都 12s 轮询无 token → JS 重置 → 重开页面 → 仍无 token → stall。6 次重试全挂。
|
||||||
|
|
||||||
|
### 第三步:隔离变量——是“无头被识破”还是“代理被风控”?
|
||||||
|
|
||||||
|
跑 `--no-headless --limit 1`(有头 + 同一代理):**同样 100% 失败**,模式完全一致。
|
||||||
|
|
||||||
|
结论:**不是无头的问题**。有头无头在代理下都挂。最大嫌疑变成代理出口 IP 被 Cloudflare 风控(机房 IP 触发交互挑战,而非 managed 模式自动放行)。
|
||||||
|
|
||||||
|
### 第四步:测直连
|
||||||
|
|
||||||
|
`curl --noproxy '*' https://mjai.ekyu.moe/` 返回 HTTP 200,1.1s,Cloudflare 边缘 IP。站点直连可达,代理并非访问必需。
|
||||||
|
|
||||||
|
让用户关掉 Clash 系统代理,跑 `--limit 1` 无头直连:
|
||||||
|
|
||||||
|
```
|
||||||
|
[Proxy] No system proxy detected, running directly.
|
||||||
|
[1/1][...] Submit slot granted, starting Turnstile
|
||||||
|
[1/1][...] Turnstile token missing, resetting widget
|
||||||
|
[1/1][...] Turnstile token still missing, reloading form (1/2)
|
||||||
|
[1/1][...] Turnstile token ready in 26.8s ← 成功
|
||||||
|
[1/1][...] OK rating=89.869 match=80.420%
|
||||||
|
```
|
||||||
|
|
||||||
|
**直连下无头自动放行成功**。根因确认:代理出口 IP 被 Turnstile 风控。
|
||||||
|
|
||||||
|
### 第五步:自动化直连——`proxy: "direct"`
|
||||||
|
|
||||||
|
不能要求用户每次手动关 Clash 系统代理。需要让浏览器强制直连、绕过系统代理。
|
||||||
|
|
||||||
|
#### 为什么不能直接用 SeleniumBase 的 `proxy=` 参数
|
||||||
|
|
||||||
|
SeleniumBase 的 `validate_proxy_string`(`seleniumbase/core/proxy_helper.py`)只接受 `ip:port` 或 `scheme://host:port` 格式,`"direct://"` 通不过校验,会被置为 `None`。而 `proxy=None` 意味着不传 `--proxy-server` 给 Chrome —— Chrome 就回退到系统代理,反而走回 Clash。所以“不传 proxy”不等于“直连”。
|
||||||
|
|
||||||
|
#### 解法:用 `chromium_arg` 注入 `--proxy-server=direct://`
|
||||||
|
|
||||||
|
Chrome 的 `--proxy-server=direct://` 会强制忽略所有代理(含系统代理),用 `direct`(无代理)scheme 直连。SeleniumBase 的 `SB()` 有 `chromium_arg` 参数可以塞额外 Chrome 启动参数。
|
||||||
|
|
||||||
|
在 `batchmortal/browser.py` 加模块级 helper:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def build_browser_kwargs(headless, proxy):
|
||||||
|
kwargs = {"uc": True, "headless": headless}
|
||||||
|
if proxy and str(proxy).strip().lower() == "direct":
|
||||||
|
kwargs["chromium_arg"] = "--proxy-server=direct://"
|
||||||
|
elif proxy:
|
||||||
|
kwargs["proxy"] = proxy
|
||||||
|
return kwargs
|
||||||
|
```
|
||||||
|
|
||||||
|
`main.py` 的 `detect_proxy` 识别 `"direct"` 关键字(大小写不敏感)返回 `"direct"`,所有 `SB()` 调用统一走 `build_browser_kwargs`。
|
||||||
|
|
||||||
|
#### 验证
|
||||||
|
|
||||||
|
Clash 系统代理开着 + `proxy: "direct"` + 无头 + 全量 40 条:**38/38 成功,0 失败**。
|
||||||
|
|
||||||
|
- 36/38 局 token 秒出(0.0~2.1s)—— 浏览器会话热起来后指纹/cookie 被信任
|
||||||
|
- 2/38 局 stall(冷启动 + 中途偶发),都经 JS 重置 + 1 次重开页面恢复,没有任何一局需要外层 `retry`
|
||||||
|
- 全程零点击、零系统权限弹窗、无头、Clash 开着
|
||||||
|
|
||||||
|
## 关键代码位置
|
||||||
|
|
||||||
|
- `batchmortal/browser.py:build_browser_kwargs` —— proxy→SB kwargs 翻译,`"direct"` 走 `chromium_arg`
|
||||||
|
- `batchmortal/browser.py:_acquire_turnstile_token` —— 纯轮询 + JS 重置 + 重开页面恢复,无点击
|
||||||
|
- `batchmortal/browser.py:_wait_for_turnstile_token` —— 纯轮询器(检测 token / 检测 `invalid captcha response` / 超时)
|
||||||
|
- `main.py:detect_proxy` —— 识别 `"direct"` 关键字
|
||||||
|
- `install_uc_driver.py:_platform_archive_info` —— 跨平台 driver 包选择
|
||||||
|
|
||||||
|
## 平台兼容性(Fedora/Linux)
|
||||||
|
|
||||||
|
核心运行路径跨平台:`detect_proxy` 用 `urllib.request.getproxies()`(Linux 读环境变量/GSettings),`browser.py` 全是 JS + SeleniumBase 调用,文件路径用 `os.path.join`,`uc_name` 按 `os.name` 分流。无头模式不需要 X server。
|
||||||
|
|
||||||
|
**唯一需要修的是 `install_uc_driver.py`**:原版硬编码 Windows 的 `chromedriver-win64.zip` / `chromedriver.exe`,在 Linux 上会下载 Windows 二进制。改成按平台选 archive 和二进制名:
|
||||||
|
|
||||||
|
| 平台 | archive | 二进制 | uc_driver 名 |
|
||||||
|
| :--- | :--- | :--- | :--- |
|
||||||
|
| Windows | `chromedriver-win64.zip` | `chromedriver.exe` | `uc_driver.exe` |
|
||||||
|
| Linux | `chromedriver-linux64.zip` | `chromedriver` | `uc_driver`(需 `chmod 755`) |
|
||||||
|
| macOS | `chromedriver-mac-{arm64,x64}.zip` | `chromedriver` | `uc_driver` |
|
||||||
|
|
||||||
|
Chrome 版本检测也加了 `google-chrome-stable`(Fedora/Debian 常见名)和 mac app 路径回退。旧版 Chrome(<115)在 npmmirror 没有 Linux/Mac 包,返回 None 交给 SeleniumBase 自带下载兜底。
|
||||||
|
|
||||||
|
## 常见问题速查
|
||||||
|
|
||||||
|
| 症状 | 原因 | 解决 |
|
||||||
|
| :--- | :--- | :--- |
|
||||||
|
| Turnstile 每局都 stall、100% 失败 | 浏览器走了机房代理 IP | 配 `proxy: "direct"` 走直连 |
|
||||||
|
| 直连下偶发 stall(~5%) | Turnstile 偶尔不出 token | 内置恢复会自动 JS 重置 + 重开页面,无需干预 |
|
||||||
|
| `proxy: "direct"` 不生效、仍走代理 | Clash 用 TUN 模式(网络层拦截) | 切回系统代理模式,或临时关 Clash;TUN 模式下 Chrome 的 `--proxy-server` 绕不过 |
|
||||||
|
| 无头下 token 一直拿不到 | 先确认是不是代理问题(有头也测一次) | 有头无头都挂 = 代理问题;仅无头挂 = 指纹问题 |
|
||||||
|
| Fedora 上 uc_driver 不可执行 | 旧版 install_uc_driver 下了 Windows 包 | 已修;或直接删 `seleniumbase/drivers/uc_driver*` 让 SB 自带下载 |
|
||||||
|
|
||||||
|
## 注意事项
|
||||||
|
|
||||||
|
- **TUN 模式陷阱**:如果 Clash 用 TUN 模式(而非系统代理模式),`--proxy-server=direct://` 绕不过网络层拦截,`proxy: "direct"` 会失效。此时只能切回系统代理模式或关 Clash。判断方法:`scutil --proxy` 显示 `HTTPEnable: 0` 但网络仍走代理,多半是 TUN。
|
||||||
|
- **断点续跑**:`get_processed_uuids` 会跳过已成功写入的 UUID,所以即使中途网络抖动或被 Turnstile 卡住导致中断,重跑会从未处理的牌谱继续,不会重复分析。
|
||||||
|
- **不要把 `proxy: "direct"` 和真正的代理混用**:`"direct"` 是关键字,不是 URL。配了 `"direct"` 就不会再走任何代理。
|
||||||
@@ -19,28 +19,64 @@ def get_chrome_version():
|
|||||||
major_version = int(full_version.split('.')[0])
|
major_version = int(full_version.split('.')[0])
|
||||||
return major_version, full_version
|
return major_version, full_version
|
||||||
except Exception:
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
# Linux / Mac 回退方案
|
||||||
|
candidates = [
|
||||||
|
["google-chrome", "--version"],
|
||||||
|
["google-chrome-stable", "--version"],
|
||||||
|
["/Applications/Google Chrome.app/Contents/MacOS/Google Chrome", "--version"],
|
||||||
|
]
|
||||||
|
for cmd in candidates:
|
||||||
try:
|
try:
|
||||||
# Linux / Mac (回退方案)
|
output = subprocess.check_output(cmd).decode("utf-8")
|
||||||
output = subprocess.check_output(["google-chrome", "--version"]).decode("utf-8")
|
|
||||||
full_version = re.search(r'([\d\.]+)', output).group(1)
|
full_version = re.search(r'([\d\.]+)', output).group(1)
|
||||||
major_version = int(full_version.split('.')[0])
|
major_version = int(full_version.split('.')[0])
|
||||||
return major_version, full_version
|
return major_version, full_version
|
||||||
except Exception:
|
except Exception:
|
||||||
return None, None
|
continue
|
||||||
|
return None, None
|
||||||
|
|
||||||
|
|
||||||
|
def _platform_archive_info(major_version):
|
||||||
|
"""根据平台和 Chrome 主版本返回 (archive_name, arch_dir, binary_suffix)。
|
||||||
|
|
||||||
|
binary_suffix 为 '.exe' 表示 Windows,'' 表示 Linux/Mac(chromedriver 无后缀)。
|
||||||
|
"""
|
||||||
|
is_windows = os.name == 'nt'
|
||||||
|
is_mac = platform.system() == 'Darwin'
|
||||||
|
|
||||||
|
if major_version >= 115:
|
||||||
|
if is_windows:
|
||||||
|
is_64bit = platform.architecture()[0] == '64bit'
|
||||||
|
return ("chromedriver-win64.zip" if is_64bit else "chromedriver-win32.zip",
|
||||||
|
"win64" if is_64bit else "win32", ".exe")
|
||||||
|
if is_mac:
|
||||||
|
is_arm = platform.machine() in ("arm64", "aarch64")
|
||||||
|
return ("chromedriver-mac-arm64.zip" if is_arm else "chromedriver-mac-x64.zip",
|
||||||
|
"mac-arm64" if is_arm else "mac-x64", "")
|
||||||
|
# Linux
|
||||||
|
return "chromedriver-linux64.zip", "linux64", ""
|
||||||
|
|
||||||
|
# Chrome 114 及以下
|
||||||
|
if is_windows:
|
||||||
|
return "chromedriver_win32.zip", None, ".exe"
|
||||||
|
# 旧版 Chrome 在 npmmirror 上只有 win32 包,Linux/Mac 无对应旧版包;
|
||||||
|
# 让上层用 None 触发 "未找到" 路径,交给 SeleniumBase 自带下载兜底
|
||||||
|
return None, None, ""
|
||||||
|
|
||||||
def get_npm_mirror_download_url(major_version, full_version):
|
def get_npm_mirror_download_url(major_version, full_version):
|
||||||
"""根据 Chrome 版本从国内 npmmirror 获取 ChromeDriver 下载链接"""
|
"""根据 Chrome 版本从国内 npmmirror 获取 ChromeDriver 下载链接"""
|
||||||
|
archive_name, arch_dir, _ = _platform_archive_info(major_version)
|
||||||
|
if archive_name is None:
|
||||||
|
print(f"❌ 错误:npmmirror 未提供 Chrome {major_version} 在当前平台的包,跳过(交给 SeleniumBase 自带下载)。")
|
||||||
|
return None
|
||||||
|
|
||||||
# Chrome 115 及以上更改了分发路径
|
# Chrome 115 及以上更改了分发路径
|
||||||
if major_version >= 115:
|
if major_version >= 115:
|
||||||
base_url = "https://registry.npmmirror.com/-/binary/chrome-for-testing/"
|
base_url = "https://registry.npmmirror.com/-/binary/chrome-for-testing/"
|
||||||
is_64bit = platform.architecture()[0] == '64bit'
|
|
||||||
archive_name = "chromedriver-win64.zip" if is_64bit else "chromedriver-win32.zip"
|
|
||||||
arch_dir = "win64" if is_64bit else "win32"
|
|
||||||
else:
|
else:
|
||||||
# Chrome 114 及以下
|
|
||||||
base_url = "https://registry.npmmirror.com/-/binary/chromedriver/"
|
base_url = "https://registry.npmmirror.com/-/binary/chromedriver/"
|
||||||
archive_name = "chromedriver_win32.zip"
|
|
||||||
arch_dir = None
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
print(f"正在向镜像站请求可用版本列表: {base_url}")
|
print(f"正在向镜像站请求可用版本列表: {base_url}")
|
||||||
@@ -112,28 +148,37 @@ def install_uc_driver():
|
|||||||
# 获取当前 Python 环境中 seleniumbase 的 driver 存放目录
|
# 获取当前 Python 环境中 seleniumbase 的 driver 存放目录
|
||||||
sb_drivers_dir = os.path.join(seleniumbase.__path__[0], 'drivers')
|
sb_drivers_dir = os.path.join(seleniumbase.__path__[0], 'drivers')
|
||||||
os.makedirs(sb_drivers_dir, exist_ok=True)
|
os.makedirs(sb_drivers_dir, exist_ok=True)
|
||||||
# 对于 Windows,最终的文件名是 uc_driver.exe
|
# Windows 下 uc_driver 带后缀,Linux/Mac 无后缀
|
||||||
uc_driver_path = os.path.join(sb_drivers_dir, 'uc_driver.exe')
|
_, _, binary_suffix = _platform_archive_info(major)
|
||||||
|
uc_driver_name = 'uc_driver.exe' if binary_suffix == '.exe' else 'uc_driver'
|
||||||
|
uc_driver_path = os.path.join(sb_drivers_dir, uc_driver_name)
|
||||||
|
# 压缩包内目标二进制名(应对不同版本的目录嵌套)
|
||||||
|
target_binary = 'chromedriver.exe' if binary_suffix == '.exe' else 'chromedriver'
|
||||||
|
|
||||||
try:
|
try:
|
||||||
print("📦 正在解压并配置...")
|
print("📦 正在解压并配置...")
|
||||||
with zipfile.ZipFile(zip_path, 'r') as zip_ref:
|
with zipfile.ZipFile(zip_path, 'r') as zip_ref:
|
||||||
exe_name = None
|
exe_name = None
|
||||||
# 在压缩包中寻找 chromedriver.exe (应对不同版本的目录嵌套)
|
|
||||||
for name in zip_ref.namelist():
|
for name in zip_ref.namelist():
|
||||||
if name.lower().endswith('chromedriver.exe'):
|
# 取路径最后一段比较,兼容 chromedriver-linux64/chromedriver 这类嵌套
|
||||||
|
last_segment = name.rstrip('/').split('/')[-1].lower()
|
||||||
|
if last_segment == target_binary.lower():
|
||||||
exe_name = name
|
exe_name = name
|
||||||
break
|
break
|
||||||
|
|
||||||
if not exe_name:
|
if not exe_name:
|
||||||
print("❌ 错误:未在压缩包中找到 chromedriver.exe")
|
print(f"❌ 错误:未在压缩包中找到 {target_binary}")
|
||||||
return
|
return
|
||||||
|
|
||||||
# 直接提取内容并写入到期望的 uc_driver.exe 文件位置
|
# 直接提取内容并写入到期望的 uc_driver 文件位置
|
||||||
with zip_ref.open(exe_name) as source, open(uc_driver_path, "wb") as target:
|
with zip_ref.open(exe_name) as source, open(uc_driver_path, "wb") as target:
|
||||||
shutil.copyfileobj(source, target)
|
shutil.copyfileobj(source, target)
|
||||||
|
|
||||||
|
# Linux/Mac 下需要可执行权限
|
||||||
|
if binary_suffix != '.exe':
|
||||||
|
os.chmod(uc_driver_path, 0o755)
|
||||||
|
|
||||||
print(f"🎉 成功!uc_driver.exe 已自动安装至:")
|
print(f"🎉 成功!{uc_driver_name} 已自动安装至:")
|
||||||
print(f" -> {uc_driver_path}")
|
print(f" -> {uc_driver_path}")
|
||||||
print("现在程序可以正常运行了。")
|
print("现在程序可以正常运行了。")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
|
|||||||
214
main.py
214
main.py
@@ -7,11 +7,27 @@ import urllib.request
|
|||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
|
|
||||||
from batchmortal.api import build_paipu_urls, get_player_records, search_player, get_player_nickname_by_id
|
from batchmortal.api import build_paipu_urls, get_player_records, search_player, get_player_nickname_by_id
|
||||||
from batchmortal.browser import BrowserAutomator, ReviewSubmissionCoordinator, normalize_review_language
|
from batchmortal.browser import (
|
||||||
|
BrowserAutomator,
|
||||||
|
ReviewSubmissionCoordinator,
|
||||||
|
build_browser_kwargs,
|
||||||
|
normalize_review_language,
|
||||||
|
normalize_review_ui,
|
||||||
|
)
|
||||||
from batchmortal.results import ResultWriter, parse_metadata, get_processed_uuids, read_result_rows
|
from batchmortal.results import ResultWriter, parse_metadata, get_processed_uuids, read_result_rows
|
||||||
|
from batchmortal.tenhou import (
|
||||||
|
build_tenhou_paipu_urls,
|
||||||
|
fetch_tenhou_player_records,
|
||||||
|
normalize_tenhou_modes,
|
||||||
|
)
|
||||||
from batchmortal.visualize import plot_results
|
from batchmortal.visualize import plot_results
|
||||||
from seleniumbase import SB
|
from seleniumbase import SB
|
||||||
from batchmortal.config import load_config
|
from batchmortal.config import (
|
||||||
|
load_config,
|
||||||
|
normalize_source_mode,
|
||||||
|
resolve_mode_config,
|
||||||
|
source_for_mode,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def configure_logging():
|
def configure_logging():
|
||||||
@@ -34,12 +50,29 @@ def parse_review_language(value):
|
|||||||
raise argparse.ArgumentTypeError(str(exc)) from exc
|
raise argparse.ArgumentTypeError(str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
def parse_review_ui(value):
|
||||||
|
try:
|
||||||
|
return normalize_review_ui(value)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise argparse.ArgumentTypeError(str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
def parse_args():
|
def parse_args():
|
||||||
pre_parser = argparse.ArgumentParser(add_help=False)
|
pre_parser = argparse.ArgumentParser(add_help=False)
|
||||||
pre_parser.add_argument("--config", help="Path to config file (yaml or toml)")
|
pre_parser.add_argument("--config", help="Path to config file (yaml or toml)")
|
||||||
|
pre_parser.add_argument("--mode")
|
||||||
|
pre_parser.add_argument("--source", "--platform", dest="legacy_source")
|
||||||
pre_args, _ = pre_parser.parse_known_args()
|
pre_args, _ = pre_parser.parse_known_args()
|
||||||
|
|
||||||
config = load_config(pre_args.config)
|
raw_config = load_config(pre_args.config)
|
||||||
|
requested_mode = pre_args.mode or pre_args.legacy_source
|
||||||
|
try:
|
||||||
|
config_mode, _, config = resolve_mode_config(
|
||||||
|
raw_config,
|
||||||
|
requested_mode=requested_mode,
|
||||||
|
)
|
||||||
|
except ValueError as exc:
|
||||||
|
pre_parser.error(str(exc))
|
||||||
|
|
||||||
parser = argparse.ArgumentParser(
|
parser = argparse.ArgumentParser(
|
||||||
description="Batch Mortal Analysis Script (Python/SeleniumBase Edition)",
|
description="Batch Mortal Analysis Script (Python/SeleniumBase Edition)",
|
||||||
@@ -48,6 +81,20 @@ def parse_args():
|
|||||||
|
|
||||||
# -- General Options --
|
# -- General Options --
|
||||||
parser.add_argument("--config", help="Path to config file (yaml or toml)")
|
parser.add_argument("--config", help="Path to config file (yaml or toml)")
|
||||||
|
source_group = parser.add_mutually_exclusive_group()
|
||||||
|
source_group.add_argument(
|
||||||
|
"--mode",
|
||||||
|
choices=["mj", "th", "0", "1"],
|
||||||
|
default=config_mode,
|
||||||
|
help="Exclusive source mode: mj/0 for Mahjong Soul, th/1 for Tenhou",
|
||||||
|
)
|
||||||
|
source_group.add_argument(
|
||||||
|
"--source", "--platform",
|
||||||
|
choices=["majsoul", "tenhou"],
|
||||||
|
default=None,
|
||||||
|
help="Legacy source selector; use --mode for new configurations",
|
||||||
|
dest="legacy_source",
|
||||||
|
)
|
||||||
dry_run_default = config.get("dry_run", False)
|
dry_run_default = config.get("dry_run", False)
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--dry-run", "--dry_run",
|
"--dry-run", "--dry_run",
|
||||||
@@ -60,7 +107,7 @@ def parse_args():
|
|||||||
# -- Target Options --
|
# -- Target Options --
|
||||||
target_group = parser.add_argument_group("Target Options")
|
target_group = parser.add_argument_group("Target Options")
|
||||||
target_group.add_argument(
|
target_group.add_argument(
|
||||||
"-p", "-u", "--player", dest="player", default=config.get("player") or config.get("nickname"), help="Player nickname"
|
"-p", "-u", "--player", dest="player", default=config.get("player") or config.get("nickname"), help="Player nickname on the selected source"
|
||||||
)
|
)
|
||||||
target_group.add_argument(
|
target_group.add_argument(
|
||||||
"-a", "--account-id", "--account_id", dest="account_id", type=int, default=config.get("account_id"), help="Directly specify player account ID"
|
"-a", "--account-id", "--account_id", dest="account_id", type=int, default=config.get("account_id"), help="Directly specify player account ID"
|
||||||
@@ -72,7 +119,9 @@ def parse_args():
|
|||||||
"--limit", type=int, default=config.get("limit", 10), help="Max records per mode"
|
"--limit", type=int, default=config.get("limit", 10), help="Max records per mode"
|
||||||
)
|
)
|
||||||
analysis_group.add_argument(
|
analysis_group.add_argument(
|
||||||
"--modes", default=str(config.get("modes", "9")), help="Comma-separated mode IDs"
|
"--modes",
|
||||||
|
default=None,
|
||||||
|
help="Comma-separated modes: Mahjong Soul numeric IDs, or Tenhou all/4p/4p-east/4p-south/3p/3p-east/3p-south",
|
||||||
)
|
)
|
||||||
analysis_group.add_argument(
|
analysis_group.add_argument(
|
||||||
"--model-tag", "--model_tag", default=config.get("model_tag", "4.1b"), help="Mortal network version", dest="model_tag"
|
"--model-tag", "--model_tag", default=config.get("model_tag", "4.1b"), help="Mortal network version", dest="model_tag"
|
||||||
@@ -90,6 +139,19 @@ def parse_args():
|
|||||||
help="Review page language; writes the mjai.ekyu.moe form field select[name='lang']",
|
help="Review page language; writes the mjai.ekyu.moe form field select[name='lang']",
|
||||||
dest="review_language"
|
dest="review_language"
|
||||||
)
|
)
|
||||||
|
review_ui_config = config.get("review_ui", config.get("ui"))
|
||||||
|
try:
|
||||||
|
review_ui_default = normalize_review_ui(review_ui_config)
|
||||||
|
except ValueError as exc:
|
||||||
|
parser.error(str(exc))
|
||||||
|
analysis_group.add_argument(
|
||||||
|
"--review-ui", "--review_ui", "--ui",
|
||||||
|
default=review_ui_default,
|
||||||
|
type=parse_review_ui,
|
||||||
|
metavar="{classic,killerducky}",
|
||||||
|
help="Review result UI; KillerDucky metadata and bad-move data are parsed from report JSON",
|
||||||
|
dest="review_ui",
|
||||||
|
)
|
||||||
analysis_group.add_argument(
|
analysis_group.add_argument(
|
||||||
"--retry", type=int, default=config.get("retry", 3), help="Retry failed review items this many times"
|
"--retry", type=int, default=config.get("retry", 3), help="Retry failed review items this many times"
|
||||||
)
|
)
|
||||||
@@ -104,16 +166,18 @@ def parse_args():
|
|||||||
|
|
||||||
# -- Browser / Network Options --
|
# -- Browser / Network Options --
|
||||||
browser_group = parser.add_argument_group("Browser & Network Options")
|
browser_group = parser.add_argument_group("Browser & Network Options")
|
||||||
headless_default = config.get("headless", False)
|
headless_default = config.get("headless", True)
|
||||||
browser_group.add_argument(
|
browser_group.add_argument(
|
||||||
"--headless",
|
"--headless",
|
||||||
action="store_true" if not headless_default else "store_false",
|
action=argparse.BooleanOptionalAction,
|
||||||
default=headless_default,
|
default=headless_default,
|
||||||
help="Run browser headlessly",
|
help="Run browser headlessly (default: on)",
|
||||||
dest="headless"
|
dest="headless"
|
||||||
)
|
)
|
||||||
browser_group.add_argument(
|
browser_group.add_argument(
|
||||||
"--proxy", default=config.get("proxy"), help="Proxy URL (e.g. http://127.0.0.1:7890)"
|
"--proxy",
|
||||||
|
default=config.get("proxy"),
|
||||||
|
help='Proxy URL (e.g. http://127.0.0.1:7890), or "direct" to bypass all proxies including the system proxy (needed for mjai.ekyu.moe Turnstile)',
|
||||||
)
|
)
|
||||||
|
|
||||||
# -- Output Options --
|
# -- Output Options --
|
||||||
@@ -184,33 +248,65 @@ def parse_args():
|
|||||||
)
|
)
|
||||||
|
|
||||||
args = parser.parse_args()
|
args = parser.parse_args()
|
||||||
|
|
||||||
if not args.player and not args.account_id:
|
if args.legacy_source:
|
||||||
parser.error("-p/--player or -a/--account-id is required either via command line arguments or config file")
|
args.mode = normalize_source_mode(args.legacy_source)
|
||||||
|
else:
|
||||||
|
args.mode = normalize_source_mode(args.mode)
|
||||||
|
args.source = source_for_mode(args.mode)
|
||||||
|
|
||||||
|
if args.modes is None:
|
||||||
|
fallback = "all" if args.source == "tenhou" else "9"
|
||||||
|
args.modes = str(config.get("modes", fallback))
|
||||||
|
|
||||||
|
if args.source == "tenhou":
|
||||||
|
if not args.player:
|
||||||
|
parser.error("Tenhou source requires -p/--player (a Tenhou player name)")
|
||||||
|
if args.account_id:
|
||||||
|
parser.error("--account-id is only supported by the majsoul source")
|
||||||
|
elif not args.player and not args.account_id:
|
||||||
|
parser.error("Mahjong Soul source requires -p/--player or -a/--account-id")
|
||||||
|
|
||||||
args.target_name = args.player if args.player else str(args.account_id)
|
args.target_name = args.player if args.player else str(args.account_id)
|
||||||
|
|
||||||
return args
|
return args
|
||||||
|
|
||||||
|
|
||||||
def build_output_path(nickname: str, output_format: str) -> tuple[str, str]:
|
def build_output_path(nickname: str, output_format: str, source: str = "majsoul") -> tuple[str, str]:
|
||||||
safe_nick = "".join(
|
safe_nick = "".join(
|
||||||
c if c.isalnum() or c in ("_", "-", "\u4e00", "\u9fa5") else "_"
|
c if c.isalnum() or c in ("_", "-", "\u4e00", "\u9fa5") else "_"
|
||||||
for c in nickname
|
for c in nickname
|
||||||
)
|
)
|
||||||
output_root = os.path.join(os.path.dirname(os.path.abspath(__file__)), "results", safe_nick)
|
results_root = os.path.join(
|
||||||
|
os.path.dirname(os.path.abspath(__file__)),
|
||||||
|
"results",
|
||||||
|
source,
|
||||||
|
)
|
||||||
|
output_root = os.path.join(results_root, safe_nick)
|
||||||
out_path = os.path.join(output_root, f"results.{output_format}")
|
out_path = os.path.join(output_root, f"results.{output_format}")
|
||||||
return output_root, out_path
|
return output_root, out_path
|
||||||
|
|
||||||
|
|
||||||
def detect_proxy(explicit_proxy: str | None) -> str | None:
|
def detect_proxy(explicit_proxy: str | None) -> str | None:
|
||||||
if explicit_proxy:
|
if explicit_proxy:
|
||||||
|
if explicit_proxy.strip().lower() == "direct":
|
||||||
|
return "direct"
|
||||||
return explicit_proxy
|
return explicit_proxy
|
||||||
sys_proxies = urllib.request.getproxies()
|
sys_proxies = urllib.request.getproxies()
|
||||||
return sys_proxies.get("https") or sys_proxies.get("http")
|
return sys_proxies.get("https") or sys_proxies.get("http")
|
||||||
|
|
||||||
|
|
||||||
def collect_tasks(account_id: int, modes: list[int], limit: int, output_root: str, processed_uuids: set) -> list[dict]:
|
def finalize_tasks(tasks: list[dict]) -> list[dict]:
|
||||||
|
total_tasks = len(tasks)
|
||||||
|
for index, task in enumerate(tasks, start=1):
|
||||||
|
task["idx"] = index
|
||||||
|
task["total"] = total_tasks
|
||||||
|
short_url = task["uuid"].split("-")[-1]
|
||||||
|
task["log_prefix"] = f"[{index}/{total_tasks}][{short_url}]"
|
||||||
|
return tasks
|
||||||
|
|
||||||
|
|
||||||
|
def collect_majsoul_tasks(account_id: int, modes: list[int], limit: int, output_root: str, processed_uuids: set) -> list[dict]:
|
||||||
tasks = []
|
tasks = []
|
||||||
for mode in modes:
|
for mode in modes:
|
||||||
log_line(f"[Mode {mode}] Fetching records...")
|
log_line(f"[Mode {mode}] Fetching records...")
|
||||||
@@ -232,6 +328,7 @@ def collect_tasks(account_id: int, modes: list[int], limit: int, output_root: st
|
|||||||
continue
|
continue
|
||||||
tasks.append(
|
tasks.append(
|
||||||
{
|
{
|
||||||
|
"source": "majsoul",
|
||||||
"mode": mode,
|
"mode": mode,
|
||||||
"uuid": item["uuid"],
|
"uuid": item["uuid"],
|
||||||
"paipu_url": item["paipuUrl"],
|
"paipu_url": item["paipuUrl"],
|
||||||
@@ -241,24 +338,48 @@ def collect_tasks(account_id: int, modes: list[int], limit: int, output_root: st
|
|||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
|
||||||
total_tasks = len(tasks)
|
return finalize_tasks(tasks)
|
||||||
for index, task in enumerate(tasks, start=1):
|
|
||||||
task["idx"] = index
|
|
||||||
task["total"] = total_tasks
|
|
||||||
short_url = task["uuid"].split("-")[-1]
|
|
||||||
task["log_prefix"] = f"[{index}/{total_tasks}][{short_url}]"
|
|
||||||
|
|
||||||
return tasks
|
|
||||||
|
def collect_tenhou_tasks(
|
||||||
|
records: list[dict],
|
||||||
|
player_name: str,
|
||||||
|
modes: tuple[str, ...],
|
||||||
|
limit: int,
|
||||||
|
output_root: str,
|
||||||
|
processed_uuids: set,
|
||||||
|
) -> list[dict]:
|
||||||
|
items = build_tenhou_paipu_urls(records, player_name, modes=modes, limit=limit)
|
||||||
|
tasks = []
|
||||||
|
for item in items:
|
||||||
|
if item["uuid"] in processed_uuids:
|
||||||
|
log_line(f"[Skip] uuid={item['uuid']} already processed.")
|
||||||
|
continue
|
||||||
|
mode = item["mode"]
|
||||||
|
tasks.append(
|
||||||
|
{
|
||||||
|
"source": "tenhou",
|
||||||
|
"mode": mode,
|
||||||
|
"uuid": item["uuid"],
|
||||||
|
"paipu_url": item["paipuUrl"],
|
||||||
|
"start_time": item.get("startTime", ""),
|
||||||
|
"end_time": item.get("endTime", ""),
|
||||||
|
"mode_dir": os.path.join(output_root, f"mode_{mode}"),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return finalize_tasks(tasks)
|
||||||
|
|
||||||
|
|
||||||
def print_summary(args, modes):
|
def print_summary(args, modes):
|
||||||
log_line("=== Batch Mortal Analysis ===")
|
log_line("=== Batch Mortal Analysis ===")
|
||||||
target_display = args.target_name + (f" (ID: {args.account_id})" if args.account_id and args.target_name != str(args.account_id) else "")
|
target_display = args.target_name + (f" (ID: {args.account_id})" if args.account_id and args.target_name != str(args.account_id) else "")
|
||||||
log_line(f" Target: {target_display}")
|
log_line(f" Target: {target_display}")
|
||||||
|
log_line(f" Mode: {args.mode} ({args.source})")
|
||||||
log_line(f" Modes: {modes}")
|
log_line(f" Modes: {modes}")
|
||||||
log_line(f" Limit: {args.limit} per mode")
|
log_line(f" Limit: {args.limit} per mode")
|
||||||
log_line(f" ModelTag: {args.model_tag}")
|
log_line(f" ModelTag: {args.model_tag}")
|
||||||
log_line(f" Language: {args.review_language}")
|
log_line(f" Language: {args.review_language}")
|
||||||
|
log_line(f" ReviewUI: {args.review_ui}")
|
||||||
log_line(f" Headless: {args.headless}")
|
log_line(f" Headless: {args.headless}")
|
||||||
log_line(f" DryRun: {args.dry_run}")
|
log_line(f" DryRun: {args.dry_run}")
|
||||||
log_line(f" Retry: {args.retry}")
|
log_line(f" Retry: {args.retry}")
|
||||||
@@ -350,6 +471,7 @@ def consume_result_event(args, writer: ResultWriter, result_event: dict, stats:
|
|||||||
timestamp = datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
timestamp = datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||||
base_row = {
|
base_row = {
|
||||||
"nickname": args.target_name,
|
"nickname": args.target_name,
|
||||||
|
"source": task.get("source", args.source),
|
||||||
"mode": task["mode"],
|
"mode": task["mode"],
|
||||||
"uuid": task["uuid"],
|
"uuid": task["uuid"],
|
||||||
"paipuUrl": task["paipu_url"],
|
"paipuUrl": task["paipu_url"],
|
||||||
@@ -444,7 +566,7 @@ def run_parallel_analysis(
|
|||||||
log_line("[Serial] Starting analysis with 1 persistent browser")
|
log_line("[Serial] Starting analysis with 1 persistent browser")
|
||||||
|
|
||||||
try:
|
try:
|
||||||
with SB(uc=True, headless=automator.headless, proxy=automator.proxy) as sb:
|
with SB(**build_browser_kwargs(automator.headless, automator.proxy)) as sb:
|
||||||
for task in tasks:
|
for task in tasks:
|
||||||
result_event = None
|
result_event = None
|
||||||
for attempt in range(args.retry + 1):
|
for attempt in range(args.retry + 1):
|
||||||
@@ -527,13 +649,26 @@ def main():
|
|||||||
start_time = time.time()
|
start_time = time.time()
|
||||||
args = parse_args()
|
args = parse_args()
|
||||||
args.retry = max(0, args.retry)
|
args.retry = max(0, args.retry)
|
||||||
modes = [int(mode.strip()) for mode in args.modes.split(",")]
|
try:
|
||||||
|
if args.source == "tenhou":
|
||||||
|
modes = normalize_tenhou_modes(args.modes)
|
||||||
|
else:
|
||||||
|
modes = [int(mode.strip()) for mode in args.modes.split(",") if mode.strip()]
|
||||||
|
if not modes:
|
||||||
|
raise ValueError("At least one Mahjong Soul mode is required.")
|
||||||
|
except ValueError as exc:
|
||||||
|
logging.error(f"[FATAL] {exc}")
|
||||||
|
sys.exit(2)
|
||||||
|
|
||||||
if not args.dry_run:
|
if not args.dry_run:
|
||||||
ensure_uc_driver()
|
ensure_uc_driver()
|
||||||
|
|
||||||
|
account_id = None
|
||||||
|
tenhou_records = None
|
||||||
try:
|
try:
|
||||||
if args.account_id:
|
if args.source == "tenhou":
|
||||||
|
args.target_name, tenhou_records = fetch_tenhou_player_records(args.player)
|
||||||
|
elif args.account_id:
|
||||||
account_id = args.account_id
|
account_id = args.account_id
|
||||||
if not args.player:
|
if not args.player:
|
||||||
# Attempt to fetch nickname to use as the target
|
# Attempt to fetch nickname to use as the target
|
||||||
@@ -549,16 +684,28 @@ def main():
|
|||||||
|
|
||||||
print_summary(args, modes)
|
print_summary(args, modes)
|
||||||
|
|
||||||
output_root, out_path = build_output_path(args.target_name, args.output)
|
output_root, out_path = build_output_path(args.target_name, args.output, args.source)
|
||||||
processed_uuids = get_processed_uuids(out_path, args.output)
|
processed_uuids = get_processed_uuids(out_path, args.output)
|
||||||
proxy = detect_proxy(args.proxy)
|
proxy = detect_proxy(args.proxy)
|
||||||
|
|
||||||
if proxy:
|
if proxy == "direct":
|
||||||
|
logging.info("[Proxy] Forcing direct connection (bypassing system proxy) for browser.")
|
||||||
|
elif proxy:
|
||||||
logging.info(f"[Proxy] Using proxy for browser: {proxy}")
|
logging.info(f"[Proxy] Using proxy for browser: {proxy}")
|
||||||
else:
|
else:
|
||||||
logging.info("[Proxy] No system proxy detected, running directly.")
|
logging.info("[Proxy] No system proxy detected, running directly.")
|
||||||
|
|
||||||
tasks = collect_tasks(account_id, modes, args.limit, output_root, processed_uuids)
|
if args.source == "tenhou":
|
||||||
|
tasks = collect_tenhou_tasks(
|
||||||
|
tenhou_records,
|
||||||
|
args.target_name,
|
||||||
|
modes,
|
||||||
|
args.limit,
|
||||||
|
output_root,
|
||||||
|
processed_uuids,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
tasks = collect_majsoul_tasks(account_id, modes, args.limit, output_root, processed_uuids)
|
||||||
total_processed = 0
|
total_processed = 0
|
||||||
total_failed = 0
|
total_failed = 0
|
||||||
analysis_stats = create_analysis_stats()
|
analysis_stats = create_analysis_stats()
|
||||||
@@ -573,6 +720,7 @@ def main():
|
|||||||
headless=args.headless,
|
headless=args.headless,
|
||||||
proxy=proxy,
|
proxy=proxy,
|
||||||
review_language=args.review_language,
|
review_language=args.review_language,
|
||||||
|
review_ui=args.review_ui,
|
||||||
submission_coordinator=None,
|
submission_coordinator=None,
|
||||||
controlled_submission=False,
|
controlled_submission=False,
|
||||||
)
|
)
|
||||||
@@ -586,6 +734,7 @@ def main():
|
|||||||
headless=args.headless,
|
headless=args.headless,
|
||||||
proxy=proxy,
|
proxy=proxy,
|
||||||
review_language=args.review_language,
|
review_language=args.review_language,
|
||||||
|
review_ui=args.review_ui,
|
||||||
submission_coordinator=submission_coordinator,
|
submission_coordinator=submission_coordinator,
|
||||||
controlled_submission=True,
|
controlled_submission=True,
|
||||||
)
|
)
|
||||||
@@ -599,6 +748,7 @@ def main():
|
|||||||
headless=args.headless,
|
headless=args.headless,
|
||||||
proxy=proxy,
|
proxy=proxy,
|
||||||
review_language=args.review_language,
|
review_language=args.review_language,
|
||||||
|
review_ui=args.review_ui,
|
||||||
submission_coordinator=submission_coordinator,
|
submission_coordinator=submission_coordinator,
|
||||||
controlled_submission=True,
|
controlled_submission=True,
|
||||||
)
|
)
|
||||||
@@ -616,7 +766,13 @@ def main():
|
|||||||
log_line(f" Time: {elapsed:.2f}s")
|
log_line(f" Time: {elapsed:.2f}s")
|
||||||
if not args.dry_run:
|
if not args.dry_run:
|
||||||
log_line(f" Output: {out_path}")
|
log_line(f" Output: {out_path}")
|
||||||
plot_results(args.target_name, args.plot, args.output, plot_limit=args.plot_limit)
|
plot_results(
|
||||||
|
args.target_name,
|
||||||
|
args.plot,
|
||||||
|
args.output,
|
||||||
|
plot_limit=args.plot_limit,
|
||||||
|
output_root=output_root,
|
||||||
|
)
|
||||||
log_line("============")
|
log_line("============")
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,4 +1,10 @@
|
|||||||
from batchmortal.browser import build_review_url, normalize_review_language
|
import pytest
|
||||||
|
|
||||||
|
from batchmortal.browser import (
|
||||||
|
build_review_url,
|
||||||
|
normalize_review_language,
|
||||||
|
normalize_review_ui,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_normalize_review_language():
|
def test_normalize_review_language():
|
||||||
@@ -17,6 +23,19 @@ def test_build_review_url():
|
|||||||
assert build_review_url("ko") == "https://mjai.ekyu.moe/ko.html"
|
assert build_review_url("ko") == "https://mjai.ekyu.moe/ko.html"
|
||||||
|
|
||||||
|
|
||||||
|
def test_normalize_review_ui():
|
||||||
|
assert normalize_review_ui(None) == "classic"
|
||||||
|
assert normalize_review_ui("classic") == "classic"
|
||||||
|
assert normalize_review_ui("killerducky") == "killerducky"
|
||||||
|
assert normalize_review_ui("killer-ducky") == "killerducky"
|
||||||
|
assert normalize_review_ui("kd") == "killerducky"
|
||||||
|
|
||||||
|
|
||||||
|
def test_normalize_review_ui_rejects_unknown_values():
|
||||||
|
with pytest.raises(ValueError, match="Unsupported review UI"):
|
||||||
|
normalize_review_ui("foo")
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
test_normalize_review_language()
|
test_normalize_review_language()
|
||||||
test_build_review_url()
|
test_build_review_url()
|
||||||
|
|||||||
79
tests/test_config_modes.py
Normal file
79
tests/test_config_modes.py
Normal file
@@ -0,0 +1,79 @@
|
|||||||
|
import pytest
|
||||||
|
|
||||||
|
from batchmortal.config import (
|
||||||
|
normalize_source_mode,
|
||||||
|
resolve_mode_config,
|
||||||
|
source_for_mode,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
CONFIG = {
|
||||||
|
"mode": "mj",
|
||||||
|
"review_language": "zh-CN",
|
||||||
|
"mj": {
|
||||||
|
"nickname": "雀魂玩家",
|
||||||
|
"limit": 70,
|
||||||
|
"modes": "12",
|
||||||
|
},
|
||||||
|
"th": {
|
||||||
|
"nickname": "天凤玩家",
|
||||||
|
"limit": 10,
|
||||||
|
"modes": "4p-south",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_mj_and_th_sections_are_selected_exclusively():
|
||||||
|
mode, source, selected = resolve_mode_config(CONFIG)
|
||||||
|
assert (mode, source) == ("mj", "majsoul")
|
||||||
|
assert selected["nickname"] == "雀魂玩家"
|
||||||
|
assert selected["modes"] == "12"
|
||||||
|
|
||||||
|
mode, source, selected = resolve_mode_config(CONFIG, requested_mode="th")
|
||||||
|
assert (mode, source) == ("th", "tenhou")
|
||||||
|
assert selected["nickname"] == "天凤玩家"
|
||||||
|
assert selected["modes"] == "4p-south"
|
||||||
|
|
||||||
|
|
||||||
|
def test_numeric_and_long_source_aliases_remain_supported():
|
||||||
|
assert normalize_source_mode(0) == "mj"
|
||||||
|
assert normalize_source_mode(1) == "th"
|
||||||
|
assert normalize_source_mode("majsoul") == "mj"
|
||||||
|
assert normalize_source_mode("tenhou") == "th"
|
||||||
|
assert source_for_mode("th") == "tenhou"
|
||||||
|
|
||||||
|
|
||||||
|
def test_normalize_source_mode_defaults_to_mj_for_none():
|
||||||
|
assert normalize_source_mode(None) == "mj"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("value", ["foo", 2])
|
||||||
|
def test_normalize_source_mode_rejects_unsupported_values(value):
|
||||||
|
with pytest.raises(ValueError, match="Unsupported source mode"):
|
||||||
|
normalize_source_mode(value)
|
||||||
|
|
||||||
|
|
||||||
|
def test_legacy_source_config_is_still_readable():
|
||||||
|
mode, source, selected = resolve_mode_config(
|
||||||
|
{
|
||||||
|
"source": "tenhou",
|
||||||
|
"nickname": "旧配置玩家",
|
||||||
|
"tenhou_modes": "4p",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
assert (mode, source) == ("th", "tenhou")
|
||||||
|
assert selected["nickname"] == "旧配置玩家"
|
||||||
|
assert selected["modes"] == "4p"
|
||||||
|
|
||||||
|
|
||||||
|
def test_conflicting_config_selectors_are_rejected():
|
||||||
|
with pytest.raises(ValueError, match="both 'mode' and legacy 'source'"):
|
||||||
|
resolve_mode_config({"mode": "mj", "source": "tenhou"})
|
||||||
|
|
||||||
|
with pytest.raises(ValueError, match="both 'mode' and legacy 'source'"):
|
||||||
|
resolve_mode_config({"mode": "mj", "source": "majsoul"})
|
||||||
|
|
||||||
|
|
||||||
|
def test_source_section_must_be_a_mapping():
|
||||||
|
with pytest.raises(ValueError, match="section 'th'"):
|
||||||
|
resolve_mode_config({"mode": "th", "th": "not-a-mapping"})
|
||||||
102
tests/test_killerducky.py
Normal file
102
tests/test_killerducky.py
Normal file
@@ -0,0 +1,102 @@
|
|||||||
|
from batchmortal.browser import (
|
||||||
|
parse_killerducky_bad_move_stats,
|
||||||
|
parse_killerducky_metadata,
|
||||||
|
)
|
||||||
|
from batchmortal.results import parse_metadata
|
||||||
|
|
||||||
|
|
||||||
|
def make_entry(actual_probability, *, is_equal=False, actual_index=1):
|
||||||
|
details = [
|
||||||
|
{"prob": 0.8},
|
||||||
|
{"prob": actual_probability},
|
||||||
|
]
|
||||||
|
return {
|
||||||
|
"is_equal": is_equal,
|
||||||
|
"actual_index": actual_index,
|
||||||
|
"details": details,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_killerducky_about_metadata():
|
||||||
|
data = {
|
||||||
|
"engine": "Mortal",
|
||||||
|
"game_length": "Hanchan",
|
||||||
|
"review_time": "2s 709ms",
|
||||||
|
"player_id": 2,
|
||||||
|
"review": {
|
||||||
|
"model_tag": "4.1b",
|
||||||
|
"rating": 0.8422832528256039,
|
||||||
|
"temperature": 0.1,
|
||||||
|
"total_matches": 131,
|
||||||
|
"total_reviewed": 173,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
metadata = parse_killerducky_metadata(data)
|
||||||
|
|
||||||
|
assert metadata == {
|
||||||
|
"engine": "Mortal",
|
||||||
|
"model tag": "4.1b",
|
||||||
|
"rating": "84.228",
|
||||||
|
"matches/total": "131/173 = 75.723%",
|
||||||
|
"temperature": "0.1",
|
||||||
|
"game length": "Hanchan",
|
||||||
|
"player id": "2",
|
||||||
|
"review duration": "2s 709ms",
|
||||||
|
}
|
||||||
|
assert parse_metadata(metadata) == {
|
||||||
|
"modelTag": "4.1b",
|
||||||
|
"rating": "84.228",
|
||||||
|
"aiConsistencyRate": "75.723%",
|
||||||
|
"aiConsistencyNumerator": "131",
|
||||||
|
"aiConsistencyDenominator": "173",
|
||||||
|
"temperature": "0.1",
|
||||||
|
"gameLength": "Hanchan",
|
||||||
|
"playerId": "2",
|
||||||
|
"reviewDuration": "2s 709ms",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_killerducky_bad_moves_uses_actual_choice_probability():
|
||||||
|
invalid_entry = {
|
||||||
|
"is_equal": False,
|
||||||
|
"actual_index": 4,
|
||||||
|
"details": [{"prob": 0.9}],
|
||||||
|
}
|
||||||
|
data = {
|
||||||
|
"review": {
|
||||||
|
"total_reviewed": 4,
|
||||||
|
"kyokus": [
|
||||||
|
{
|
||||||
|
"entries": [
|
||||||
|
make_entry(0.01, is_equal=True, actual_index=0),
|
||||||
|
make_entry(0.04),
|
||||||
|
make_entry(0.08),
|
||||||
|
invalid_entry,
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
assert parse_killerducky_bad_move_stats(data) == {
|
||||||
|
"badMoveRate5": "25.000%",
|
||||||
|
"badMoveCount5": "1",
|
||||||
|
"badMoveRate10": "50.000%",
|
||||||
|
"badMoveCount10": "2",
|
||||||
|
"badMoveDenominator": "4",
|
||||||
|
"badMoveOrderLossCount": "3",
|
||||||
|
"badMoveUnparsedCount": "1",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_killerducky_bad_moves_handles_missing_review_data():
|
||||||
|
assert parse_killerducky_bad_move_stats({}) == {
|
||||||
|
"badMoveRate5": "",
|
||||||
|
"badMoveCount5": "0",
|
||||||
|
"badMoveRate10": "",
|
||||||
|
"badMoveCount10": "0",
|
||||||
|
"badMoveDenominator": "",
|
||||||
|
"badMoveOrderLossCount": "0",
|
||||||
|
"badMoveUnparsedCount": "0",
|
||||||
|
}
|
||||||
14
tests/test_output_paths.py
Normal file
14
tests/test_output_paths.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from main import build_output_path
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("source", ["majsoul", "tenhou"])
|
||||||
|
def test_results_are_grouped_by_source(source):
|
||||||
|
output_root, output_file = build_output_path("测试 玩家", "xlsx", source)
|
||||||
|
|
||||||
|
root_path = Path(output_root)
|
||||||
|
assert root_path.parts[-3:] == ("results", source, "测试_玩家")
|
||||||
|
assert Path(output_file) == root_path / "results.xlsx"
|
||||||
179
tests/test_tenhou.py
Normal file
179
tests/test_tenhou.py
Normal file
@@ -0,0 +1,179 @@
|
|||||||
|
import pytest
|
||||||
|
|
||||||
|
from batchmortal.tenhou import (
|
||||||
|
build_tenhou_paipu_urls,
|
||||||
|
decode_tenhou_viewpoint,
|
||||||
|
format_tenhou_timestamp,
|
||||||
|
normalize_tenhou_modes,
|
||||||
|
parse_tenhou_log_url,
|
||||||
|
tenhou_mode_matches,
|
||||||
|
tenhou_record_mode,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
PLAYER = "ププリン"
|
||||||
|
|
||||||
|
|
||||||
|
def make_record(
|
||||||
|
log_id: str,
|
||||||
|
*,
|
||||||
|
starttime: int,
|
||||||
|
playernum: int = 4,
|
||||||
|
playlength: int = 2,
|
||||||
|
url_host: str = "tenhou.net",
|
||||||
|
) -> dict:
|
||||||
|
record = {
|
||||||
|
"starttime": starttime,
|
||||||
|
"during": 32,
|
||||||
|
"playernum": playernum,
|
||||||
|
"playlength": playlength,
|
||||||
|
"player1": PLAYER,
|
||||||
|
"player2": "Tatsuno7",
|
||||||
|
"player3": "Navitas",
|
||||||
|
"tw": "198",
|
||||||
|
"url": f"http://{url_host}/0/?log={log_id}",
|
||||||
|
}
|
||||||
|
if playernum == 4:
|
||||||
|
record["player4"] = "tca00"
|
||||||
|
return record
|
||||||
|
|
||||||
|
|
||||||
|
def test_decode_nodocchi_viewpoint_from_real_example():
|
||||||
|
# Nodocchi returns 198 for this game. ププリン is player1 (first place),
|
||||||
|
# and the site's two-bit permutation decoding resolves that to Tenhou seat 2.
|
||||||
|
assert decode_tenhou_viewpoint("198", 1) == 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_decode_tenhou_viewpoint_rejects_non_numeric_tw():
|
||||||
|
assert decode_tenhou_viewpoint("abc", 1) is None
|
||||||
|
assert decode_tenhou_viewpoint(None, 1) is None
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("player_order", [0, 5, None])
|
||||||
|
def test_decode_tenhou_viewpoint_rejects_invalid_player_order(player_order):
|
||||||
|
assert decode_tenhou_viewpoint("198", player_order) is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_decode_tenhou_viewpoint_returns_none_for_unmapped_packed_value():
|
||||||
|
assert decode_tenhou_viewpoint("999", 1) is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_real_example_paipu_url():
|
||||||
|
record = make_record(
|
||||||
|
"2026071212gm-00a9-0000-2dab9d24",
|
||||||
|
starttime=1783828560,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert build_tenhou_paipu_urls([record], PLAYER) == [
|
||||||
|
{
|
||||||
|
"source": "tenhou",
|
||||||
|
"mode": "4p-south",
|
||||||
|
"uuid": "2026071212gm-00a9-0000-2dab9d24",
|
||||||
|
"paipuUrl": "http://tenhou.net/0/?log=2026071212gm-00a9-0000-2dab9d24&tw=2",
|
||||||
|
"startTime": format_tenhou_timestamp(1783828560),
|
||||||
|
"endTime": format_tenhou_timestamp(1783828560 + 32 * 60),
|
||||||
|
}
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_tenhou_paipu_urls_deduplicates_log_ids():
|
||||||
|
log_id = "2026071212gm-00a9-0000-11111111"
|
||||||
|
records = [
|
||||||
|
make_record(log_id, starttime=200),
|
||||||
|
make_record(log_id, starttime=100),
|
||||||
|
]
|
||||||
|
|
||||||
|
items = build_tenhou_paipu_urls(records, PLAYER)
|
||||||
|
|
||||||
|
assert len(items) == 1
|
||||||
|
assert items[0]["uuid"] == log_id
|
||||||
|
assert items[0]["startTime"] == format_tenhou_timestamp(200)
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_tenhou_paipu_urls_skips_invalid_viewpoints():
|
||||||
|
valid_log_id = "2026071212gm-00a9-0000-11111111"
|
||||||
|
invalid_tw = make_record(
|
||||||
|
"2026071212gm-00a9-0000-22222222",
|
||||||
|
starttime=200,
|
||||||
|
)
|
||||||
|
invalid_tw["tw"] = "999"
|
||||||
|
missing_player = make_record(
|
||||||
|
"2026071212gm-00a9-0000-33333333",
|
||||||
|
starttime=100,
|
||||||
|
)
|
||||||
|
missing_player["player1"] = "another-player"
|
||||||
|
|
||||||
|
items = build_tenhou_paipu_urls(
|
||||||
|
[make_record(valid_log_id, starttime=300), invalid_tw, missing_player],
|
||||||
|
PLAYER,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert [item["uuid"] for item in items] == [valid_log_id]
|
||||||
|
|
||||||
|
|
||||||
|
def test_mode_filter_and_limit_are_applied_per_actual_mode():
|
||||||
|
records = [
|
||||||
|
make_record("2026071213gm-00a9-0000-aaaaaaaa", starttime=300, playlength=2),
|
||||||
|
make_record("2026071212gm-00a9-0000-bbbbbbbb", starttime=200, playlength=2),
|
||||||
|
make_record("2026071211gm-00a9-0000-cccccccc", starttime=100, playlength=1),
|
||||||
|
]
|
||||||
|
|
||||||
|
items = build_tenhou_paipu_urls(records, PLAYER, modes="4p", limit=1)
|
||||||
|
|
||||||
|
assert [item["uuid"] for item in items] == [
|
||||||
|
"2026071213gm-00a9-0000-aaaaaaaa",
|
||||||
|
"2026071211gm-00a9-0000-cccccccc",
|
||||||
|
]
|
||||||
|
assert [item["mode"] for item in items] == ["4p-south", "4p-east"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_non_tenhou_urls_are_not_mistaken_for_tenhou_records():
|
||||||
|
record = make_record(
|
||||||
|
"2026071212gm-00a9-0000-2dab9d24",
|
||||||
|
starttime=1783828560,
|
||||||
|
url_host="game.maj-soul.com",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert build_tenhou_paipu_urls([record], PLAYER) == []
|
||||||
|
assert parse_tenhou_log_url(record["url"]) is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_records_without_url_or_viewpoint_are_skipped():
|
||||||
|
no_url = make_record("2026071212gm-00a9-0000-2dab9d24", starttime=200)
|
||||||
|
no_url.pop("url")
|
||||||
|
no_viewpoint = make_record("2026071213gm-00a9-0000-aaaaaaaa", starttime=300)
|
||||||
|
no_viewpoint.pop("tw")
|
||||||
|
|
||||||
|
assert build_tenhou_paipu_urls([no_url, no_viewpoint], PLAYER) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_tenhou_mode_aliases_and_labels():
|
||||||
|
assert normalize_tenhou_modes("四南,3p-east") == ("4p-south", "3p-east")
|
||||||
|
assert normalize_tenhou_modes("*") == ("all",)
|
||||||
|
assert tenhou_record_mode({"playernum": 3, "playlength": 1}) == "3p-east"
|
||||||
|
|
||||||
|
|
||||||
|
def test_normalize_tenhou_modes_rejects_unsupported_label():
|
||||||
|
with pytest.raises(ValueError, match="Unsupported Tenhou mode 'foo'"):
|
||||||
|
normalize_tenhou_modes(("foo",))
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("modes", ["", (), ("",), (" ",), (" ", "", " ")])
|
||||||
|
def test_normalize_tenhou_modes_empty_values_fall_back_to_all(modes):
|
||||||
|
assert normalize_tenhou_modes(modes) == ("all",)
|
||||||
|
|
||||||
|
|
||||||
|
def test_tenhou_mode_matches_normalized_modes():
|
||||||
|
four_player = normalize_tenhou_modes(("4p",))
|
||||||
|
assert tenhou_mode_matches("4p-east", four_player)
|
||||||
|
assert tenhou_mode_matches("4p-south", four_player)
|
||||||
|
assert not tenhou_mode_matches("3p-east", four_player)
|
||||||
|
assert not tenhou_mode_matches("3p-south", four_player)
|
||||||
|
|
||||||
|
east_only = normalize_tenhou_modes(("4p-east",))
|
||||||
|
assert tenhou_mode_matches("4p-east", east_only)
|
||||||
|
assert not tenhou_mode_matches("4p-south", east_only)
|
||||||
|
|
||||||
|
south_only = normalize_tenhou_modes(("4p-south",))
|
||||||
|
assert tenhou_mode_matches("4p-south", south_only)
|
||||||
|
assert not tenhou_mode_matches("4p-east", south_only)
|
||||||
223
tests/test_visualize.py
Normal file
223
tests/test_visualize.py
Normal file
@@ -0,0 +1,223 @@
|
|||||||
|
import pytest
|
||||||
|
|
||||||
|
from batchmortal import visualize
|
||||||
|
from batchmortal.visualize import (
|
||||||
|
prepare_dashboard_data,
|
||||||
|
rolling_average,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def make_result(
|
||||||
|
index: int,
|
||||||
|
*,
|
||||||
|
rating: float,
|
||||||
|
ai_rate: str = "",
|
||||||
|
ai_numerator: str = "",
|
||||||
|
ai_denominator: str = "",
|
||||||
|
) -> dict:
|
||||||
|
return {
|
||||||
|
"nickname": "测试玩家",
|
||||||
|
"source": "majsoul",
|
||||||
|
"mode": "16",
|
||||||
|
"uuid": f"game-{index}",
|
||||||
|
"startTime": f"2026-07-{index:02d} 12:00:00",
|
||||||
|
"resultUrl": f"https://example.com/report/{index}",
|
||||||
|
"modelTag": "4.1b",
|
||||||
|
"rating": str(rating),
|
||||||
|
"aiConsistencyRate": ai_rate,
|
||||||
|
"aiConsistencyNumerator": ai_numerator,
|
||||||
|
"aiConsistencyDenominator": ai_denominator,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_rolling_average_requires_a_full_window():
|
||||||
|
assert rolling_average([1, 2, 3, 4, 5], 3) == [None, None, 2.0, 3.0, 4.0]
|
||||||
|
|
||||||
|
with pytest.raises(ValueError, match="positive"):
|
||||||
|
rolling_average([1, 2], 0)
|
||||||
|
|
||||||
|
|
||||||
|
def test_dashboard_data_keeps_missing_ai_null_and_weights_valid_samples():
|
||||||
|
missing_ai = make_result(1, rating=90)
|
||||||
|
measured = make_result(
|
||||||
|
2,
|
||||||
|
rating=92,
|
||||||
|
ai_rate="1%",
|
||||||
|
ai_numerator="8",
|
||||||
|
ai_denominator="10",
|
||||||
|
)
|
||||||
|
measured.update(
|
||||||
|
{
|
||||||
|
"badMoveCount5": "1",
|
||||||
|
"badMoveCount10": "2",
|
||||||
|
"badMoveDenominator": "20",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
data = prepare_dashboard_data([missing_ai, measured])
|
||||||
|
|
||||||
|
assert data is not None
|
||||||
|
assert data["points"][0]["aiRate"] is None
|
||||||
|
assert data["points"][1]["aiRate"] == pytest.approx(80.0)
|
||||||
|
assert data["aiRate"] == pytest.approx(80.0)
|
||||||
|
assert data["aiDenominator"] == 10
|
||||||
|
assert data["aiWeighted"] is True
|
||||||
|
assert data["aiAxisMin"] == 60
|
||||||
|
assert data["badRate5"] == pytest.approx(5.0)
|
||||||
|
assert data["badRate10"] == pytest.approx(10.0)
|
||||||
|
assert data["badRateAxisMax"] == 12
|
||||||
|
assert data["badRateAxisInterval"] == 3
|
||||||
|
assert data["trendWindow"] is None
|
||||||
|
assert data["histogram"] == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_dashboard_data_uses_recent_comparison_and_rolling_trend():
|
||||||
|
records = [make_result(index, rating=79 + index) for index in range(1, 21)]
|
||||||
|
|
||||||
|
data = prepare_dashboard_data(records)
|
||||||
|
|
||||||
|
assert data is not None
|
||||||
|
assert data["trendWindow"] == 10
|
||||||
|
assert data["ratingMean"] == pytest.approx(89.5)
|
||||||
|
assert data["comparisonWindow"] == 10
|
||||||
|
assert data["recentAverage"] == pytest.approx(94.5)
|
||||||
|
assert data["comparisonDelta"] == pytest.approx(10.0)
|
||||||
|
assert data["ratingRolling"][8] is None
|
||||||
|
assert data["ratingRolling"][9] == pytest.approx(84.5)
|
||||||
|
assert data["ratingDenseLower"] == pytest.approx(84.75)
|
||||||
|
assert data["ratingDenseUpper"] == pytest.approx(94.25)
|
||||||
|
assert "ratingBandLower" not in data
|
||||||
|
assert "ratingBandUpper" not in data
|
||||||
|
assert len(data["histogram"]) == 10
|
||||||
|
assert data["histogram"][0]["lower"] == 80
|
||||||
|
assert data["histogram"][-1]["upper"] == 100
|
||||||
|
|
||||||
|
|
||||||
|
def test_rating_batches_are_newest_anchored_in_twenty_game_windows():
|
||||||
|
records = [make_result(index, rating=80 + index) for index in range(1, 46)]
|
||||||
|
|
||||||
|
data = prepare_dashboard_data(records)
|
||||||
|
|
||||||
|
assert data is not None
|
||||||
|
assert [batch["count"] for batch in data["ratingBatches"]] == [5, 20, 20]
|
||||||
|
assert [batch["startLabel"] for batch in data["ratingBatches"]] == [
|
||||||
|
"#1",
|
||||||
|
"#6",
|
||||||
|
"#26",
|
||||||
|
]
|
||||||
|
assert [batch["endLabel"] for batch in data["ratingBatches"]] == [
|
||||||
|
"#5",
|
||||||
|
"#25",
|
||||||
|
"#45",
|
||||||
|
]
|
||||||
|
assert data["ratingBatches"][-1]["ratingMean"] == pytest.approx(115.5)
|
||||||
|
|
||||||
|
|
||||||
|
def test_bad_rate_axis_uses_padded_readable_quarters():
|
||||||
|
assert visualize._rate_axis_scale([1.0, 14.4828]) == (16.0, 4.0)
|
||||||
|
assert visualize._rate_axis_scale([0.0]) == (1.0, 0.25)
|
||||||
|
assert visualize._rate_axis_scale([]) == (10.0, 2.5)
|
||||||
|
|
||||||
|
|
||||||
|
def test_generate_html_contains_professional_sections_and_escapes_nickname(
|
||||||
|
monkeypatch,
|
||||||
|
tmp_path,
|
||||||
|
):
|
||||||
|
records = [
|
||||||
|
make_result(
|
||||||
|
index,
|
||||||
|
rating=88 + index,
|
||||||
|
ai_numerator=str(80 + index),
|
||||||
|
ai_denominator="100",
|
||||||
|
)
|
||||||
|
for index in range(1, 11)
|
||||||
|
]
|
||||||
|
monkeypatch.setattr(
|
||||||
|
visualize,
|
||||||
|
"read_results",
|
||||||
|
lambda nickname, output_format, output_root=None: records,
|
||||||
|
)
|
||||||
|
output_path = tmp_path / "report.html"
|
||||||
|
|
||||||
|
result = visualize.generate_html("<测试玩家>", str(output_path))
|
||||||
|
rendered = output_path.read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
assert result == str(output_path)
|
||||||
|
assert "<测试玩家>" in rendered
|
||||||
|
assert "Rating 推移" in rendered
|
||||||
|
assert "AI 一致率推移" in rendered
|
||||||
|
assert "Rating 分布" in rendered
|
||||||
|
assert "检讨候选" in rendered
|
||||||
|
assert "半庄移动平均" in rendered
|
||||||
|
assert '"trendWindow":10' in rendered
|
||||||
|
assert "recentFocusMark" not in rendered
|
||||||
|
assert "recentFocusArea" not in rendered
|
||||||
|
assert "ratingContextArea" in rendered
|
||||||
|
assert "ratingDensityArea" in rendered
|
||||||
|
assert "全样本中间50%" in rendered
|
||||||
|
assert "density-key" in rendered
|
||||||
|
assert "Rating 中间50%" in rendered
|
||||||
|
assert '"ratingDenseLower"' in rendered
|
||||||
|
assert '"ratingDenseUpper"' in rendered
|
||||||
|
assert "trendReferenceLine" in rendered
|
||||||
|
assert "recentMarkArea" not in rendered
|
||||||
|
assert "bandRange" not in rendered
|
||||||
|
assert "smoothMonotone" in rendered
|
||||||
|
assert "rawEndLabel" in rendered
|
||||||
|
assert "trendEndLabel" not in rendered
|
||||||
|
assert "rollingBandSeries" not in rendered
|
||||||
|
assert "confidenceBandSeries" not in rendered
|
||||||
|
assert "95% 置信区间" not in rendered
|
||||||
|
assert '"ratingConfidenceLower"' not in rendered
|
||||||
|
assert '"aiConfidenceLower"' not in rendered
|
||||||
|
assert "rawSeriesData" in rendered
|
||||||
|
assert "selectedRawSeriesData" in rendered
|
||||||
|
assert "rating-selected-raw" in rendered
|
||||||
|
assert "rating-bad-rate-10-bars" in rendered
|
||||||
|
assert "rating-bad-rate-5-bars" in rendered
|
||||||
|
assert "badRateBarData" in rendered
|
||||||
|
assert "badRateXAxisConfig" in rendered
|
||||||
|
assert '"badRateAxisMax"' in rendered
|
||||||
|
assert '"badRateAxisInterval"' in rendered
|
||||||
|
assert "Rating、5% 与 10% 恶手率推移" in rendered
|
||||||
|
assert "position: 'left'" in rendered
|
||||||
|
assert "rating-batch-summary" in rendered
|
||||||
|
assert "bindBatchSelection" in rendered
|
||||||
|
assert "refreshBatchSelection" in rendered
|
||||||
|
assert rendered.count("markArea: ratingContextArea()") == 4
|
||||||
|
assert rendered.count("barMaxWidth: 7") == 2
|
||||||
|
assert '"ratingBatches"' in rendered
|
||||||
|
assert "rating-bin-highlight" in rendered
|
||||||
|
assert "rating-distribution-bars" in rendered
|
||||||
|
assert "distributionBarData" in rendered
|
||||||
|
assert "selectedRatingBinIndexes" in rendered
|
||||||
|
assert "ratingSelectionPalette" in rendered
|
||||||
|
assert "sourceEvent.ctrlKey" in rendered
|
||||||
|
assert "sourceEvent.metaKey" in rendered
|
||||||
|
assert "symbolSize: 12" in rendered
|
||||||
|
assert "rgba(255, 255, 255, 0)" not in rendered
|
||||||
|
assert "color: '#ffffff', borderColor: colors.indigoDark, borderWidth: 2" in rendered
|
||||||
|
assert "borderWidth: 2.4" not in rendered
|
||||||
|
assert "rating-selection-clear" in rendered
|
||||||
|
assert "取消选择" in rendered
|
||||||
|
assert "selectedRatingBinIndexes.length === 1 && existingPosition === 0" in rendered
|
||||||
|
assert "updateRatingSelectionControl" in rendered
|
||||||
|
assert "batch.startIndex" in rendered
|
||||||
|
assert "ai-selected-raw" in rendered
|
||||||
|
assert "rawPointData" in rendered
|
||||||
|
assert 'data-trend-view="overview"' in rendered
|
||||||
|
assert 'data-trend-view="detail"' in rendered
|
||||||
|
assert "header-metrics" in rendered
|
||||||
|
assert "header-metrics-grid" in rendered
|
||||||
|
assert "header-metrics-vertical" not in rendered
|
||||||
|
assert "header-metric-primary" in rendered
|
||||||
|
assert "全样本 Rating 平均值" in rendered
|
||||||
|
assert "header-dual-metric" in rendered
|
||||||
|
assert "review-snapshot" not in rendered
|
||||||
|
assert '"ratingBandLower"' not in rendered
|
||||||
|
assert "半庄中间50%" not in rendered
|
||||||
|
assert "观察常见水平" not in rendered
|
||||||
|
assert "section-note" not in rendered
|
||||||
|
assert "趋势预测线" not in rendered
|
||||||
|
assert "linear-gradient" not in rendered
|
||||||
|
assert "$payload" not in rendered
|
||||||
Reference in New Issue
Block a user