feat: 项目经费进度播报助手首版 v1.0.1(track-fund-progress-default)
All checks were successful
技能自动化发布 / release (push) Successful in 7s

This commit is contained in:
chenfuqiang
2026-09-13 14:25:23 +08:00
commit a91ead3862
207 changed files with 30383 additions and 0 deletions

View File

@@ -0,0 +1,204 @@
# Real Browser RPA Example
这是一个真实网站浏览器 RPA skill 的可复制参考实现,适用于需要登录、人工验证、搜索、滚动加载、采集和入库的场景。
**说明:**
- 本示例是通用模板,不绑定具体平台。
- 示例抽象自已跑通的真实浏览器 RPA skill(抖音视频链接采集),但代码已改成中性示例命名。
- 新 skill 开发时可以复制本目录,再替换目标站点、选择器、解析逻辑和业务字段。
- 只用于授权范围内的正常界面操作;禁止破解、逆向、私有协议、验证码绕过、自动换号或其他规避平台限制的扩展。
## 适用场景
- 真实网站操作(非 sandbox / mock)
- 需要登录态与 persistent Chrome/Edge profile
- 需要通过 account-manager 管理账号和 `profile_dir`
- 需要处理人工登录、滑块验证码、短信验证码
- 需要关键词搜索、滚动加载、采集可见条目
- 需要结构化日志、用户友好的停止原因、失败留痕(截图/视频)
## 目录结构
```
real_browser_rpa/
├── README.md
├── scripts/
│ ├── service/
│ │ ├── account_client.py # subprocess 调用 account-manager
│ │ ├── browser_session.py # 系统 Chrome + persistent context
│ │ ├── human_verification.py # 滑块/短信/保存登录弹窗检测(不依赖 task_rpa 错误类型)
│ │ ├── task_rpa.py # RPA 主流程
│ │ └── task_service.py # 轻量编排(示例无 DB)
│ └── util/
│ ├── constants.py # 平台常量、selector 占位
│ └── logging.py # 结构化日志与脱敏
├── assets/
│ └── element-map.sample.yaml # 元素清单模板;真实拾取后填写
└── tests/
├── test_human_verification.py # 验证码误判防护
└── test_stop_reason.py # 停止原因文案
```
## 核心流程
1. **ensure account** — 通过 `account_client.pick_web_account()`(底层 `ensure-web`)获取 `profile_dir` 与租约;无账号则自动登记,全忙则 `LEASE_CONFLICT` 不重复建号。登记 ≠ 站点已登录。
2. **启动浏览器** — `launch_persistent_context` + `new_page()` + `page.goto(start_url)`
3. **人工验证(启动后)** — 检测滑块/短信,等待用户完成
4. **等待登录** — 检测登录按钮消失 / 登录态 marker 出现
5. **保存登录弹窗** — 等待“是否保存登录信息”倒计时消失
6. **人工验证(登录后)** — 再次检测
7. **搜索** — 定位搜索框 → 输入关键词 → 点击搜索
8. **等待结果容器** — `#search-result-container`(示例 selector)
9. **滚动采集循环** — 采集可见条目 → 判断 stop_reason → 滚动加载
10. **返回结果** — `ScrapeRunResult` + `format_stop_reason_for_user()`
11. **释放租约** — `finally` 中 `release_lease()`
## 开发新 skill 时如何复制
1. **先阅读**本 README 及下方「Copy map」「复制边界」章节
2. **先完成元素工程**:阅读 `development/RPA_ELEMENT_LOCATING.md` 与 `development/RPA_BROWSER.md`,在真实站用 Pick Locator/F12 拾取并填写 element map
3. 按 copy map **选择性复制**到新技能 `scripts/`(**不要**整包照搬 `examples/real_browser_rpa/`)
4. 修改 `util/constants.py` 中的 `TARGET_PLATFORM`、`DEFAULT_START_URL`、selector 常量
5. 替换 `task_rpa.py` 中的登录态 marker、搜索框/结果解析逻辑(**必须 F12 实测**)
6. 替换 `human_verification.py` 中的验证码 selector(若目标站点 DOM 不同)
7. 在 `task_service.py` 中接入真实 DB、RpaVideoSession
8. 在新技能根目录补齐 `scripts/main.py` CLI;`SKILL.md` 与根 `README.md` 按模板主目录规则编写,**不要**复制本 README
## Copy map(复制到新技能 `scripts/`)
| 来源(examples) | 目标(新技能) | 说明 |
|---|---|---|
| `examples/real_browser_rpa/scripts/service/browser_session.py` | `scripts/service/browser_session.py` | persistent context 启动范式 |
| `examples/real_browser_rpa/scripts/service/account_client.py` | `scripts/service/account_client.py` | account-manager CLI 集中封装 |
| `examples/real_browser_rpa/scripts/service/human_verification.py` | `scripts/service/human_verification.py` | 人工验证等待 |
| `examples/real_browser_rpa/scripts/service/task_rpa.py` | `scripts/service/<业务名>_rpa.py` 或 `scripts/service/task_rpa.py` | RPA 主流程,按业务改名 |
| `examples/real_browser_rpa/scripts/service/task_service.py` | `scripts/service/task_service.py` | 参考并合并编排层 |
| `examples/real_browser_rpa/scripts/util/constants.py` | `scripts/util/constants.py` | 平台常量与 selector(须替换) |
| `examples/real_browser_rpa/scripts/util/logging.py` | `scripts/util/logging.py` | 可选,按需合并 |
| `examples/real_browser_rpa/assets/element-map.sample.yaml` | `assets/<platform>-element-map.yaml` | 真实 Pick Locator/F12 拾取后填写 |
| `examples/real_browser_rpa/tests/` | `tests/` | 可按需复制测试思路 |
| `examples/real_browser_rpa/README.md` | — | **只读参考**,不复制为新技能根 README |
**复制时必须替换:** selector、URL、平台名称、账号平台字段、slug、logger 名等业务标识。**不要**照搬示例字段到其他平台。
**浏览器与启动约束:**
- **先 Profile 再开浏览器**:必须 `pick_web_account`(底层 ensure-web)拿到 `profile_dir`;拿不到则失败退出,禁止落到系统默认用户目录(见模板 `development/RPA.md` §0.2)
- **RPA 等待**:主路径用 `interruptible_sleep`,禁止裸 `asyncio.sleep`(`POLICY-CONTROL-003`)
- **有头**:生产 / 联调 `OPENCLAW_BROWSER_HEADLESS=0`;勿把无头当交付常态
- **登录按平台**:是否要求站点登录由本技能约定;「可不登录」≠「可不接 account-manager」
- **不要**在技能内安装 Playwright;浏览器与 Python 包由宿主/runtime 提供
- `launch_persistent_context` 的 `args` **只放 Chrome 参数**;**不要把 URL 放进 `args`**
- 页面必须通过 `new_page()` + `goto()`,或通过真实地址栏/点击进入
## 复制边界:哪些能复制,哪些必须替换
### 可以直接参考/复制的
- `browser_session.py` 的启动结构:**persistent context → new_page → goto**
- `human_verification.py` 的人工验证等待模式(容器级检测 + 等待人工,不自动破解)
- `task_rpa.py` 的流程分层、结构化日志点、`stop_reason` 设计
- `account_client.py` 的 account-manager CLI **集中封装**方式(subprocess 仅限此文件)
- `tests/test_human_verification.py` 的误判防护测试思路
### 必须替换的
- `SKILL_SLUG`
- `TARGET_PLATFORM`
- `LEASE_HOLDER`
- `DEFAULT_START_URL`
- `SEARCH_INPUT_SELECTORS` / `SEARCH_BUTTON_SELECTOR`
- `RESULT_CONTAINER_SELECTOR` / `RESULT_ITEM_SELECTOR`
- `SCROLL_CONTAINER_SELECTORS`
- `RESULT_END_TEXT`
- `LOGGED_IN_SELECTORS` / `LOGGED_OUT_SELECTOR`(见下方说明)
- `_parse_item_locator()` 业务字段解析
- 用户可见中文文案
- 数据库写入逻辑
- 录屏/截图业务标题
### 禁止照抄的
- 不要照抄目标站点 selector(示例 selector 不等于真实 DOM)
- 不要照抄平台名、slug、logger 名
- 不要依赖 `D:\OpenClaw\client-commons\account-manager\scripts\main.py` 等开发机绝对路径作为生产依赖
- 不要把示例里的 result selector 当成真实站点 selector
- 不要复制“扫描所有 input 再按宽度/位置打分”的诊断式猜测作为生产主定位器
- 不要用 `.first` / `.nth` 绕过多匹配;关键动作必须先解决唯一性
- 不要把示例代码原样发布为业务 skill
- 不要把真实账号、手机号、cookie、token 写入代码或 README
### 登录 selector 说明
`task_rpa.py` 中的 `LOGGED_IN_SELECTORS` 与 `LOGGED_OUT_SELECTOR` **只是示例**,偏中文「登录」按钮和常见站点结构。复制到新 skill 后:
- **必须**用目标站点 F12 / DOM 实测后替换
- **不允许**凭经验猜 selector
- **不允许**因为示例在本机曾跑通就直接照搬到其他平台
### account-manager 路径解析说明
`account_client.py` 末尾的 `D:\OpenClaw\client-commons\account-manager\scripts\main.py` 是**开发环境兜底路径**,仅用于模板/本机调试。复制到真实 skill 后:
- **优先**通过 `ACCOUNT_MANAGER_ROOT`、Platform Kit `get_sibling_skills_root` 或宿主运行环境解析 account-manager
- **不要**把个人机器路径作为生产依赖
## 必须保留的安全原则
- **不 import account-manager 内部模块** — 只通过 CLI/subprocess 调用
- **不自动破解验证码** — 滑块/短信只检测 + 等待人工完成
- **不对抗平台风控** — 频控、账号风险、封禁或权限不足时停止并交由人工处理
- **日志脱敏** — 不输出完整手机号/账号
- **租约释放** — `ensure-web` / `pick-web --lease` 后必须在 `finally` 释放
- **失败留痕** — 真实 skill 应在关键失败点截图(示例中已留注释位)
## 常见坑
| 坑 | 正确做法 |
|---|---|
| 直接 `import account_manager.service...` | 通过 `subprocess` 调用 `account-manager/scripts/main.py` |
| 把 URL 放进 `launch_persistent_context(args=[url])` | `args` 只放 Chrome 参数;URL 用 `page.goto(start_url)` |
| 启动后直接操作 DOM | 先 `context.new_page()` 再 `page.goto()` |
| 用整页 `body.inner_text()` 判断验证码 | 限定在 `[role='dialog']`、`.vc-captcha-verify` 等容器 |
| 搜索结果含“验证码教程”文字就误判 | 单元测试覆盖:无验证 DOM 时不应 `present=True` |
| 无结构化日志 | 关键节点打 `rpa_start` / `browser_ready` / `collect_round` 等 |
| 把 CLI 参数暴露给普通用户 | 用户说明放根 `README.md`;运行契约与触发规则放 `SKILL.md`(LLM/平台入口) |
## 需要替换的内容清单
| 文件 | 替换项 |
|---|---|
| `util/constants.py` | `SKILL_SLUG`、`TARGET_PLATFORM`、`DEFAULT_START_URL`、所有 selector |
| `browser_session.py` | `get_start_url()` 配置来源、stealth 脚本(可选) |
| `human_verification.py` | `SLIDER_SELECTORS`、`SMS_SELECTORS`、`DIALOG_TEXT_CONTAINERS` |
| `task_rpa.py` | `LOGGED_IN_SELECTORS`、`LOGGED_OUT_SELECTOR`、`_parse_item_locator()` |
| `account_client.py` | `PLACEHOLDER_PLATFORM`、`LEASE_HOLDER`、账号 setup 文案 |
| `task_service.py` | 接入 DB、entitlement、视频留痕 |
## 依赖说明
本目录是 **示例**,不影响 skill-template 根目录运行。真实 skill 需要:
- Python 3.10+
- Python 包 `playwright`:属于**宿主共享 runtime** 的通用能力;技能侧**不要**在 `requirements.txt` 中重复声明,也**不要**在代码或脚本中执行 `pip install playwright`
- 真实 skill **默认使用系统 Chrome/Edge**,**不使用** Playwright 下载的内置 Chromium
- 技能侧**不要**执行 `playwright install`、`playwright install chromium` 或类似命令
- 若本地独立开发环境缺少 Playwright Python 包,只能作为开发机临时环境补装;**生产/宿主运行**由共享 runtime 解决
- `jiangchang_skill_core`(如 `find_chrome_executable` 等,由 platform-kit 提供)
- 兄弟 skill `account-manager`
## 运行测试
```bash
cd examples/real_browser_rpa
python -m pytest tests/ -v
```
测试为纯函数/Mock DOM,**不需要启动真实浏览器**。
## 当前状态
- 示例代码已就绪,可直接作为新 skill 复制起点。
- 未绑定具体平台业务字段;selector 为占位示例。
- 未接入真实数据库与 CLI 入口。

View File

@@ -0,0 +1,45 @@
surface: browser
status: placeholder_requires_real_site_inspection
elements:
- logical_name: search_input
purpose: 输入搜索关键词
scope:
kind: main
business_container: 待真实拾取
primary_locator:
strategy: role_or_label
value: 待真实拾取
fallback_locators: []
expected_count: 1
ready_condition:
visible: true
enabled: true
postcondition:
description: 搜索结果容器出现
verified_target_version: 待填写
- logical_name: search_button
purpose: 提交搜索
scope:
logical_parent: search_form
primary_locator:
strategy: role
value: button,name=搜索
fallback_locators: []
expected_count: 1
postcondition:
description: 结果页或业务错误状态出现
verified_target_version: 待填写
- logical_name: result_container
purpose: 承载搜索结果
scope:
kind: main
primary_locator:
strategy: stable_business_container
value: 待真实拾取
fallback_locators: []
expected_count: 1
postcondition:
description: 列表项、空结果或结束标记可区分
verified_target_version: 待填写

View File

@@ -0,0 +1,355 @@
"""account-manager CLI 集成(仅 subprocess,不 import 兄弟 skill 内部模块)。
业务首启默认 ``ensure-web``(有则用、无则 add-web);登记 ≠ 站点已登录。
"""
from __future__ import annotations
import json
import logging
import os
import subprocess
import sys
from typing import Any, Dict, List, Optional, Tuple
from jiangchang_skill_core import config
from jiangchang_skill_core.runtime_env import get_sibling_skills_root
from util.constants import LEASE_HOLDER, LEASE_TTL_SEC, TARGET_PLATFORM
from util.logging import mask_text
logger = logging.getLogger(__name__)
PLACEHOLDER_PLATFORM = TARGET_PLATFORM
ACCOUNT_SETUP_MESSAGE = (
f"未能取得可用的 {PLACEHOLDER_PLATFORM} 账号(ensure-web 失败),所以还没有打开浏览器。"
f"请确认已安装 account-manager,并检查 platform={PLACEHOLDER_PLATFORM} 账号状态后重试。"
)
LEASE_BUSY_MESSAGE = (
"该平台下已有账号,但均被其他任务占用。请稍后重试或释放租约;不会自动新建账号。"
)
DEFAULT_AUTH_STRATEGY = "qr_code_manual"
DEFAULT_ACCOUNT_LABEL = f"{PLACEHOLDER_PLATFORM} 默认"
DEFAULT_START_URL = "https://www.example.com/"
class AccountManagerError(Exception):
def __init__(self, code: str, message: str) -> None:
super().__init__(message)
self.code = code
self.message = message
def mask_login_id(login_id: str) -> str:
return mask_text(login_id)
def format_account_label(account: dict) -> str:
"""人读账号标识:优先 login_id / 备注,最后才用数字 id。"""
for key in ("login_id", "account_label", "label"):
v = account.get(key)
if v is not None and str(v).strip():
return str(v).strip()
for key in ("id", "account_id"):
v = account.get(key)
if v is not None and str(v).strip():
return str(v).strip()
return ""
def _is_placeholder_config(raw: str) -> bool:
s = (raw or "").strip()
if not s:
return True
if s.startswith("#"):
return True
low = s.lower()
if low in ("default-account", "none", "null", "n/a", "-"):
return True
return False
def resolve_account_selectors_from_config() -> Tuple[Optional[str], Optional[str]]:
"""从配置解析 (account_id, login_id)。
- ``DEFAULT_ACCOUNT_ID``:仅纯数字视为账号主键;其它非空非占位当作 login_id
- ``DEFAULT_LOGIN_ID``:手机号/用户名(可覆盖上一项推导的 login_id)
- 两者皆空:返回 (None, None) → 调用方走按平台 ensure-web
"""
account_id: Optional[str] = None
login_id: Optional[str] = None
aid_raw = (config.get("DEFAULT_ACCOUNT_ID") or "").strip()
if not _is_placeholder_config(aid_raw):
if aid_raw.isdigit():
account_id = aid_raw
else:
login_id = aid_raw
logger.info("default_account_id_treated_as_login_id value=%s", aid_raw)
lid_raw = (config.get("DEFAULT_LOGIN_ID") or "").strip()
if not _is_placeholder_config(lid_raw):
login_id = lid_raw
if account_id:
return account_id, None
return None, login_id
def _resolve_account_manager_main() -> str:
"""解析 account-manager CLI 入口路径。
优先级:ACCOUNT_MANAGER_ROOT → get_sibling_skills_root → 开发机兜底。
末尾 dev 路径仅用于模板/本机调试,复制到真实 skill 后不要依赖个人机器绝对路径。
"""
env_root = (config.get("ACCOUNT_MANAGER_ROOT") or os.getenv("ACCOUNT_MANAGER_ROOT") or "").strip()
if env_root and os.path.isfile(os.path.join(env_root, "scripts", "main.py")):
return os.path.join(os.path.abspath(env_root), "scripts", "main.py")
scripts_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
skills_root = get_sibling_skills_root(scripts_dir)
candidate = os.path.join(skills_root, "account-manager", "scripts", "main.py")
if os.path.isfile(candidate):
return candidate
# 开发环境兜底:仅模板/本机调试;生产依赖宿主注入的路径变量,勿硬编码个人目录
dev = r"D:\OpenClaw\client-commons\account-manager\scripts\main.py"
if os.path.isfile(dev):
return dev
raise AccountManagerError(
"ACCOUNT_NOT_FOUND",
"未找到 account-manager,请配置 ACCOUNT_MANAGER_ROOT 或安装 account-manager 技能。",
)
def _parse_last_json(stdout: str) -> Any:
lines = [ln.strip() for ln in (stdout or "").splitlines() if ln.strip()]
for raw in reversed(lines):
if raw.startswith("ERROR:"):
continue
try:
return json.loads(raw)
except json.JSONDecodeError:
continue
raise AccountManagerError("UNKNOWN_ERROR", "stdout 中没有可解析的 JSON。")
def _run_argv(argv_suffix: List[str]) -> subprocess.CompletedProcess[str]:
main_py = _resolve_account_manager_main()
env = os.environ.copy()
return subprocess.run(
[sys.executable, main_py, *argv_suffix],
capture_output=True,
text=True,
encoding="utf-8",
errors="replace",
env=env,
)
def _normalize_error_code(raw: str) -> str:
code = (raw or "").strip()
if code.startswith("ERROR:"):
code = code[6:]
return code
def _validate_pick_payload(payload: dict[str, Any]) -> dict[str, Any]:
if not isinstance(payload, dict):
raise AccountManagerError("ACCOUNT_NOT_FOUND", "ensure-web 返回格式异常。")
if payload.get("success") is False:
err = payload.get("error") if isinstance(payload.get("error"), dict) else {}
code = _normalize_error_code(str(err.get("code") or ""))
message = str(err.get("message") or "")
if code == "LEASE_CONFLICT" or "LEASE_CONFLICT" in code:
raise AccountManagerError("LEASE_CONFLICT", message or LEASE_BUSY_MESSAGE)
if code == "AMBIGUOUS_ACCOUNT" or "AMBIGUOUS_ACCOUNT" in code:
raise AccountManagerError(
"AMBIGUOUS_ACCOUNT",
message or "匹配到多条账号,请改用数字 id 或更精确的登录标识。",
)
if code in ("NO_ACCOUNT", "ACCOUNT_NOT_FOUND") or "NO_ACCOUNT" in code:
raise AccountManagerError("NO_ACCOUNT", message or "没有可用账号。")
raise AccountManagerError(code or "ENSURE_WEB_FAILED", message or "ensure-web 失败。")
if not payload.get("profile_dir"):
raise AccountManagerError("ACCOUNT_NOT_FOUND", "ensure-web 返回的账号缺少 profile_dir。")
if not payload.get("lease_token"):
raise AccountManagerError("LEASE_REQUIRED", "account-manager 未返回 lease_token,禁止打开浏览器。")
return payload
def _ensure_web_with_lease(
platform: str,
*,
login_id: Optional[str] = None,
) -> Dict[str, Any]:
start_url = (config.get("TARGET_BASE_URL") or DEFAULT_START_URL).strip() or DEFAULT_START_URL
lid = (login_id or "").strip() or None
label = f"{platform} {lid}" if lid else DEFAULT_ACCOUNT_LABEL
argv = [
"account",
"ensure-web",
"--platform",
platform,
"--url",
start_url,
"--auth-strategy",
DEFAULT_AUTH_STRATEGY,
"--label",
label,
"--lease",
"--holder",
LEASE_HOLDER,
"--ttl-sec",
LEASE_TTL_SEC,
"--purpose",
"rpa",
]
if lid:
argv.extend(["--login-id", lid])
proc = _run_argv(argv)
out = proc.stdout or ""
if proc.returncode != 0 and not out.strip():
raise AccountManagerError(
"ENSURE_WEB_FAILED",
(proc.stderr or "").strip() or "ensure-web 子进程失败",
)
payload = _parse_last_json(out)
if isinstance(payload, dict) and payload.get("success") is False:
return _validate_pick_payload(payload)
if not isinstance(payload, dict):
raise AccountManagerError("ACCOUNT_NOT_FOUND", "ensure-web 返回格式异常。")
return _validate_pick_payload(payload)
def _pick_existing_with_lease(
platform: str,
*,
login_id: Optional[str] = None,
label: Optional[str] = None,
) -> Dict[str, Any]:
argv = [
"account",
"pick-web",
"--platform",
platform,
"--lease",
"--holder",
LEASE_HOLDER,
"--ttl-sec",
LEASE_TTL_SEC,
"--purpose",
"rpa",
]
if login_id:
argv.extend(["--login-id", login_id])
elif label:
argv.extend(["--label", label])
else:
raise AccountManagerError("AMBIGUOUS_ACCOUNT", "指定账号缺少可用于带租约选择的登录标识或名称。")
proc = _run_argv(argv)
out = proc.stdout or ""
if proc.returncode != 0 and not out.strip():
raise AccountManagerError(
"PICK_WEB_FAILED",
(proc.stderr or "").strip() or "pick-web 子进程失败",
)
payload = _parse_last_json(out)
return _validate_pick_payload(payload)
def _pick_by_id_with_lease(platform: str, account_id: int) -> Dict[str, Any]:
proc = _run_argv(["account", "get", str(account_id)])
out = proc.stdout or ""
if proc.returncode != 0 or out.startswith("ERROR:"):
raise AccountManagerError("ACCOUNT_NOT_FOUND", f"指定账号 {account_id} 不存在或获取失败。")
raw = _parse_last_json(out)
if isinstance(raw, dict) and "data" in raw and isinstance(raw["data"], dict):
data = raw["data"]
elif isinstance(raw, dict):
data = raw
else:
raise AccountManagerError("ACCOUNT_NOT_FOUND", "account get 返回非对象 JSON。")
if data.get("success") is False:
raise AccountManagerError("ACCOUNT_NOT_FOUND", f"指定账号 {account_id} 不存在或获取失败。")
platform_key = str(data.get("platform_key") or "").lower()
if platform_key != platform.lower():
raise AccountManagerError(
"ACCOUNT_NOT_FOUND",
f"账号 {account_id} 不是 {platform} 平台账号。",
)
profile_dir = str(data.get("profile_dir") or "").strip()
if not profile_dir:
raise AccountManagerError("ACCOUNT_NOT_FOUND", f"账号 {account_id} 缺少 profile_dir。")
if str(data.get("status") or "active").lower() != "active":
raise AccountManagerError("ACCOUNT_NOT_FOUND", f"账号 {account_id} 当前不是 active 状态。")
try:
selected = _pick_existing_with_lease(
platform,
login_id=(str(data.get("login_id") or "").strip() or None),
label=(str(data.get("account_label") or "").strip() or None),
)
except AccountManagerError as exc:
if exc.code in ("NO_ACCOUNT", "ACCOUNT_NOT_FOUND"):
raise AccountManagerError("LEASE_CONFLICT", LEASE_BUSY_MESSAGE) from exc
raise
if str(selected.get("id")) != str(account_id):
release_lease(str(selected.get("lease_token") or ""))
raise AccountManagerError(
"AMBIGUOUS_ACCOUNT",
f"无法唯一取得指定账号 {account_id} 的租约,请检查重复的登录标识或账号名称。",
)
return selected
def pick_web_account(
platform: str,
account_id: Optional[str] = None,
login_id: Optional[str] = None,
) -> dict:
"""获取网页账号(profile_dir + lease_token)。
优先级:数字 account_id → login_id(ensure-web --login-id)→ 按平台 ensure-web。
"""
platform_key = (platform or PLACEHOLDER_PLATFORM).strip() or PLACEHOLDER_PLATFORM
aid_raw = (account_id or "").strip()
if aid_raw and not _is_placeholder_config(aid_raw):
if not aid_raw.isdigit():
if not (login_id or "").strip():
login_id = aid_raw
aid_raw = ""
else:
return _pick_by_id_with_lease(platform_key, int(aid_raw))
lid = (login_id or "").strip() or None
if lid and _is_placeholder_config(lid):
lid = None
try:
return _ensure_web_with_lease(platform_key, login_id=lid)
except AccountManagerError as exc:
if exc.code in ("NO_ACCOUNT", "ACCOUNT_NOT_FOUND") or "NO_ACCOUNT" in exc.code:
raise AccountManagerError("ACCOUNT_SETUP_REQUIRED", ACCOUNT_SETUP_MESSAGE) from exc
raise
def release_lease(lease_token: Optional[str]) -> None:
token = (lease_token or "").strip()
if not token:
return
try:
proc = _run_argv(["lease", "release", token])
if proc.returncode != 0:
logger.warning("lease_release_failed stdout=%s", (proc.stdout or "").strip())
except Exception:
logger.warning("lease_release_exception", exc_info=True)

View File

@@ -0,0 +1,135 @@
"""浏览器会话启动(系统 Chrome/Edge + Playwright persistent context)。"""
from __future__ import annotations
import logging
import os
from typing import Any, Optional, Tuple
from util.constants import DEFAULT_START_URL
logger = logging.getLogger(__name__)
DEFAULT_PAGE_TIMEOUT_MS = 15_000
GOTO_TIMEOUT_MS = 60_000
STEALTH_INIT_SCRIPT = """
(() => {
try { Object.defineProperty(navigator, 'webdriver', { get: () => undefined }); } catch (e) {}
})();
"""
def get_start_url() -> str:
return (os.getenv("TARGET_START_URL") or DEFAULT_START_URL).strip() or DEFAULT_START_URL
def _headless_from_env() -> bool:
v = (os.getenv("OPENCLAW_BROWSER_HEADLESS") or "0").strip().lower()
return v in ("1", "true", "yes", "on")
def _find_chrome_executable() -> str | None:
try:
from jiangchang_skill_core.runtime_env import find_chrome_executable
chrome = find_chrome_executable()
if chrome and os.path.isfile(chrome):
return chrome
except ImportError:
pass
candidates = [
os.path.expandvars(r"%ProgramFiles%\Google\Chrome\Application\chrome.exe"),
os.path.expandvars(r"%ProgramFiles(x86)%\Google\Chrome\Application\chrome.exe"),
os.path.expandvars(r"%ProgramFiles%\Microsoft\Edge\Application\msedge.exe"),
os.path.expandvars(r"%ProgramFiles(x86)%\Microsoft\Edge\Application\msedge.exe"),
]
for path in candidates:
if path and os.path.isfile(path):
return path
return None
async def start_browser_session(
profile_dir: str,
*,
headless: Optional[bool] = None,
) -> Tuple[Any, Any, Any]:
"""
启动持久化 Chrome profile,new_page + goto 入口页,返回 (playwright, context, page)。
真实 skill 需要安装 playwright 与 jiangchang_skill_core。
"""
from playwright.async_api import async_playwright
hl = headless if headless is not None else _headless_from_env()
start_url = get_start_url()
chrome = _find_chrome_executable()
if not chrome:
raise RuntimeError(
"ERROR:MISSING_BROWSER 未检测到 Chrome 或 Edge 浏览器,请先安装系统浏览器后重试。"
)
launch_args = ["--start-maximized", "--disable-blink-features=AutomationControlled"]
logger.info(
"browser_launch chrome=%s profile_dir=%s start_url=%s args=%s headless=%s",
chrome,
profile_dir,
start_url,
launch_args,
hl,
)
print(f"[浏览器] 系统 Chrome: {chrome}")
print(f"[浏览器] profile_dir: [{profile_dir}]")
print(f"[浏览器] startup_url={start_url}")
print(f"[浏览器] args={launch_args}")
pw = await async_playwright().start()
context = None
try:
context = await pw.chromium.launch_persistent_context(
user_data_dir=profile_dir,
headless=hl,
executable_path=chrome,
locale="zh-CN",
no_viewport=True,
args=launch_args,
ignore_default_args=["--enable-automation"],
)
await context.add_init_script(STEALTH_INIT_SCRIPT)
page = await context.new_page()
page.set_default_timeout(DEFAULT_PAGE_TIMEOUT_MS)
try:
await page.goto(start_url, wait_until="domcontentloaded", timeout=GOTO_TIMEOUT_MS)
except Exception as exc:
raise RuntimeError(
f"浏览器已启动,但未能打开目标首页 {start_url}。"
"请检查网络、Chrome profile、代理或页面访问状态。"
f"({exc})"
) from exc
return pw, context, page
except Exception:
if context is not None:
try:
await context.close()
except Exception:
pass
await pw.stop()
raise
async def close_browser_context(pw, context) -> None:
try:
if context is not None:
await context.close()
except Exception:
pass
try:
if pw is not None:
await pw.stop()
except Exception:
pass

View File

@@ -0,0 +1,450 @@
"""人工验证等待:保存登录倒计时、滑块、短信验证码等(不自动破解)。"""
from __future__ import annotations
import asyncio
import logging
import random
import re
import time
from dataclasses import dataclass
from typing import Optional
from util.constants import LOG_LOGGER_NAME
logger = logging.getLogger(LOG_LOGGER_NAME)
TEXT_EXCERPT_MAX = 120
SAVE_LOGIN_STRONG_MARKERS = (
"是否保存登录信息",
"下次登录更便捷",
"保存登录信息",
)
SAVE_LOGIN_CONTAINER_SELECTORS = (
"[role=\"dialog\"]",
".semi-modal",
".semi-modal-content",
)
SLIDER_SELECTORS = [
".vc-captcha-verify.slide",
".captcha_verify_bar--title",
".captcha-slider-tips",
"#captcha_verify_image",
".captcha-slider-btn",
]
SMS_SELECTORS = [
".second-verify-panel",
".uc-ui-verify_sms-input",
".uc-ui-verify-new_header-title",
"input[placeholder='手机号']",
"input[placeholder='请输入验证码']",
]
DIALOG_TEXT_CONTAINERS = [
"[role='dialog']",
".semi-modal",
".semi-modal-content",
".vc-captcha-verify",
".second-verify-panel",
]
RISK_MARKERS = (
"操作频繁",
"访问过于频繁",
"风险验证",
"安全验证",
)
@dataclass
class HumanVerificationState:
present: bool
kind: str
message: str
selector: str = ""
text_excerpt: str = ""
@dataclass
class HumanVerificationWaitResult:
timed_out: bool
kind: str = ""
code: str = ""
message: str = ""
stage: str = ""
def _text_excerpt(text: str) -> str:
cleaned = re.sub(r"\s+", " ", (text or "").strip())
if len(cleaned) <= TEXT_EXCERPT_MAX:
return cleaned
return cleaned[:TEXT_EXCERPT_MAX] + "..."
def _contains_any(text: str, markers: tuple[str, ...]) -> bool:
return any(m in text for m in markers)
async def _selector_visible(page, selector: str) -> bool:
try:
loc = page.locator(selector).first
return await loc.is_visible(timeout=500)
except Exception:
return False
async def _selector_inner_text(page, selector: str) -> str:
try:
return await page.locator(selector).first.inner_text(timeout=1000)
except Exception:
return ""
async def _visible_text_from_first_visible(page, selectors: tuple[str, ...] | list[str]) -> tuple[str, str]:
for sel in selectors:
try:
loc = page.locator(sel).first
if await loc.is_visible(timeout=500):
text = await loc.inner_text(timeout=1000)
return sel, text or ""
except Exception:
continue
return "", ""
def _save_login_popup_from_text(text: str) -> bool:
if not text:
return False
if _contains_any(text, SAVE_LOGIN_STRONG_MARKERS):
return True
if "保存" in text and "取消" in text and ("登录" in text or "登录信息" in text):
return True
return False
async def detect_save_login_popup(page) -> bool:
sel, text = await _visible_text_from_first_visible(page, SAVE_LOGIN_CONTAINER_SELECTORS)
if sel and _save_login_popup_from_text(text):
return True
return False
async def wait_save_login_countdown_if_present(page, timeout_sec: int = 8) -> bool:
"""若出现“保存登录信息”倒计时弹窗则等待其消失;未出现则立即返回 False。"""
if not await detect_save_login_popup(page):
return False
logger.info("save_login_popup_detected")
print("[登录] 检测到“保存登录信息”倒计时弹窗,等待其自动关闭...")
start = time.monotonic()
deadline = start + timeout_sec
while time.monotonic() < deadline:
if not await detect_save_login_popup(page):
elapsed = round(time.monotonic() - start, 2)
logger.info("save_login_popup_gone elapsed_sec=%s", elapsed)
return True
await asyncio.sleep(0.5)
elapsed = round(time.monotonic() - start, 2)
logger.warning(
"save_login_popup_still_visible elapsed_sec=%s timeout_sec=%s",
elapsed,
timeout_sec,
)
print(
f"[登录] 警告:保存登录信息弹窗超过 {timeout_sec} 秒仍未消失,"
"将继续后续流程(您可能已手工点击保存/取消)。"
)
return True
async def detect_slider_verification(page) -> Optional[HumanVerificationState]:
sel_slide = ".vc-captcha-verify.slide"
if await _selector_visible(page, sel_slide):
text = await _selector_inner_text(page, sel_slide)
return HumanVerificationState(
present=True,
kind="slider",
message="滑块验证码",
selector=sel_slide,
text_excerpt=_text_excerpt(text),
)
sel_title = ".captcha_verify_bar--title"
if await _selector_visible(page, sel_title):
title_text = await _selector_inner_text(page, sel_title)
if "请完成下列验证后继续" in title_text:
return HumanVerificationState(
present=True,
kind="slider",
message="滑块验证码",
selector=sel_title,
text_excerpt=_text_excerpt(title_text),
)
sel_tips = ".captcha-slider-tips"
if await _selector_visible(page, sel_tips):
tips_text = await _selector_inner_text(page, sel_tips)
if "拖动完成上方拼图" in tips_text:
return HumanVerificationState(
present=True,
kind="slider",
message="滑块验证码",
selector=sel_tips,
text_excerpt=_text_excerpt(tips_text),
)
if await _selector_visible(page, "#captcha_verify_image"):
for btn_sel in (".captcha-slider-btn", ".captcha_verify_slide--button", ".captcha-slider"):
if await _selector_visible(page, btn_sel):
return HumanVerificationState(
present=True,
kind="slider",
message="滑块验证码",
selector=f"#captcha_verify_image+{btn_sel}",
text_excerpt="",
)
return None
async def detect_sms_verification(page) -> Optional[HumanVerificationState]:
sel_panel = ".second-verify-panel"
if await _selector_visible(page, sel_panel):
text = await _selector_inner_text(page, sel_panel)
return HumanVerificationState(
present=True,
kind="sms",
message="短信验证码",
selector=sel_panel,
text_excerpt=_text_excerpt(text),
)
sel_sms = ".uc-ui-verify_sms-input"
if await _selector_visible(page, sel_sms):
text = await _selector_inner_text(page, sel_sms)
return HumanVerificationState(
present=True,
kind="sms",
message="短信验证码",
selector=sel_sms,
text_excerpt=_text_excerpt(text),
)
sel_hdr = ".uc-ui-verify-new_header-title"
if await _selector_visible(page, sel_hdr):
hdr_text = await _selector_inner_text(page, sel_hdr)
if "短信验证" in hdr_text:
return HumanVerificationState(
present=True,
kind="sms",
message="短信验证码",
selector=sel_hdr,
text_excerpt=_text_excerpt(hdr_text),
)
phone_visible = await _selector_visible(
page,
".second-verify-panel input[placeholder='手机号'], .uc-ui-verify_sms-input input[placeholder='手机号']",
)
code_visible = await _selector_visible(
page,
".second-verify-panel input[placeholder='请输入验证码'], .uc-ui-verify_sms-input input[placeholder='请输入验证码']",
)
if phone_visible and code_visible:
return HumanVerificationState(
present=True,
kind="sms",
message="短信验证码",
selector="sms-phone-and-code-inputs",
text_excerpt="手机号+请输入验证码",
)
return None
async def detect_human_verification_from_containers(page) -> Optional[HumanVerificationState]:
sel, text = await _visible_text_from_first_visible(page, DIALOG_TEXT_CONTAINERS)
if not sel or not text:
return None
if _contains_any(text, RISK_MARKERS):
return HumanVerificationState(
present=True,
kind="risk",
message="安全/频控验证",
selector=sel,
text_excerpt=_text_excerpt(text),
)
if sel in (".vc-captcha-verify", ".vc-captcha-verify-visibility"):
if "请完成下列验证后继续" in text or "拖动完成上方拼图" in text:
return HumanVerificationState(
present=True,
kind="slider",
message="滑块验证码",
selector=sel,
text_excerpt=_text_excerpt(text),
)
if sel in (".second-verify-panel", ".uc-ui-verify_sms-input"):
if "短信验证" in text or "获取验证码" in text:
return HumanVerificationState(
present=True,
kind="sms",
message="短信验证码",
selector=sel,
text_excerpt=_text_excerpt(text),
)
return None
async def detect_human_verification(page) -> HumanVerificationState:
"""主检测入口:仅在验证码/弹窗容器级检测,不扫整页 body。"""
state = await detect_slider_verification(page)
if state is not None:
return state
state = await detect_sms_verification(page)
if state is not None:
return state
state = await detect_human_verification_from_containers(page)
if state is not None:
return state
return HumanVerificationState(present=False, kind="", message="", selector="", text_excerpt="")
def detect_human_verification_from_text(text: str) -> HumanVerificationState:
"""
根据 HTML/容器文本判断验证弹窗(单元测试辅助,不能作为真实页面主检测入口)。
只检查验证容器 DOM 标记,不根据正文关键词误判。
"""
html_lower = (text or "").lower()
has_slider_dom = (
"vc-captcha-verify" in html_lower
and "slide" in html_lower
and (
"captcha-slider-tips" in html_lower
or "captcha_verify_bar--title" in html_lower
or "captcha_verify_image" in html_lower
)
)
if has_slider_dom:
selector = ".vc-captcha-verify.slide"
if "captcha-slider-tips" in html_lower:
selector = ".captcha-slider-tips"
return HumanVerificationState(
present=True,
kind="slider",
message="滑块验证码",
selector=selector,
text_excerpt="",
)
has_sms_dom = "second-verify-panel" in html_lower or "uc-ui-verify_sms-input" in html_lower
if has_sms_dom:
selector = ".second-verify-panel" if "second-verify-panel" in html_lower else ".uc-ui-verify_sms-input"
return HumanVerificationState(
present=True,
kind="sms",
message="短信验证码",
selector=selector,
text_excerpt="",
)
return HumanVerificationState(present=False, kind="", message="", selector="", text_excerpt="")
def _wait_log_message(state: HumanVerificationState, wait_sec: int) -> str:
if state.kind == "slider":
return f"[人工验证] 检测到滑块验证码,请在浏览器中手工完成,最多等待 {wait_sec} 秒..."
if state.kind == "sms":
return f"[人工验证] 检测到短信验证码,请在浏览器中手工完成,最多等待 {wait_sec} 秒..."
return f"[人工验证] 检测到安全验证,请在浏览器中手工处理,最多等待 {wait_sec} 秒..."
def _timeout_error_code(kind: str) -> str:
if kind == "slider":
return "SLIDER_VERIFICATION_TIMEOUT"
if kind == "sms":
return "SMS_VERIFICATION_TIMEOUT"
return "HUMAN_VERIFICATION_TIMEOUT"
def _timeout_error_message(kind: str, wait_sec: int) -> str:
if kind == "slider":
return (
f"滑块验证码未在 {wait_sec} 秒内完成,任务已停止。"
"请重新运行任务并及时在浏览器中完成验证。"
)
if kind == "sms":
return (
f"短信验证码未在 {wait_sec} 秒内完成,任务已停止。"
"请重新运行任务并及时完成短信验证。"
)
return (
f"安全验证未在 {wait_sec} 秒内完成,任务已停止。"
"请重新运行任务并及时在浏览器中完成验证。"
)
async def wait_human_verification_if_present(
page,
*,
wait_sec: int = 180,
stage: str = "unknown",
) -> Optional[HumanVerificationWaitResult]:
"""
若存在人工验证则等待用户完成;无验证或已完成返回 None;超时返回 HumanVerificationWaitResult。
不自动拖动滑块、不自动填写短信验证码。
"""
state = await detect_human_verification(page)
if not state.present:
return None
logger.warning(
"human_verification_detected stage=%s kind=%s selector=%s text=%s",
stage,
state.kind,
state.selector,
state.text_excerpt,
)
print(_wait_log_message(state, wait_sec))
last_state = state
start = time.monotonic()
deadline = start + wait_sec
while time.monotonic() < deadline:
state = await detect_human_verification(page)
if not state.present:
elapsed = round(time.monotonic() - start, 2)
logger.info(
"human_verification_passed stage=%s kind=%s elapsed_sec=%s",
stage,
last_state.kind,
elapsed,
)
await asyncio.sleep(random.uniform(1.0, 2.0))
return None
last_state = state
await asyncio.sleep(2.0)
logger.warning(
"human_verification_timeout stage=%s kind=%s wait_sec=%s selector=%s",
stage,
last_state.kind,
wait_sec,
last_state.selector,
)
return HumanVerificationWaitResult(
timed_out=True,
kind=last_state.kind,
code=_timeout_error_code(last_state.kind),
message=_timeout_error_message(last_state.kind, wait_sec),
stage=stage,
)

View File

@@ -0,0 +1,637 @@
"""真实网站关键词搜索 RPA 示例(async Playwright)。"""
from __future__ import annotations
import logging
import os
import random
import re
import time
from dataclasses import dataclass, field
from typing import Any, Callable, Dict, List, Optional, Tuple
from jiangchang_skill_core import config
from jiangchang_skill_core.activity import interruptible_sleep
from jiangchang_skill_core.rpa import poll_until
from service.browser_session import close_browser_context, get_start_url, start_browser_session
from service.human_verification import (
HumanVerificationWaitResult,
wait_human_verification_if_present,
wait_save_login_countdown_if_present,
)
from util.constants import (
DEFAULT_MAX_ITEMS,
DEFAULT_MAX_SCROLLS,
DEFAULT_NO_NEW_ROUNDS_LIMIT,
HUMAN_WAIT_TIMEOUT,
LOG_LOGGER_NAME,
RESULT_CONTAINER_SELECTOR,
RESULT_END_TEXT,
RESULT_ITEM_SELECTOR,
SCROLL_CONTAINER_SELECTORS,
SEARCH_BUTTON_SELECTOR,
SEARCH_INPUT_SELECTORS,
)
from util.logging import mask_text, safe_log_value
logger = logging.getLogger(LOG_LOGGER_NAME)
@dataclass
class ScrapeError:
code: str
message: str
browser_started: bool = False
stage: Optional[str] = None
@dataclass
class ScrapeRunResult:
success: bool
items: List[Dict[str, Any]] = field(default_factory=list)
stop_reason: Optional[str] = None
error: Optional[ScrapeError] = None
StepCallback = Callable[[str], None]
# 示例登录态 selector — 复制到新 skill 后必须用目标站点 F12/DOM 实测替换,禁止凭经验猜测或跨平台照抄
LOGGED_IN_SELECTORS = (
"a[href*='/user/self']",
"[data-user-avatar]",
".user-avatar",
)
LOGGED_OUT_SELECTOR = "button:has-text('登录')"
def format_stop_reason_for_user(stop_reason: str | None, collected_count: int) -> str:
"""将内部 stop_reason 枚举翻译为用户可理解的中文说明。"""
if stop_reason == "max_items":
return (
f"已达到本次默认采集规模({collected_count} 条)。"
"如需更多,可以继续说「继续采集更多」或「尽量多采集」。"
)
if stop_reason == "end_marker":
return "页面提示没有更多结果,本次采集已完成。"
if stop_reason == "no_new_rounds":
return "连续多次加载没有发现新结果,本次采集已完成。"
if stop_reason == "max_scrolls":
return (
"已达到本次页面加载保护范围,本次采集已完成。"
"如需更深度采集,可以说明「尽量多采集」。"
)
if stop_reason == "human_verification_timeout":
return "人工验证未在规定时间内完成,任务已停止。"
return "本次采集已完成。"
def _mask_account_ref(account: Dict[str, Any]) -> str:
for key in ("login_id", "phone", "mobile"):
raw = str(account.get(key) or "").strip()
if raw:
return mask_text(raw)
aid = account.get("id") or account.get("account_id")
if aid is not None:
return f"id:{aid}"
return "unknown"
def _step_cb(cb: Optional[StepCallback], text: str) -> None:
if cb:
cb(text)
async def _random_delay() -> None:
lo = max(0.0, config.get_float("STEP_DELAY_MIN", 1.0))
hi = max(lo, config.get_float("STEP_DELAY_MAX", 3.0))
await interruptible_sleep(random.uniform(lo, hi))
async def _scroll_wait() -> None:
await interruptible_sleep(random.uniform(1.2, 3.5))
def _headless() -> bool:
v = (os.getenv("OPENCLAW_BROWSER_HEADLESS") or "0").strip().lower()
return v in ("1", "true", "yes", "on")
def _no_new_limit() -> int:
try:
return int(os.getenv("RPA_NO_NEW_ROUNDS_LIMIT") or str(DEFAULT_NO_NEW_ROUNDS_LIMIT))
except ValueError:
return DEFAULT_NO_NEW_ROUNDS_LIMIT
async def _visible(locator) -> bool:
try:
return await locator.count() > 0 and await locator.is_visible(timeout=500)
except Exception:
return False
async def _is_logged_out(page) -> bool:
try:
btn = page.locator(LOGGED_OUT_SELECTOR).first
return await _visible(btn)
except Exception:
return False
async def _is_logged_in(page) -> bool:
if await _is_logged_out(page):
return False
for sel in LOGGED_IN_SELECTORS:
try:
loc = page.locator(sel).first
if await _visible(loc):
return True
except Exception:
continue
return False
async def _open_login_panel_if_needed(page) -> None:
if not await _is_logged_out(page):
return
try:
btn = page.locator(LOGGED_OUT_SELECTOR).first
if await _visible(btn):
await btn.click()
await interruptible_sleep(random.uniform(1.0, 3.0))
except Exception:
pass
async def _ensure_logged_in(page, *, wait_sec: int) -> None:
if await _is_logged_in(page):
return
await _open_login_panel_if_needed(page)
if await _is_logged_in(page):
return
print(f"[登录] 请在浏览器中完成登录,最多等待 {wait_sec} 秒...")
deadline = time.monotonic() + wait_sec
while time.monotonic() < deadline:
if await _is_logged_in(page):
print("[登录] 检测到登录成功")
return
await interruptible_sleep(2.0)
raise RuntimeError(
f"ERROR:LOGIN_TIMEOUT 浏览器已打开,但未完成登录。"
f"请重新运行任务并在浏览器打开后及时完成登录。"
)
async def _find_search_input(page) -> Tuple[Optional[Any], str]:
for sel in SEARCH_INPUT_SELECTORS:
try:
loc = page.locator(sel)
count = await loc.count()
if count == 1 and await _visible(loc):
return loc, sel
if count > 1:
logger.warning("search_input_ambiguous selector=%s count=%s", sel, count)
except Exception:
continue
try:
ph = page.get_by_placeholder(re.compile("搜索"))
count = await ph.count()
if count == 1 and await ph.is_visible():
return ph, "placeholder=搜索"
if count > 1:
logger.warning("search_input_ambiguous selector=placeholder_search count=%s", count)
except Exception:
pass
return None, ""
async def _find_search_button(page):
try:
btn = page.get_by_role("button", name="搜索")
count = await btn.count()
if count == 1 and await btn.is_visible():
return btn
if count > 1:
logger.warning("search_button_ambiguous selector=role_button_search count=%s", count)
except Exception:
pass
try:
btn = page.locator(SEARCH_BUTTON_SELECTOR)
count = await btn.count()
if count == 1 and await btn.is_visible():
return btn
if count > 1:
logger.warning(
"search_button_ambiguous selector=%s count=%s",
SEARCH_BUTTON_SELECTOR,
count,
)
except Exception:
pass
return None
async def _wait_search_results(page, timeout_sec: float = 45.0) -> bool:
"""有界等待搜索结果容器出现(kit poll_until;区分加载中与超时未就绪)。"""
async def _ready() -> bool:
try:
url = page.url or ""
if "/search" in url or "search" in url.lower():
if await page.locator(RESULT_CONTAINER_SELECTOR).count() > 0:
return True
except Exception:
return False
return False
result = await poll_until(_ready, timeout_sec=timeout_sec, interval_sec=0.8)
return result.ok
async def _has_end_marker(page) -> bool:
try:
container = page.locator(RESULT_CONTAINER_SELECTOR).first
if await container.count() > 0:
text = await container.inner_text(timeout=2000)
if RESULT_END_TEXT in text or "暂时没有更多了" in text:
return True
except Exception:
pass
try:
if await page.locator(f"text={RESULT_END_TEXT}").count() > 0:
return True
except Exception:
pass
return False
async def _mouse_wheel_scroll(page) -> Tuple[str, int]:
wheel_delta = random.randint(650, 1200)
for sel in SCROLL_CONTAINER_SELECTORS:
loc = page.locator(sel)
try:
if await loc.count() > 0 and await loc.first.is_visible():
await loc.first.hover()
await _random_delay()
await page.mouse.wheel(0, wheel_delta)
return sel, wheel_delta
except Exception:
continue
await page.mouse.wheel(0, wheel_delta)
return "page", wheel_delta
async def _parse_item_locator(item) -> Dict[str, Any]:
item_id = await item.get_attribute("data-item-id") or await item.get_attribute("id") or ""
title = ""
try:
title = (await item.inner_text(timeout=1000)).strip()[:200]
except Exception:
pass
return {
"item_id": item_id or title[:32] or str(id(item)),
"title": title,
}
async def _collect_visible_items(page, seen_ids: set[str]) -> List[Dict[str, Any]]:
new_items: List[Dict[str, Any]] = []
try:
items = await page.locator(RESULT_ITEM_SELECTOR).all()
except Exception:
items = []
for item in items:
try:
parsed = await _parse_item_locator(item)
item_id = parsed.get("item_id")
if not item_id or item_id in seen_ids:
continue
seen_ids.add(str(item_id))
new_items.append(parsed)
except Exception:
continue
return new_items
async def _count_visible_items(page) -> int:
try:
return await page.locator(RESULT_ITEM_SELECTOR).count()
except Exception:
return 0
def _fail_result(error: ScrapeError, *, collected: int = 0) -> ScrapeRunResult:
logger.warning(
"rpa_failed code=%s stage=%s message=%s collected=%s",
error.code,
error.stage,
error.message,
collected,
)
return ScrapeRunResult(success=False, error=error)
def _fail_on_verification_timeout(
verification_timeout: HumanVerificationWaitResult,
*,
collected: int = 0,
) -> ScrapeRunResult:
return _fail_result(
ScrapeError(
code=verification_timeout.code,
message=verification_timeout.message,
browser_started=True,
stage=verification_timeout.stage or "unknown",
),
collected=collected,
)
async def run_keyword_search_async(
account: Dict[str, Any],
keyword: str,
*,
max_items: int = DEFAULT_MAX_ITEMS,
max_scrolls: int = DEFAULT_MAX_SCROLLS,
data_dir: str = "",
batch_id: str = "",
step_callback: Optional[StepCallback] = None,
) -> ScrapeRunResult:
profile_dir = account.get("profile_dir") or ""
account_ref = _mask_account_ref(account)
wait_sec = int(os.getenv("HUMAN_WAIT_TIMEOUT") or str(HUMAN_WAIT_TIMEOUT))
no_new_limit = _no_new_limit()
logger.info(
"rpa_start keyword=%s max_items=%s max_scrolls=%s wait_sec=%s batch_id=%s account_ref=%s",
safe_log_value(keyword),
max_items,
max_scrolls,
wait_sec,
batch_id,
account_ref,
)
if not profile_dir:
err = ScrapeError(
"ACCOUNT_NOT_FOUND",
"账号缺少 profile_dir。",
browser_started=False,
stage="account_prepare",
)
return _fail_result(err)
pw = None
context = None
page = None
browser_started = False
collected_count = 0
try:
_step_cb(step_callback, "启动关键词搜索采集任务")
try:
pw, context, page = await start_browser_session(profile_dir, headless=_headless())
except RuntimeError as exc:
msg = str(exc)
err = ScrapeError(
"MISSING_BROWSER" if "MISSING_BROWSER" in msg else "UNKNOWN_ERROR",
msg.replace("ERROR:MISSING_BROWSER ", "").replace("ERROR:", ""),
browser_started="未能打开目标首页" in msg,
stage="browser_start" if "未能打开目标首页" in msg else "account_prepare",
)
return _fail_result(err)
browser_started = True
logger.info(
"browser_ready start_url=%s current_url=%s profile_dir_exists=%s",
get_start_url(),
page.url,
os.path.isdir(profile_dir),
)
_step_cb(step_callback, "打开目标站点首页")
await _random_delay()
verification_timeout = await wait_human_verification_if_present(
page, wait_sec=wait_sec, stage="browser_start"
)
if verification_timeout:
return _fail_on_verification_timeout(verification_timeout)
_step_cb(step_callback, "检查账号登录状态")
logger.info("login_check_start current_url=%s", page.url)
try:
await _ensure_logged_in(page, wait_sec=wait_sec)
except RuntimeError as exc:
msg = str(exc)
code = "LOGIN_TIMEOUT" if "LOGIN_TIMEOUT" in msg else "REQUIRE_LOGIN"
err = ScrapeError(
code,
msg.replace("ERROR:LOGIN_TIMEOUT ", "").replace("ERROR:", ""),
browser_started=True,
stage="login_check",
)
return _fail_result(err)
logger.info("login_check_done current_url=%s", page.url)
await wait_save_login_countdown_if_present(page)
verification_timeout = await wait_human_verification_if_present(
page, wait_sec=wait_sec, stage="login_check"
)
if verification_timeout:
return _fail_on_verification_timeout(verification_timeout)
_step_cb(step_callback, "定位搜索框")
search_input, input_selector = await _find_search_input(page)
if search_input is None:
err = ScrapeError(
"SEARCH_INPUT_NOT_FOUND",
"未找到搜索输入框。",
browser_started=True,
stage="search",
)
return _fail_result(err)
logger.info("search_input_found selector=%s", input_selector)
_step_cb(step_callback, f"输入关键词:{keyword}")
await search_input.click()
await _random_delay()
await page.keyboard.press("Control+A")
await page.keyboard.type(keyword, delay=random.randint(90, 240))
await _random_delay()
_step_cb(step_callback, "点击搜索")
search_btn = await _find_search_button(page)
search_method = "button"
if search_btn is not None:
await search_btn.click()
else:
search_method = "enter"
await page.keyboard.press("Enter")
logger.info(
"search_submitted keyword=%s method=%s current_url=%s",
safe_log_value(keyword),
search_method,
page.url,
)
await _random_delay()
verification_timeout = await wait_human_verification_if_present(
page, wait_sec=wait_sec, stage="search_verification"
)
if verification_timeout:
return _fail_on_verification_timeout(verification_timeout)
_step_cb(step_callback, "等待搜索结果")
if not await _wait_search_results(page):
verification_timeout = await wait_human_verification_if_present(
page, wait_sec=wait_sec, stage="search_results"
)
if verification_timeout:
return _fail_on_verification_timeout(verification_timeout)
if not await _wait_search_results(page, timeout_sec=10.0):
err = ScrapeError(
"RESULT_CONTAINER_NOT_FOUND",
"搜索结果未出现,且未检测到需要人工处理的验证码,请检查关键词或页面状态。",
browser_started=True,
stage="search",
)
return _fail_result(err)
container_count = await page.locator(RESULT_CONTAINER_SELECTOR).count()
logger.info(
"search_results_ready current_url=%s container_count=%s",
page.url,
container_count,
)
try:
await page.locator(RESULT_CONTAINER_SELECTOR).first.wait_for(state="visible", timeout=15_000)
except Exception:
err = ScrapeError(
"RESULT_CONTAINER_NOT_FOUND",
"搜索结果容器未出现。",
browser_started=True,
stage="search",
)
return _fail_result(err)
seen_ids: set[str] = set()
collected: List[Dict[str, Any]] = []
no_new_rounds = 0
stop_reason = "max_scrolls"
rounds_executed = 0
for scroll_round in range(max_scrolls + 1):
rounds_executed = scroll_round
_step_cb(step_callback, "读取当前结果列表")
visible_count = await _count_visible_items(page)
batch = await _collect_visible_items(page, seen_ids)
new_count = len(batch)
if batch:
collected.extend(batch)
no_new_rounds = 0
else:
no_new_rounds += 1
collected_count = len(collected)
end_marker = await _has_end_marker(page)
logger.info(
"collect_round round=%s max_scrolls=%s visible_count=%s new_count=%s "
"collected=%s no_new_rounds=%s no_new_limit=%s end_marker=%s current_url=%s",
scroll_round,
max_scrolls,
visible_count,
new_count,
collected_count,
no_new_rounds,
no_new_limit,
end_marker,
page.url,
)
if len(collected) >= max_items:
collected = collected[:max_items]
stop_reason = "max_items"
break
if end_marker:
stop_reason = "end_marker"
_step_cb(step_callback, "发现结果到底或达到采集数量")
break
if no_new_rounds >= no_new_limit:
stop_reason = "no_new_rounds"
logger.info(
"collect_no_new_stop round=%s no_new_rounds=%s no_new_limit=%s collected=%s",
scroll_round,
no_new_rounds,
no_new_limit,
collected_count,
)
break
if scroll_round >= max_scrolls:
break
_step_cb(step_callback, "滚动加载更多结果")
scroll_target, wheel_delta = await _mouse_wheel_scroll(page)
logger.info(
"scroll round=%s target=%s wheel_delta=%s",
scroll_round,
scroll_target,
wheel_delta,
)
await _scroll_wait()
verification_timeout = await wait_human_verification_if_present(
page, wait_sec=wait_sec, stage="collect"
)
if verification_timeout:
return _fail_on_verification_timeout(
verification_timeout,
collected=collected_count,
)
logger.info(
"collect_stop reason=%s collected=%s rounds=%s no_new_rounds=%s",
stop_reason,
collected_count,
rounds_executed,
no_new_rounds,
)
if not collected:
err = ScrapeError(
"NO_RESULTS",
"未采集到任何结果。",
browser_started=True,
stage="collect",
)
return _fail_result(err, collected=0)
_step_cb(step_callback, "采集完成")
logger.info(
"rpa_success collected=%s stop_reason=%s",
len(collected[:max_items]),
stop_reason,
)
return ScrapeRunResult(success=True, items=collected[:max_items], stop_reason=stop_reason)
except Exception as exc:
err = ScrapeError(
"UNKNOWN_ERROR",
str(exc),
browser_started=browser_started,
stage="rpa",
)
return _fail_result(err, collected=collected_count)
finally:
await close_browser_context(pw, context)

View File

@@ -0,0 +1,146 @@
"""轻量任务编排示例(无真实数据库依赖)。"""
from __future__ import annotations
import asyncio
import uuid
from typing import Any, Optional
from service.account_client import (
AccountManagerError,
pick_web_account,
release_lease,
resolve_account_selectors_from_config,
)
from service.task_rpa import (
ScrapeRunResult,
format_stop_reason_for_user,
run_keyword_search_async,
)
from util.constants import (
DEFAULT_MAX_ITEMS,
DEFAULT_MAX_SCROLLS,
SKILL_SLUG,
TARGET_PLATFORM,
)
def _error_payload(
code: str,
message: str,
*,
browser_started: Optional[bool] = None,
stage: Optional[str] = None,
) -> dict[str, Any]:
payload: dict[str, Any] = {
"success": False,
"error": {"code": code, "message": message},
}
if browser_started is not None:
payload["browser_started"] = browser_started
if stage:
payload["stage"] = stage
return payload
def _success_payload(
*,
keyword: str,
collected_count: int,
items: list[dict[str, Any]],
stop_reason: Optional[str],
) -> dict[str, Any]:
return {
"success": True,
"skill": SKILL_SLUG,
"keyword": keyword,
"collected_count": collected_count,
"items": items,
"stop_reason": stop_reason,
"stop_reason_message": format_stop_reason_for_user(stop_reason, collected_count),
}
async def run_keyword_search_task(
keyword: str,
*,
max_items: int = DEFAULT_MAX_ITEMS,
max_scrolls: int = DEFAULT_MAX_SCROLLS,
account_id: Optional[str] = None,
platform: str = TARGET_PLATFORM,
) -> dict[str, Any]:
"""
编排一次关键词搜索采集。
真实 skill 中还应:
- 写 SQLite 入库(items 表)
- 使用 RpaVideoSession 录制步骤留痕
- 失败时 capture_failure 截图
"""
batch_id = uuid.uuid4().hex[:12]
data_dir = "" # 示例占位;真实 skill 使用 get_skill_data_dir()
lease_token: Optional[str] = None
account: Optional[dict[str, Any]] = None
try:
cfg_account_id, cfg_login_id = resolve_account_selectors_from_config()
account = pick_web_account(
platform,
account_id=account_id or cfg_account_id,
login_id=cfg_login_id,
)
lease_token = account.get("lease_token")
scrape_result: ScrapeRunResult = await run_keyword_search_async(
account,
keyword,
max_items=max_items,
max_scrolls=max_scrolls,
data_dir=data_dir,
batch_id=batch_id,
)
if not scrape_result.success:
err = scrape_result.error
return _error_payload(
err.code if err else "UNKNOWN_ERROR",
err.message if err else "未知错误",
browser_started=err.browser_started if err else True,
stage=err.stage if err else "rpa",
)
collected_count = len(scrape_result.items)
return _success_payload(
keyword=keyword,
collected_count=collected_count,
items=scrape_result.items[:10],
stop_reason=scrape_result.stop_reason,
)
except AccountManagerError as exc:
return _error_payload(
exc.code,
exc.message,
browser_started=False,
stage="account_prepare",
)
finally:
release_lease(lease_token)
def run_keyword_search_task_sync(
keyword: str,
*,
max_items: int = DEFAULT_MAX_ITEMS,
max_scrolls: int = DEFAULT_MAX_SCROLLS,
account_id: Optional[str] = None,
platform: str = TARGET_PLATFORM,
) -> dict[str, Any]:
return asyncio.run(
run_keyword_search_task(
keyword,
max_items=max_items,
max_scrolls=max_scrolls,
account_id=account_id,
platform=platform,
)
)

View File

@@ -0,0 +1,31 @@
"""示例常量(复制到新 skill 时请替换为目标站点与业务字段)。"""
SKILL_SLUG = "example-real-browser-rpa"
LOG_LOGGER_NAME = "openclaw.skill.example_real_browser_rpa"
TARGET_PLATFORM = "target_platform"
LEASE_HOLDER = "example-real-browser-rpa"
LEASE_TTL_SEC = "1800"
DEFAULT_START_URL = "https://example.com"
DEFAULT_MAX_ITEMS = 100
DEFAULT_MAX_SCROLLS = 40
DEFAULT_NO_NEW_ROUNDS_LIMIT = 5
HUMAN_WAIT_TIMEOUT = 180
RESULT_END_TEXT = "没有更多结果了"
# 示例 selector — 复制到新 skill 时按目标站点 DOM 替换
SEARCH_INPUT_SELECTORS = (
'input[type="search"]',
'input[name="q"]',
'input[placeholder*="搜索"]',
)
SEARCH_BUTTON_SELECTOR = 'button:has-text("搜索")'
RESULT_CONTAINER_SELECTOR = "#search-result-container"
RESULT_ITEM_SELECTOR = "[data-result-item]"
SCROLL_CONTAINER_SELECTORS = (
"#search-result-container",
".search-result-scroll",
".route-scroll-container",
)

View File

@@ -0,0 +1,34 @@
"""示例结构化日志工具。"""
from __future__ import annotations
import logging
import re
from typing import Any
from util.constants import LOG_LOGGER_NAME
def get_logger(name: str | None = None) -> logging.Logger:
return logging.getLogger(name or LOG_LOGGER_NAME)
def mask_text(value: str, *, keep_start: int = 2, keep_end: int = 2) -> str:
s = (value or "").strip()
if len(s) >= 11 and s.isdigit():
return s[:3] + "****" + s[-4:]
if len(s) > keep_start + keep_end:
return s[:keep_start] + "****" + s[-keep_end:]
return "****"
def safe_log_value(value: Any) -> str:
if value is None:
return ""
text = str(value)
if re.fullmatch(r"\d{7,}", text):
return mask_text(text)
if "@" in text and len(text) > 6:
local, _, domain = text.partition("@")
return f"{mask_text(local, keep_start=1, keep_end=1)}@{domain}"
return text

View File

@@ -0,0 +1,10 @@
"""pytest 路径配置:将 examples/real_browser_rpa/scripts 加入 import 路径。"""
from __future__ import annotations
import sys
from pathlib import Path
SCRIPTS = Path(__file__).resolve().parents[1] / "scripts"
if str(SCRIPTS) not in sys.path:
sys.path.insert(0, str(SCRIPTS))

View File

@@ -0,0 +1,63 @@
from __future__ import annotations
import json
import subprocess
import unittest
from unittest.mock import patch
from service.account_client import AccountManagerError, pick_web_account
def _completed(payload: dict) -> subprocess.CompletedProcess[str]:
return subprocess.CompletedProcess(
args=["python", "account-manager"],
returncode=0,
stdout=json.dumps(payload, ensure_ascii=False),
stderr="",
)
class TestExplicitAccountLease(unittest.TestCase):
@patch("service.account_client._run_argv")
def test_explicit_account_id_is_reselected_with_lease(self, run_argv) -> None:
account = {
"id": 42,
"platform_key": "target-platform",
"account_label": "主账号",
"login_id": "user@example.com",
"profile_dir": "D:/data/profile-42",
"status": "active",
}
leased = {**account, "lease_token": "lease-42"}
run_argv.side_effect = [_completed(account), _completed(leased)]
result = pick_web_account("target-platform", account_id="42")
self.assertEqual(result["id"], 42)
self.assertEqual(result["lease_token"], "lease-42")
self.assertEqual(run_argv.call_args_list[0].args[0], ["account", "get", "42"])
pick_args = run_argv.call_args_list[1].args[0]
self.assertIn("pick-web", pick_args)
self.assertIn("--lease", pick_args)
self.assertIn("user@example.com", pick_args)
@patch("service.account_client._run_argv")
def test_missing_lease_token_is_rejected(self, run_argv) -> None:
account = {
"id": 42,
"platform_key": "target-platform",
"account_label": "主账号",
"login_id": "user@example.com",
"profile_dir": "D:/data/profile-42",
"status": "active",
}
run_argv.side_effect = [_completed(account), _completed(account)]
with self.assertRaises(AccountManagerError) as raised:
pick_web_account("target-platform", account_id="42")
self.assertEqual(raised.exception.code, "LEASE_REQUIRED")
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,170 @@
# -*- coding: utf-8 -*-
"""人工验证检测单元测试(DOM 精准检测,不依赖真实浏览器)。"""
from __future__ import annotations
import asyncio
import time
import unittest
from service.human_verification import (
HumanVerificationWaitResult,
detect_human_verification,
detect_human_verification_from_text,
wait_human_verification_if_present,
wait_save_login_countdown_if_present,
)
SLIDER_HTML = """
<div class="vc-captcha-verify slide">
<div class="captcha_verify_bar--title">请完成下列验证后继续</div>
<div class="captcha-slider-tips">按住左边按钮拖动完成上方拼图</div>
</div>
"""
SMS_HTML = """
<div class="second-verify-panel">
<div class="uc-ui-verify_sms-input">
<div class="uc-ui-verify-new_header-title">短信验证</div>
<input placeholder="手机号" />
<input placeholder="请输入验证码" />
<button>获取验证码</button>
</div>
</div>
"""
SEARCH_RESULTS_HTML = """
<div id="search-result-container">
<div data-result-item="1">RPA 滑块验证码识别教程</div>
<div data-result-item="2">短信验证码自动化</div>
</div>
"""
class MockLocator:
def __init__(self, selector: str, visible: set[str], texts: dict[str, str]) -> None:
self._selector = selector
self._visible = visible
self._texts = texts
@property
def first(self) -> MockLocator:
return self
async def is_visible(self, timeout: int = 500) -> bool:
return self._selector in self._visible
async def inner_text(self, timeout: int = 1000) -> str:
return self._texts.get(self._selector, "")
async def count(self) -> int:
return 1 if self._selector in self._visible else 0
class MockPage:
def __init__(self, visible: set[str], texts: dict[str, str] | None = None) -> None:
self._visible = visible
self._texts = texts or {}
def locator(self, selector: str) -> MockLocator:
return MockLocator(selector, self._visible, self._texts)
class TestHumanVerificationFromText(unittest.TestCase):
def test_slider_dom_detect(self) -> None:
state = detect_human_verification_from_text(SLIDER_HTML)
self.assertTrue(state.present)
self.assertEqual(state.kind, "slider")
self.assertTrue(state.selector)
def test_sms_dom_detect(self) -> None:
state = detect_human_verification_from_text(SMS_HTML)
self.assertTrue(state.present)
self.assertEqual(state.kind, "sms")
self.assertTrue(state.selector)
def test_search_results_not_misclassified(self) -> None:
state = detect_human_verification_from_text(SEARCH_RESULTS_HTML)
self.assertFalse(state.present)
def test_body_keywords_without_dom_not_misclassified(self) -> None:
text = "RPA 滑块验证码识别教程\n短信验证码自动化"
state = detect_human_verification_from_text(text)
self.assertFalse(state.present)
class TestHumanVerificationDomDetect(unittest.TestCase):
def test_slider_dom_visible(self) -> None:
page = MockPage(
{".vc-captcha-verify.slide"},
{".vc-captcha-verify.slide": "请完成下列验证后继续"},
)
state = asyncio.run(detect_human_verification(page))
self.assertTrue(state.present)
self.assertEqual(state.kind, "slider")
self.assertEqual(state.selector, ".vc-captcha-verify.slide")
def test_sms_dom_visible(self) -> None:
page = MockPage(
{".second-verify-panel"},
{".second-verify-panel": "短信验证 获取验证码"},
)
state = asyncio.run(detect_human_verification(page))
self.assertTrue(state.present)
self.assertEqual(state.kind, "sms")
def test_no_verification_when_empty(self) -> None:
page = MockPage(set())
state = asyncio.run(detect_human_verification(page))
self.assertFalse(state.present)
class TestSaveLoginCountdown(unittest.TestCase):
def test_wait_not_present_no_delay(self) -> None:
page = MockPage(set())
start = time.monotonic()
result = asyncio.run(wait_save_login_countdown_if_present(page, timeout_sec=8))
elapsed = time.monotonic() - start
self.assertFalse(result)
self.assertLess(elapsed, 1.0)
class TestHumanVerificationWait(unittest.TestCase):
def test_wait_timeout_slider_dom(self) -> None:
page = MockPage(
{".vc-captcha-verify.slide"},
{".vc-captcha-verify.slide": "请完成下列验证后继续"},
)
result = asyncio.run(
wait_human_verification_if_present(page, wait_sec=1, stage="search_verification")
)
self.assertIsInstance(result, HumanVerificationWaitResult)
self.assertTrue(result.timed_out)
self.assertEqual(result.code, "SLIDER_VERIFICATION_TIMEOUT")
self.assertIn("滑块验证码未在 1 秒内完成", result.message)
self.assertEqual(result.stage, "search_verification")
def test_wait_timeout_sms_dom(self) -> None:
page = MockPage(
{".second-verify-panel"},
{".second-verify-panel": "短信验证 获取验证码"},
)
result = asyncio.run(
wait_human_verification_if_present(page, wait_sec=1, stage="login_check")
)
self.assertIsInstance(result, HumanVerificationWaitResult)
self.assertTrue(result.timed_out)
self.assertEqual(result.code, "SMS_VERIFICATION_TIMEOUT")
self.assertEqual(result.stage, "login_check")
def test_collect_no_verification_immediate_none(self) -> None:
page = MockPage(set())
start = time.monotonic()
err = asyncio.run(wait_human_verification_if_present(page, wait_sec=180, stage="collect"))
elapsed = time.monotonic() - start
self.assertIsNone(err)
self.assertLess(elapsed, 1.0)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,34 @@
# -*- coding: utf-8 -*-
"""stop_reason 用户友好文案单元测试。"""
from __future__ import annotations
import unittest
from service.task_rpa import format_stop_reason_for_user
class TestFormatStopReasonForUser(unittest.TestCase):
def test_max_items(self) -> None:
msg = format_stop_reason_for_user("max_items", 100)
self.assertIn("默认采集规模", msg)
self.assertIn("100", msg)
def test_end_marker(self) -> None:
msg = format_stop_reason_for_user("end_marker", 42)
self.assertIn("没有更多结果", msg)
def test_no_new_rounds(self) -> None:
msg = format_stop_reason_for_user("no_new_rounds", 30)
self.assertIn("连续多次加载没有发现新结果", msg)
def test_max_scrolls(self) -> None:
msg = format_stop_reason_for_user("max_scrolls", 80)
self.assertIn("页面加载保护范围", msg)
def test_none_or_unknown(self) -> None:
self.assertEqual(format_stop_reason_for_user(None, 10), "本次采集已完成。")
self.assertEqual(format_stop_reason_for_user("unknown", 10), "本次采集已完成。")
if __name__ == "__main__":
unittest.main()