16 Commits

Author SHA1 Message Date
4a9d358aef refactor: ffmpeg desktop capture replaces Playwright recording
All checks were successful
Publish Python Package to Gitea / publish (push) Successful in 28s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-02 14:07:00 +08:00
f3fc7b888b feat: add RpaVideoSession for service-layer RPA video output
All checks were successful
Publish Python Package to Gitea / publish (push) Successful in 18s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-02 13:29:49 +08:00
84bb085250 chore: package env example in skill releases
All checks were successful
Publish Python Package to Gitea / publish (push) Successful in 31s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-02 10:33:55 +08:00
8a4a3c8f5d 完善标准
All checks were successful
Publish Python Package to Gitea / publish (push) Successful in 32s
2026-05-31 10:34:42 +08:00
9d4068e4af chore: include skill requirements in release package
All checks were successful
Publish Python Package to Gitea / publish (push) Successful in 21s
2026-05-29 17:36:46 +08:00
abe02f26cf 提交完善
All checks were successful
Publish Python Package to Gitea / publish (push) Successful in 17s
2026-05-28 18:20:49 +08:00
ea25fc2638 修改打包方式
All checks were successful
Publish Python Package to Gitea / publish (push) Successful in 1m2s
2026-05-28 16:49:30 +08:00
43ec2d66a3 feat(screencast): 自动激活窗口 / 多显示器 / 字幕独立时长
基于 SDK 新模块(前两个 commit)的 screencast 工具能力升级:

- runner.py:
  · 新增 activate_window: bool = True 参数,录屏前自动调
    activate_and_maximize_jiangchang_window,避免开头几秒录到 PowerShell/桌面
  · 新增 monitor_index: Optional[int] = None 参数,透传给 ScreenRecorder
  · 自带 SDK 路径兜底 _ensure_sdk_on_path,让 record_screencast.py 直接 python
    跑时(不走 pytest conftest)也能 import jiangchang_desktop_sdk.window_win32

- recorder.py:
  · ScreenRecorder.__init__ 加 monitor_index 参数
  · _capture_loop 用 idx 选 monitor,越界兜底到 0
  · None = 旧行为(monitors[0] 所有屏合并),1 = 主屏,2+ = 第 N 屏
  · 多显示器场景推荐显式传 monitor_index=1 避免录出半边空白

- subtitle.py:
  · SubtitleEngine.process_line 支持二元组(兼容旧脚本)和三元组(每条独立 duration)
  · 长等待场景可以让某条字幕停留更久(如 30s)覆盖中间无日志的"字幕真空"

所有改动向后兼容,全部新增参数有默认值。

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 11:14:11 +08:00
311441dab0 feat(sdk): 新增 e2e_helpers 共享模块,沉淀 desktop E2E 通用能力
把 disburse-payroll-icbc 实战出来的 6 个 desktop E2E 通用 helper 抽到
src/jiangchang_desktop_sdk/e2e_helpers.py,所有 skill 共用:

- drop_file_into_composer: DataTransfer 拖拽附件(绕开 SDK send_file
  在 v2.0.17+ 抛 NotImplementedError 的限制)
- wait_for_attachment_ready: 轮询附件 chip 出现
- wait_for_composer_enabled: 等输入框可见且非 disabled,避免 canSend=false 静默 no-op
- send_prompt_via_composer: fill + press_sequentially + Enter 发送
- wait_for_user_message: 60s 宽容找 user 节点(绕开 SDK ask() user_idx<0
  15s 死循环 bug,对脏 session 鲁棒)
- wait_for_agent_complete: 4 信号合流等 agent 完成
  (chat-page data-sending / window.__jc_sending / 无展开 execution-graph /
  末位 assistant has-body 且非 streaming + 文本稳定)

不抽业务断言(normalize_money / assert_zero_failures),那些是各 skill 自己的事。

所有 helper 只接受 Playwright Page 入口,不依赖 SDK client 实例。

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 11:13:47 +08:00
6e3f79dde9 feat(sdk): 抽出 window_win32 共享模块,client 激活方法薄壳化
把 _activate_and_maximize_windows 内嵌的 50+ 行 ctypes 实现抽到独立的 SDK 子模块
src/jiangchang_desktop_sdk/window_win32.py,暴露公开函数
activate_and_maximize_jiangchang_window,给 screencast 工具也复用(见后续 commit)。

client.py 的 _activate_and_maximize_windows 退化为 6 行薄壳,调用新模块。

设计:
- window_win32 是 SDK 子模块(放 src/jiangchang_desktop_sdk/ 下),不放 tools/,
  避免 SDK 反向依赖工具层
- ctypes EnumWindows + ShowWindow(SW_MAXIMIZE) + SetForegroundWindow 逻辑不变
- 跨平台行为不变:Windows 真执行,其他系统返回 False
- bring_to_front 调用方完全不感知变化(向后兼容)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 11:13:27 +08:00
ffb64b2cfc feat(screencast): 统一字幕样式 / 修字幕重叠 / 修子进程乱码
- composer.py:抽出 _JIANGCHANG_SUBTITLE_STYLE 常量,FontSize 28→14 /
  Outline 2→2(保持),1080p 屏幕约占 1.3% 高度,演示视频留足画面呼吸感
- subtitle.py:generate_srt 加 post-process 排队,相邻字幕互相让位
  (MIN_DURATION=1.0s / GAP=0.1s),任意时刻屏幕上最多 1 条字幕,
  避免开头几秒密集触发的字幕叠成两行
- runner.py:subprocess.Popen 注入 PYTHONIOENCODING=utf-8 + PYTHONUTF8=1,
  让 Windows 子进程 Python 强制 UTF-8 输出,runner 端 UTF-8 解码即对齐,
  修中文乱码

所有 skill 录屏受益,向后兼容(非 Windows 系统 UTF-8 注入无副作用)。

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:14:03 +08:00
c8d136fb79 feat(sdk): bring_to_front 用 Win32 API 激活并最大化主窗口
新增 _activate_and_maximize_windows 私有方法:用 ctypes 调 EnumWindows
找匠厂主窗口(标题含 匠厂/Jiangchang/ClawX/OpenClaw),
ShowWindow(SW_MAXIMIZE) + SetForegroundWindow 让窗口铺满桌面。

bring_to_front 改 OS 分支:
- Windows: 优先 ctypes 激活+最大化(dev / 安装版通吃,比 jiangchang://
  协议在 dev 模式下因 cwd 解析失败而报错可靠);失败时退回协议作为兜底
- macOS / Linux: 维持原协议激活路径(open / xdg-open)

适用场景:录屏 / 自动化测试需要 OS 级窗口前置。page.bring_to_front()
只切 Playwright tab 不影响 z-order,必须 OS 级 API 才能把 Electron
主窗口拉到桌面最前。

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:13:43 +08:00
34f9a2c141 docs: Stage 2 SDK data-jcid migration report
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-18 16:05:16 +08:00
4fc4cfb802 refactor(sdk): drop data-jcid, switch to semantic anchors + ClawX testids
- Introduce centralized SELECTOR constant table at top of client.py
- Replace all [data-jcid=...] selectors with:
  * Semantic anchors (data-role / data-message-id / data-streaming /
    data-sending / data-message-count) guarded by jiangchang main repo
  * ClawX upstream data-testids (chat-page / chat-composer-* / etc.)
- _latest_assistant_after: has_body now judged by inner_text non-empty
  (no longer depends on message-body sub-element)
- _gateway_state: degrade to window.__jc_gateway_state__ readback,
  returns 'unknown' until host exposes it
- new_task: rely solely on data-message-count==0 (welcome-screen no
  longer has a testid)
- wait_gateway_ready: degrade to chat-page selector wait
- send_file: raise NotImplementedError (Electron IPC hook missing)

Public API signatures unchanged. window.__jc_sending__ remains the
core sending signal.

Compatibility: jiangchang v2.0.17+ only.
Refs: jiangchang/docs/sdk-migration-discovery.md
      jiangchang/docs/JIANGCHANG_CUSTOM_ANCHORS.md
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-18 16:05:04 +08:00
089f00c3ff feat(tools): add screencast recording engine
Adds tools/screencast/ — a reusable screen-recording pipeline for
skill demo videos. Wraps a pytest desktop E2E run with:
  - ScreenRecorder: mss-based full-screen frame capture (background thread)
  - SubtitleEngine: stdout keyword → timestamped SRT generation
  - compose_video: FFmpeg filter_complex merge (video + subtitles + BGM)
  - run_screencast: main orchestrator wiring all three layers

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-17 20:38:47 +08:00
20294fd23a 新增依赖 2026-05-06 19:33:46 +08:00
41 changed files with 3236 additions and 132 deletions

View File

@@ -2,8 +2,10 @@ name: Publish Python Package to Gitea
on:
push:
branches:
- main
tags:
- 'v*'
- "v*"
workflow_dispatch:
jobs:
@@ -23,6 +25,13 @@ jobs:
steps:
- uses: https://git.jc2009.com/admin/actions-checkout@v4
- name: Resolve and patch package version
env:
GITHUB_REF: ${{ github.ref }}
GITHUB_RUN_NUMBER: ${{ github.run_number }}
GITHUB_RUN_ID: ${{ github.run_id }}
run: python3.12 tools/ci_set_package_version.py
- name: Install build tools
run: python3.12 -m pip install --upgrade build twine --break-system-packages

View File

@@ -113,6 +113,20 @@ jobs:
"copy_dir('tests', os.path.join(PACKAGE, 'tests'), common_ignore)" \
"copy_dir('evals', os.path.join(PACKAGE, 'evals'), common_ignore)" \
'' \
"req_src = os.path.abspath('requirements.txt')" \
"req_dst = os.path.join(PACKAGE, 'requirements.txt')" \
'if os.path.isfile(req_src):' \
" shutil.copy2(req_src, req_dst)" \
" print('Copied requirements.txt')" \
'' \
"env_example_src = os.path.abspath('.env.example')" \
"env_example_dst = os.path.join(PACKAGE, '.env.example')" \
'if os.path.isfile(env_example_src):' \
" shutil.copy2(env_example_src, env_example_dst)" \
" print('Copied .env.example')" \
'if os.path.isfile(env_example_src) and not os.path.isfile(env_example_dst):' \
" raise RuntimeError('.env.example exists in skill root but was not packaged')" \
'' \
"print('Package top-level entries:')" \
'for name in sorted(os.listdir(PACKAGE)):' \
" print(' -', name)" \

117
README.md
View File

@@ -54,12 +54,117 @@ finally:
client.disconnect()
```
## 5. 发版流程(维护者
### 桌面 E2E`jiangchang_desktop_sdk.e2e_helpers`
```bash
git tag v0.1.0
git push origin v0.1.0
# CI 自动打包并发布到 Gitea Package Registry见 .github/workflows/publish.yml
Skill 的桌面端到端测试应通过宿主 IPC 获取**真实**当前用户数据目录,不要在 pytest 进程里手工设置 `JIANGCHANG_USER_ID` / `CLAW_USER_ID` 来伪造用户(否则会回退到 `_anon`)。
推荐 preflight 流程:
1. `require_logged_in_skill_data_dir(page, skill_slug)` — 经 `jiangchang:resolve-skill-data-dir` 解析路径,并断言已登录(非 `_anon` / `anon` / `anonymous`)。
2. `assert_skill_env_file(ctx, ["API_KEY", ...])` — 检查 `{skill_data}/config/.env` 中必填项已配置且非占位符E2E 不会自动生成真实密钥。
3. `send_prompt_via_composer(page, prompt)` — 普通文本逐字键入;内容中的 `\n` / `\r\n` / `\r`**Shift+Enter** 输入换行;最后用 **Enter** 发送(禁止 DOM 注入、剪贴板、JS 伪造 input/change
诊断时可调用 `get_runtime_env(page)` 查看主进程解析的环境,**不要**用它拼用户数据路径。
```python
from jiangchang_desktop_sdk.e2e_helpers import (
assert_skill_env_file,
require_logged_in_skill_data_dir,
send_prompt_via_composer,
)
ctx = require_logged_in_skill_data_dir(page, "demo-skill")
assert_skill_env_file(ctx, ["API_KEY"])
send_prompt_via_composer(page, "第一行\n第二行")
```
仓库内还保留 `examples/``tools/``python-runtime/` 目录及既有 Skill / 前端相关的 GitHub Actions 工作流,与本 PyPI 包的发布彼此独立。
## 5. 发版流程(维护者)
### main 分支自动预发布
每次推送到 `main` 会触发 `.github/workflows/publish.yml`,自动构建并发布 **唯一** 的 post 版本(基于 `pyproject.toml` 中的基础版本,例如 `0.1.0.post42`)。版本号不会写回仓库,仅在 CI 构建前临时改写 `pyproject.toml``jiangchang_desktop_sdk.__version__`
```bash
git push origin main
```
测试人员升级安装:
```bash
python -m pip install --upgrade \
--index-url https://git.jc2009.com/api/packages/client-jiangchang/pypi/simple/ \
--extra-index-url https://pypi.org/simple \
jiangchang-platform-kit
```
### 正式版本tag
打 tag 并推送后,发布 **去掉 `v` 前缀** 的正式版本号(例如 `v0.1.1` → PyPI 包 `0.1.1`
```bash
git tag v0.1.1
git push origin v0.1.1
```
### 安装后验证
```bash
python -c "from jiangchang_desktop_sdk.e2e_helpers import send_prompt_via_composer; print('e2e_helpers OK')"
python -c "from screencast import run_screencast; print('screencast OK')"
```
仓库内还保留 `examples/``tools/``python-runtime/` 目录及既有 Skill / 前端相关的 GitHub Actions 工作流,与本 PyPI 包的发布彼此独立。`tools/screencast/` 为兼容薄壳pip 包内实现位于 `screencast` 包(`src/screencast/`)。
## v0.2.0 — data-jcid migration (internal refactor)
### TL;DR
本次内部重构:SDK 不再依赖匠厂主仓库的 `data-jcid` 属性。
所有公开 API(`ask` / `read` / `new_task` / `wait_gateway_ready` 等)
**签名与行为保持不变**,只有内部选择器实现切换到新的语义锚点 + ClawX testid。
### 改变了什么
- `client.py` 内所有 `[data-jcid="..."]` 选择器已移除
- 改用匠厂主仓库为 SDK 承诺的语义锚点:`data-role` / `data-message-id` /
`data-streaming`(在 ChatMessage 上)、`data-sending` / `data-message-count`
(在 chat-page 根上)
- `window.__jc_sending__` 仍是核心 sending 信号,保持不变
- 这些锚点由匠厂主仓库的以下机制守护:
- `docs/JIANGCHANG_CUSTOM_ANCHORS.md` 中央清单
- 源码内 `JIANGCHANG CUSTOM ANCHOR` 边界注释
- `harness/specs/jiangchang-custom-anchors.spec.ts` E2E 守护测试
- Gitea CI grep 检查 step
### 已知限制
- `send_file()` 暂未实现,抛 `NotImplementedError`
原因:附件上传走 Electron IPC,SDK 需要主仓库提供新 hook。
- `_gateway_state()` 永远返回 `'unknown'`
原因:gateway 状态不再暴露 DOM data-state;本方法已预留未来通过
`window.__jc_gateway_state__` 读取的实现路径。
这两个限制对常规 ask/read 工作流**无影响**,sending 完成判定依靠
`window.__jc_sending__` + `data-sending` + 文本稳定性合流即可。
### 升级影响
- 在 jiangchang v2.0.17(含)以上版本上跑:应正常工作
- 在 jiangchang v2.0.16(及更早含 data-jcid 的版本)上跑:不再兼容
如果需要在旧版本上跑,固定 SDK 版本到 v0.1.x
### 验证清单(集成方建议跑一次)
打开匠厂客户端 → 用 SDK 跑下面 3 步,确认基础流程通:
```python
from jiangchang_desktop_sdk import JiangchangDesktopClient
with JiangchangDesktopClient() as c:
c.ensure_app_running()
c.wait_gateway_ready()
c.new_task()
answer = c.ask("你好,请回复确认 SDK 可用")
assert answer, f"Empty answer: {answer!r}"
print("OK:", answer[:200])
```

View File

@@ -0,0 +1,131 @@
# Stage 2 — SDK data-jcid Migration Report
## 1. 执行摘要
- 起始 HEAD: `089f00c3ff6b75125d77f40b2e2dcb664a9ac5bf`
- 结束 HEAD: 本阶段未 commit,工作区状态见下
- 影响文件:
- `src/jiangchang_desktop_sdk/client.py`(主要改动,7 步合计)
- `README.md`(新增 v0.2.0 章节)
- `STAGE2_SDK_MIGRATION_REPORT.md`(本回执)
- 其他例外文件:无(`types.py` / `exceptions.py` 未改)
## 2. 7 步改动总览
| 步 | 内容 | 影响范围 |
|----|------|---------|
| 1 | SELECTOR 常量表 | client.py 顶部新增 36 行 |
| 2 | _get_*_locator x4 + 2 处 raise 文案 | client.py 局部 |
| 3 | read() | client.py 局部 |
| 4 | 9 个私有 helper | client.py 多处 |
| 5 | _wait_for_streaming_done | client.py 局部(docstring + 注释) |
| 6 | new_task / wait_gateway_ready / send_file | client.py 局部 |
| 7 | README + 扫尾 | README.md + 验证 |
**第 7 步附带**: `ask()` 兜底分支仍残留可执行 `[data-jcid="message"]` / `data-jcid-role`,已改为 `SEL_MESSAGE_ROW` + `data-role`(否则 7.1 可执行代码 grep 无法通过)。
## 3. 公开 API 兼容性证明
逐项列出公开 API 改造前后签名,确认一字不差:
| 方法 | 改前签名 | 改后签名 | 一致? |
|------|---------|---------|-------|
| ensure_app_running | (self, wait_timeout_s: float = 30.0) -> None | 同左 | ✓ |
| bring_to_front | (self) -> None | 同左 | ✓ |
| connect | (self, url: Optional[str] = None) -> None | 同左 | ✓ |
| launch_app | (self, options: Optional[LaunchOptions] = None) -> None | 同左 | ✓ |
| disconnect | (self) -> None | 同左 | ✓ |
| is_connected | (self) -> bool | 同左 | ✓ |
| get_page | (self) -> Page | 同左 | ✓ |
| new_task | (self) -> None | 同左 | ✓ |
| wait_gateway_ready | (self, timeout_ms: int = 30000) -> None | 同左 | ✓ |
| wait_for_response | (self, timeout: int = 120000) -> None | 同左 | ✓ |
| ask | (self, question: str, options: Optional[AskOptions] = None) -> str | 同左 | ✓ |
| send_file | (self, file_path: str, message: Optional[str] = None) -> None | 同左(行为变 NotImplementedError) | ✓ |
| read | (self) -> List[JiangchangMessage] | 同左 | ✓ |
| assert_contains | (self, expected: str, options: Optional[AssertOptions] = None) -> None | 同左 | ✓ |
| snapshot | (self, target_dir: str, tag: str = "snapshot") -> dict | 同左 | ✓ |
| __enter__ | (self) -> "JiangchangDesktopClient" | 同左 | ✓ |
| __exit__ | (self, exc_type, exc_val, exc_tb) -> None | 同左 | ✓ |
## 4. 临时限制说明
### send_file
- 状态:抛 NotImplementedError
- 原因:Electron IPC stage-paths,SDK 无 DOM 入口
- 恢复条件:匠厂主仓库暴露 `window.__jc_stage_paths__` 或带 testid 的隐藏 input
### _gateway_state
- 状态:永远返回 'unknown'
- 原因:HEAD 上 gateway 状态不暴露 DOM
- 行为影响:_wait_for_streaming_done 中 `if gw not in ("running", "unknown")`
实质 no-op,但保留以便未来生效
- 恢复条件:匠厂在 main / store 暴露 `window.__jc_gateway_state__`
## 5. 扫尾验证实际输出
### 5.1 `grep -n 'data-jcid' src/jiangchang_desktop_sdk/client.py`
```
src/jiangchang_desktop_sdk/client.py:10:3. 每次提问前新建任务(点击侧栏 `data-jcid="new-task-button"`),避免上下文污染。 (模块 docstring 历史描述)
src/jiangchang_desktop_sdk/client.py:14: 这些 DOM 节点 **不再带 `data-jcid="message"`**,而是 `data-testid="chat-execution-graph"` (模块 docstring 历史描述)
src/jiangchang_desktop_sdk/client.py:18: · **锚点**`ask()` 发送后等到最后一条 `[data-jcid-role="user"]` 节点(就是我们刚发的 (模块 docstring 历史描述)
src/jiangchang_desktop_sdk/client.py:19: 问题)在 DOM 里出现,记住它在 `[data-jcid="message"]` 列表里的 index作为切片起点 (模块 docstring 历史描述)
src/jiangchang_desktop_sdk/client.py:21: · `chat-root[data-jcid-sending] === 'false'` (模块 docstring 历史描述)
src/jiangchang_desktop_sdk/client.py:25: · 我们这条 user 之后存在 `[data-jcid-role="assistant"]` 且其内部有 (模块 docstring 历史描述)
src/jiangchang_desktop_sdk/client.py:26: `[data-jcid="message-body"]` (模块 docstring 历史描述)
src/jiangchang_desktop_sdk/client.py:29:5. read() 严格按 `data-jcid-role` 分类,没有该属性的 DOM 节点不再默认归为 assistant。 (模块 docstring 历史描述)
src/jiangchang_desktop_sdk/client.py:526: """返回 `[data-jcid="message"]` 节点列表中最后一条 role=user 的 index找不到返回 -1。""" (docstring 历史描述)
src/jiangchang_desktop_sdk/client.py:561: "streaming": bool, # data-jcid-streaming 属性 (docstring 历史描述)
src/jiangchang_desktop_sdk/client.py:610: """chat-root[data-jcid-sending] 的 bool 化;读不到返回 None。""" (docstring 历史描述)
src/jiangchang_desktop_sdk/client.py:796: "我们刚发出的 user 节点在 [data-jcid=\"message\"] 列表里的 index"。后续所有 (ask docstring 历史描述)
```
可执行代码(非 docstring / 非 `#` 注释):**0 命中**。
### 5.2 `grep -n '__jc_sending__' src/jiangchang_desktop_sdk/client.py`
```
src/jiangchang_desktop_sdk/client.py:22: · `window.__jc_sending__` !== true
src/jiangchang_desktop_sdk/client.py:633: """从 window.__jc_sending__ 读取当前 sending 状态;不可用返回 None。"""
src/jiangchang_desktop_sdk/client.py:636: val = page.evaluate("() => window.__jc_sending__")
src/jiangchang_desktop_sdk/client.py:673: C同时 window.__jc_sending__ 不是 true允许 None / False以容忍
```
### 5.3 import 可用性
```
import OK
```
## 6. 集成验证
- 是否在本地跑过 README 末尾的 3 步验证流程?
- [ ] 跑过 → 结果(pass/fail + 关键输出片段)
- [x] 未跑 → 原因:验收环境未启动匠厂客户端 / 本阶段仅完成代码与静态扫尾
- 是否运行了 platform-kit 自带的任何测试?
- [ ] 跑过 → 结果
- [x] 未跑 → 原因:本包无针对 desktop_sdk 的自动化集成测试;需实机 CDP
## 7. 未触及的事项
- `examples/``tools/screencast/`:未在本阶段验证;预期不直接依赖已移除的 data-jcid 选择器
- platform-kit stash 中的 `runtime_env.py` 改动:本阶段未碰,仍在 stash(`stash@{0}: wip: claw env var fallback...`)
- `D:\AI\jiangchang` 主仓库:未修改
- 模块级 / 部分 helper / `ask()` docstring 中 data-jcid 历史措辞:保留,见 §9
## 8. 自检
- [x] grep 'data-jcid' 在可执行代码中 → 0 命中
- [x] 公开 API 签名表逐项核对一致
- [x] send_file 抛 NotImplementedError 而非静默
- [x] _gateway_state 含 TODO 注释
- [x] README 新增 v0.2.0 章节,无格式错乱
- [x] 未触碰 D:\AI\jiangchang 主仓库任何文件
- [x] 未触碰 platform-kit 非 client.py / README.md / 本回执 的文件
- [ ] git status 显示工作区干净(除本阶段预期修改 + 已有 stash) → 见 §1,未 commit
## 9. 给验收方的备注
1. **docstring 历史 data-jcid**: 模块顶 docstring(L10L29)、`_last_user_node_index`(L526)、`_latest_assistant_after` 内联说明(L561)、`_chat_root_sending`(L610)、`ask()` docstring(L796) 仍描述旧 UI 模型;不影响运行。若需文档一致性,可单独开文档清理 PR。
2. **send_file**: 调用即 `NotImplementedError`,无连接检查(按规格整段替换方法体)。
3. **wait_gateway_ready**: 仅等待 `SEL_CHAT_PAGE` 可见,不保证 Gateway 进程已 ready。
4. **建议验收**: 在已安装 v2.0.17+ 匠厂 + Playwright 环境跑 README 末尾脚本;并确认 `send_file(...)` 按预期抛错。

View File

@@ -12,6 +12,8 @@ license = "MIT"
authors = [{ name = "client-jiangchang" }]
dependencies = [
"requests>=2.31.0",
"playwright>=1.42.0",
"mss>=9.0.1",
]
[project.urls]

68
sanity_check.py Normal file
View File

@@ -0,0 +1,68 @@
"""
Stage 2 集成验证脚本——一次性 sanity check。
跑前准备:
1. 启动匠厂客户端(确认能正常打开聊天页 + 至少配好一个可用模型)
2. 在本仓库激活 Python venv,确认 playwright 已装
3. 直接 python sanity_check.py 即可
"""
import sys
import time
import traceback
print("=" * 60)
print("Stage 2 SDK Sanity Check")
print("=" * 60)
try:
from jiangchang_desktop_sdk import JiangchangDesktopClient
print("[1/6] import OK")
except Exception as e:
print(f"[FAIL] import failed: {e}")
sys.exit(1)
try:
with JiangchangDesktopClient() as c:
print("[2/6] context entered")
c.ensure_app_running()
print("[3/6] ensure_app_running OK")
c.wait_gateway_ready()
print("[4/6] wait_gateway_ready OK (note: returns even if gw not actually ready)")
c.new_task()
print("[5/6] new_task OK")
t0 = time.time()
answer = c.ask("你好,请用一句话回复,确认 SDK 可用")
elapsed = time.time() - t0
print(f"[6/6] ask() returned in {elapsed:.1f}s")
print(f" answer length: {len(answer)} chars")
print(f" answer preview: {answer[:200]!r}")
assert answer.strip(), "Empty answer!"
print()
print("[BONUS] verifying read() works")
msgs = c.read()
print(f" read() returned {len(msgs)} messages")
for i, m in enumerate(msgs[-4:]):
print(f" [-{len(msgs)-i}] role={m.role!r} id={m.id!r} "
f"content_len={len(m.content) if m.content else 0}")
print()
print("[BONUS] verifying send_file raises NotImplementedError")
try:
c.send_file("/tmp/fake.txt")
print(" [FAIL] send_file did NOT raise!")
except NotImplementedError as e:
print(f" [OK] send_file raised: {str(e)[:80]}...")
print()
print("=" * 60)
print("ALL CHECKS PASSED ✓")
print("=" * 60)
except Exception as e:
print(f"\n[FAIL] sanity check crashed:")
traceback.print_exc()
sys.exit(2)

View File

@@ -54,6 +54,42 @@ from .exceptions import (
)
from .types import AskOptions, AssertOptions, JiangchangMessage, LaunchOptions
# ============================================================================
# JIANGCHANG DOM SELECTORS
# ============================================================================
# 本节集中维护所有 Playwright 选择器,作为对匠厂前端的契约。
# 选择器分两类:
#
# (A) 主仓库语义锚点(由 docs/JIANGCHANG_CUSTOM_ANCHORS.md 守护,稳定)
# 这些是匠厂为 SDK 暴露的承诺契约,被 E2E 守护测试 + Gitea CI 双重保护。
#
# (B) 主仓库 ClawX 上游 data-testid(随上游同步可能漂移)
# 这些是 ClawX 上游或我们扩展的测试钩子,本身用于内部 E2E,
# SDK 借用但不保证不变。
#
# 修改前请确认 docs/sdk-migration-discovery.md 中对应元素仍存在。
# ============================================================================
# --- (A) 匠厂语义锚点 ---
SEL_CHAT_PAGE = '[data-testid="chat-page"]'
SEL_CHAT_PAGE_SENDING = '[data-testid="chat-page"][data-sending="true"]'
SEL_MESSAGE_ROW = '[data-role]' # ChatMessage 行容器
SEL_MESSAGE_ROW_USER = '[data-role="user"]'
SEL_MESSAGE_ROW_ASSISTANT = '[data-role="assistant"]'
SEL_MESSAGE_ROW_STREAMING = '[data-streaming="true"]'
# --- (B) ClawX/上游 data-testid ---
SEL_SIDEBAR_NEW_CHAT = '[data-testid="sidebar-new-chat"]'
SEL_CHAT_COMPOSER_INPUT = '[data-testid="chat-composer-input"]'
SEL_CHAT_COMPOSER_SEND = '[data-testid="chat-composer-send"]'
SEL_CHAT_MESSAGE_INDEXED = '[data-testid^="chat-message-"]' # chat-message-${idx}
SEL_EXECUTION_GRAPH = '[data-testid="chat-execution-graph"]'
SEL_EXECUTION_GRAPH_EXPANDED = '[data-testid="chat-execution-graph"][data-collapsed="false"]'
SEL_MAIN_LAYOUT = '[data-testid="main-layout"]'
# --- (C) 通用兜底(无 testid 时使用) ---
SEL_FALLBACK_TEXTAREA = 'textarea'
# ── 调试日志 ───────────────────────────────────────────────────────
_logger = logging.getLogger("jiangchang_desktop_sdk")
_handler_added = False
@@ -182,6 +218,15 @@ class JiangchangDesktopClient:
_logger.warning("[activate] 协议激活失败:%s", exc)
return False
def _activate_and_maximize_windows(self) -> bool:
"""Windows 专属:把匠厂主窗口拉到桌面最前并最大化。
实现已抽到 jiangchang_desktop_sdk.window_win32 模块SDK 子模块),
给 screencast 工具也复用。本方法保留为类内入口(兼容旧调用)。
"""
from .window_win32 import activate_and_maximize_jiangchang_window
return activate_and_maximize_jiangchang_window()
def _launch_via_executable(self, cdp_port: int) -> bool:
"""回退:直接 spawn 可执行文件,带 --remote-debugging-port。"""
exe = self._get_default_executable_path()
@@ -234,12 +279,38 @@ class JiangchangDesktopClient:
)
def bring_to_front(self) -> None:
"""把主窗口聚焦到最前。"""
"""把主窗口聚焦到桌面最前。
分两层:
1) page.bring_to_front() —— Playwright 把 page 切到当前 context 内的最前
tab。**不影响 OS 级窗口 z-order**。
2) OS 级窗口前置(录屏 / 自动化必需):
- Windows: 用 ctypes 调 Win32 EnumWindows 找匠厂主窗口 →
ShowWindow(SW_MAXIMIZE) + SetForegroundWindow让窗口铺满桌面
(详见 _activate_and_maximize_windows。失败时退回协议激活作为
兜底,但 dev 模式pnpm dev / electron .)下协议本身会因 cwd 解析
失败而无效,只有安装版协议兜底才可能成功。
- macOS / Linux: 走 jiangchang:// 协议open / xdg-open
非 Windows 系统下协议处理器没有 cwd 解析问题)。
"""
try:
page = self.get_page()
page.bring_to_front()
except Exception as exc:
_logger.debug("[bring_to_front] 失败:%s", exc)
_logger.debug("[bring_to_front] Playwright 切 page 失败:%s", exc)
if os.name == "nt":
if self._activate_and_maximize_windows():
return
try:
self._activate_via_protocol()
except Exception as exc:
_logger.debug("[bring_to_front] 协议激活失败:%s", exc)
else:
try:
self._activate_via_protocol()
except Exception as exc:
_logger.debug("[bring_to_front] 协议激活失败:%s", exc)
# ─── 连接 / 断开 ──────────────────────────────────────────────
@@ -375,51 +446,46 @@ class JiangchangDesktopClient:
def _get_chat_input_locator(self):
page = self.get_page()
locator = page.locator('[data-jcid="chat-input"]')
locator = page.locator(SEL_CHAT_COMPOSER_INPUT)
if locator.count() > 0:
return locator
locator = page.locator("textarea").last
locator = page.locator(SEL_FALLBACK_TEXTAREA).last
if locator.count() > 0:
return locator
raise ConnectionError("找不到聊天输入框textarea请确认匠厂已加 data-jcid=\"chat-input\"")
raise ConnectionError(
'找不到聊天输入框,请确认匠厂已加 data-testid="chat-composer-input" '
"或页面中存在 textarea"
)
def _get_send_button_locator(self):
page = self.get_page()
locator = page.locator('[data-jcid="send-button"]')
locator = page.locator(SEL_CHAT_COMPOSER_SEND)
if locator.count() > 0:
return locator
locator = page.locator('button[type="submit"]')
if locator.count() > 0:
return locator
raise ConnectionError("找不到发送按钮,请确认匠厂已加 data-jcid=\"send-button\"")
raise ConnectionError(
'找不到发送按钮,请确认匠厂已加 data-testid="chat-composer-send" '
'或 button[type="submit"]'
)
def _get_new_task_button_locator(self):
page = self.get_page()
locator = page.locator('[data-jcid="new-task-button"]')
locator = page.locator(SEL_SIDEBAR_NEW_CHAT)
if locator.count() > 0:
return locator
# 回退:旧版本 data-testid
locator = page.locator('[data-testid="sidebar-new-chat"]')
if locator.count() > 0:
return locator
# 最后兜底:文本"新建任务"
locator = page.get_by_text("新建任务", exact=False).first
return locator
def _get_message_list_locator(self):
page = self.get_page()
locator = page.locator('[data-jcid="message-list"]')
if locator.count() > 0:
return locator
locator = page.locator('[class*="space-y-3"]').first
if locator.count() > 0:
return locator
raise ConnectionError("找不到消息列表容器 data-jcid=\"message-list\"")
return page.locator(SEL_CHAT_PAGE)
# ─── 新建任务 ─────────────────────────────────────────────────
def new_task(self) -> None:
"""点击"新建任务"按钮,等待消息列表清空或欢迎屏出现"""
"""点击"新建任务"按钮,等待消息列表清空chat-page data-message-count=0"""
if not self._connected:
raise ConnectionError("未连接到应用")
page = self.get_page()
@@ -431,16 +497,12 @@ class JiangchangDesktopClient:
_logger.warning("[new_task] 点击失败:%s", exc)
return
# 等待 chat-root.message-count=0 或 welcome-screen 出现,最多 5s
deadline = time.time() + 5
while time.time() < deadline:
try:
count = page.locator('[data-jcid="chat-root"]').first.get_attribute("data-jcid-message-count")
count = page.locator(SEL_CHAT_PAGE).first.get_attribute("data-message-count")
if count == "0":
_logger.debug("[new_task] 已进入新任务message-count=0")
return
if page.locator('[data-jcid="welcome-screen"]').count() > 0:
_logger.debug("[new_task] 欢迎屏已出现")
_logger.debug("[new_task] 已进入新任务(data-message-count=0")
return
except Exception:
pass
@@ -450,25 +512,35 @@ class JiangchangDesktopClient:
# ─── 网关健康检查 ─────────────────────────────────────────────
def _gateway_state(self) -> str:
"""Read gateway state via Electron IPC bridge (jiangchang exposes no DOM data-state).
Strategy: try to read window.__jc_gateway_state__ if exposed;
otherwise return 'unknown' and let upper logic continue.
"""
# TODO: 等匠厂在 src/stores/gateway/* 或 main 进程暴露
# window.__jc_gateway_state__ 之后,本方法将自动开始返回真实状态。
# 当前永远返回 'unknown',行为退化为"假定 gateway 健康"。
try:
page = self.get_page()
el = page.locator('[data-jcid="gateway-status"]').first
if el.count() == 0:
return "unknown"
return el.get_attribute("data-state") or "unknown"
state = self.get_page().evaluate(
"() => window.__jc_gateway_state__"
)
if isinstance(state, str) and state:
return state
except Exception:
return "unknown"
pass
return 'unknown'
def wait_gateway_ready(self, timeout_ms: int = 30000) -> None:
"""等待匠厂前端就绪。
由于 v2.0.17 起 gateway 状态不再暴露 DOM data-state,本方法降级为
等待 chat-page 根元素可见(隐含 React + gateway store 已初始化)。
Gateway 实际就绪与否,由后续 ask() 内的 _gateway_state 检查兜底
(当前永远返回 'unknown')。
"""
page = self.get_page()
try:
page.wait_for_function(
"""() => {
const el = document.querySelector('[data-jcid="gateway-status"]');
return el && el.dataset.state === 'running';
}""",
timeout=timeout_ms,
)
page.wait_for_selector(SEL_CHAT_PAGE, timeout=timeout_ms)
except Exception as exc:
_logger.warning("[wait_gateway_ready] 超时/异常:%s(继续,但后续可能报 GatewayDown", exc)
@@ -481,7 +553,7 @@ class JiangchangDesktopClient:
def _message_node_count(self) -> int:
try:
return self.get_page().locator('[data-jcid="message"]').count()
return self.get_page().locator(SEL_MESSAGE_ROW).count()
except Exception:
return 0
@@ -489,10 +561,10 @@ class JiangchangDesktopClient:
"""返回 `[data-jcid="message"]` 节点列表中最后一条 role=user 的 index找不到返回 -1。"""
try:
page = self.get_page()
total = page.locator('[data-jcid="message"]').count()
total = page.locator(SEL_MESSAGE_ROW).count()
for i in range(total - 1, -1, -1):
node = page.locator('[data-jcid="message"]').nth(i)
role = node.get_attribute("data-jcid-role") or ""
node = page.locator(SEL_MESSAGE_ROW).nth(i)
role = node.get_attribute("data-role") or ""
if role.lower() == "user":
return i
return -1
@@ -507,7 +579,7 @@ class JiangchangDesktopClient:
return False
try:
page = self.get_page()
node = page.locator('[data-jcid="message"]').nth(user_idx)
node = page.locator(SEL_MESSAGE_ROW).nth(user_idx)
text = node.inner_text(timeout=2000) or ""
t = re.sub(r"\s+", "", text)
q = re.sub(r"\s+", "", question)
@@ -529,22 +601,19 @@ class JiangchangDesktopClient:
return None
try:
page = self.get_page()
nodes = page.locator('[data-jcid="message"]')
nodes = page.locator(SEL_MESSAGE_ROW)
total = nodes.count()
for i in range(total - 1, user_idx, -1):
node = nodes.nth(i)
role = (node.get_attribute("data-jcid-role") or "").lower()
role = (node.get_attribute("data-role") or "").lower()
if role != "assistant":
continue
body = node.locator('[data-jcid="message-body"]')
has_body = body.count() > 0
body_text = ""
if has_body:
try:
body_text = body.first.inner_text(timeout=2000) or ""
body_text = node.inner_text(timeout=2000) or ""
except Exception:
body_text = ""
streaming = (node.get_attribute("data-jcid-streaming") or "false").lower() == "true"
has_body = len(body_text.strip()) > 0
streaming = (node.get_attribute("data-streaming") or "false").lower() == "true"
return {
"index": i,
"has_body": has_body,
@@ -575,8 +644,8 @@ class JiangchangDesktopClient:
def _chat_root_sending(self) -> Optional[bool]:
"""chat-root[data-jcid-sending] 的 bool 化;读不到返回 None。"""
try:
val = self.get_page().locator('[data-jcid="chat-root"]').first.get_attribute(
"data-jcid-sending"
val = self.get_page().locator(SEL_CHAT_PAGE).first.get_attribute(
"data-sending"
)
if val is None:
return None
@@ -586,8 +655,8 @@ class JiangchangDesktopClient:
def _chat_root_message_count(self) -> Optional[int]:
try:
val = self.get_page().locator('[data-jcid="chat-root"]').first.get_attribute(
"data-jcid-message-count"
val = self.get_page().locator(SEL_CHAT_PAGE).first.get_attribute(
"data-message-count"
)
if val is None:
return None
@@ -626,16 +695,16 @@ class JiangchangDesktopClient:
"""
等待 agent 整轮(可能包含多次 thinking / tool_use 往返)完成。
核心判据 —— 只有 agent 最终答复才有 `message-body`(文本气泡)
核心判据 —— agent 最终答复在 data-role=assistant 行上有非空 inner_text
源码依据D:\\AI\\jiangchang/src/stores/chat/helpers.ts
- isToolOnlyMessage 显式允许 thinking 与 tool_use 共存仍视为中间轮
- 中间工具轮的 ChatMessage 无 hasText → 不渲染 MessageBubble →
也就不会有 `data-jcid="message-body"`
- 只有最终含 text 的助手消息才会渲染 message-body
- 中间工具轮可能无可见正文气泡 → data-role 行 inner_text 为空
- 只有最终含 text 的助手消息行 inner_text 非空has_body
因此可靠的完成条件为:
A最后一条 assistant message 存在 [data-jcid="message-body"]
B其文本连续 `stable_seconds` 秒无变化
Auser_msg_index 之后最后一条 assistant 的 has_body 为 True
(由 _latest_assistant_after 根据 inner_text 判定)
B其 body_text 连续 `stable_seconds` 秒无变化;
C同时 window.__jc_sending__ 不是 true允许 None / False以容忍
早期 React 未挂载注入钩子的情况)。
@@ -675,10 +744,10 @@ class JiangchangDesktopClient:
assistant = self._latest_assistant_after(user_msg_index)
# 完成判据需同时满足:
# (1) 三路 sending 信号全部静默chat-root / window / 未收起的 execution graph
# (2) user_msg_index 之后存在 assistant 消息
# (3) 该 assistant 存在 message-body
# (4) 该 assistant 的 data-jcid-streaming != 'true'
# (1) 三路 sending 信号全部静默chat-page data-sending / window / 未收起的 execution graph
# (2) user_msg_index 之后存在 assistant 消息data-role 行)
# (3) 该 assistant has_body 为 True_latest_assistant_afterinner_text 非空)
# (4) 该 assistant 的 data-streaming != 'true'
# (5) body 文本连续 stable_seconds 秒无变化
can_check_stable = (
not sending_any
@@ -740,7 +809,7 @@ class JiangchangDesktopClient:
raise JcTimeoutError(
f"等待 AI 回复超时({timeout_ms}ms。user_msg_index={user_msg_index} 最后阶段 phase={phase}"
"可能原因:模型思考时间过长 / 工具链循环未结束 / UI 未挂载 message-body / Gateway 卡住。"
"可能原因:模型思考时间过长 / 工具链循环未结束 / assistant 正文未出现data-role 行 inner_text 为空)/ Gateway 卡住。"
)
# ─── 核心交互ask / read / assert ────────────────────────────
@@ -844,70 +913,56 @@ class JiangchangDesktopClient:
poll_interval=opts.poll_interval,
)
# 6. 选答:只在 my_user_idx 之后的 assistant 中找带 message-body 的最后一条
# 6. 选答:只在 my_user_idx 之后的 assistant 中找 has_body 的最后一条
picked = self._latest_assistant_after(my_user_idx)
if picked is None or not picked["has_body"]:
_logger.warning(
"[ask] user_idx=%d 之后未找到带 message-body 的 assistantpicked=%s",
"[ask] user_idx=%d 之后未找到 has_body 的 assistantpicked=%s",
my_user_idx, picked,
)
# 兜底:取我们 user 之后的任意最后一条 assistant inner_text
try:
page = self.get_page()
nodes = page.locator('[data-jcid="message"]')
nodes = page.locator(SEL_MESSAGE_ROW)
total = nodes.count()
for i in range(total - 1, my_user_idx, -1):
node = nodes.nth(i)
if (node.get_attribute("data-jcid-role") or "").lower() == "assistant":
if (node.get_attribute("data-role") or "").lower() == "assistant":
return node.inner_text(timeout=2000) or ""
except Exception:
pass
return ""
_logger.debug(
"[ask] 选定 assistant_idx=%duser_idx=%d 之后第一条带 message-bodylen=%d",
"[ask] 选定 assistant_idx=%duser_idx=%d 之后 has_bodylen=%d",
picked["index"], my_user_idx, len(picked["body_text"]),
)
return picked["body_text"]
def send_file(self, file_path: str, message: Optional[str] = None) -> None:
if not self._connected:
raise ConnectionError("未连接到应用")
page = self.get_page()
"""[暂未实现] 发送文件附件。
attach_btn = page.locator('[data-jcid="attach-button"]')
if attach_btn.count() == 0:
attach_btn = page.locator("button").filter(
has_text=re.compile(r"附件|上传|paperclip|attach", re.I)
匠厂 v2.0.17(ClawX v0.4.3 合并后)的附件上传通道由原生 Electron
showOpenDialog + IPC stage-paths 实现,不再暴露 HTML <input type="file">。
Playwright 无法直接触发 Electron 原生对话框。需要匠厂主仓库新增:
- window.__jc_stage_paths__(paths: string[]) 调用桥,或
- 一个隐藏 <input type="file"> 备用通道,带 data-testid
在匠厂提供其中之一之前,本方法抛 NotImplementedError 而非静默成功。
Tracking: 见 docs/sdk-migration-discovery.md §7
"""
raise NotImplementedError(
"send_file is temporarily unavailable on jiangchang v2.0.17+. "
"Reason: attachments now go through Electron IPC; SDK requires "
"a new DOM-or-window hook from the host application. "
"See docs/sdk-migration-discovery.md §7."
)
file_input = page.locator('input[type="file"]')
if file_input.count() == 0 and attach_btn.count() > 0:
attach_btn.first.click()
time.sleep(0.5)
file_input = page.locator('input[type="file"]')
if file_input.count() == 0:
raise AssertError(
"找不到文件上传 input[type='file']。请确认匠厂已加 data-jcid=\"attach-button\""
)
abs_path = os.path.abspath(file_path)
if not os.path.exists(abs_path):
raise AppNotFoundError(f"上传文件不存在: {abs_path}")
file_input.set_input_files(abs_path)
time.sleep(1)
if message:
input_locator = self._get_chat_input_locator()
input_locator.click()
input_locator.press_sequentially(message, delay=25)
input_locator.press("Enter")
def read(self) -> List[JiangchangMessage]:
"""
严格读取:仅接受带 data-jcid="message" + data-jcid-role 节点。
严格读取:仅接受带 data-role 语义锚点的 ChatMessage 节点。
无 role 的节点返回 role='unknown'(不再默认为 assistant
"""
if not self._connected:
@@ -917,41 +972,35 @@ class JiangchangDesktopClient:
messages: List[JiangchangMessage] = []
try:
msg_list = self._get_message_list_locator()
msg_list.wait_for(state="visible", timeout=5000)
page.locator(SEL_CHAT_PAGE).wait_for(state="visible", timeout=5000)
except Exception as exc:
_logger.debug("[read] 消息列表不可见:%s", exc)
_logger.debug("[read] 聊天页不可见:%s", exc)
return messages
items = page.locator('[data-jcid="message"]')
items = page.locator(SEL_MESSAGE_ROW)
total = items.count()
_logger.debug("[read] data-jcid=\"message\" 节点数=%d", total)
_logger.debug("[read] data-role 消息行数=%d", total)
for i in range(total):
el = items.nth(i)
try:
role_attr = el.get_attribute("data-jcid-role")
streaming_attr = el.get_attribute("data-jcid-streaming") or "false"
# 优先从 message-body正文气泡取干净文本避免带入 Thinking / ToolCards
body_locator = el.locator('[data-jcid="message-body"]')
if body_locator.count() > 0:
content = body_locator.first.inner_text(timeout=2000)
else:
role_attr = el.get_attribute("data-role")
streaming_attr = el.get_attribute("data-streaming") or "false"
content = el.inner_text(timeout=2000)
role = (role_attr or "unknown").lower()
tool_status = "streaming" if streaming_attr == "true" else None
messages.append(
JiangchangMessage(
id=el.get_attribute("data-jcid-message-id") or f"msg-{i}",
id=el.get_attribute("data-message-id") or f"msg-{i}",
role=role,
content=content,
timestamp=time.time(),
is_error=(el.get_attribute("data-jcid-error") == "true"),
tool_name=el.get_attribute("data-jcid-tool"),
is_error=False,
tool_name=None,
tool_status=tool_status,
)
)
if streaming_attr == "true":
messages[-1].tool_status = "streaming"
except Exception as exc:
_logger.debug("[read] 读取第%d条异常:%s", i, exc)
continue

View File

@@ -0,0 +1,470 @@
# -*- coding: utf-8 -*-
"""
桌面 E2E 通用 helper所有 skill 的 desktop E2E 测试共用。
为什么独立出来:
- disburse-payroll-icbc 实战验证SDK 的 ask() 在 user_idx<0 时有死循环 bug
且 send_file() 是 NotImplementedError。需要绕开它们自写完整测试流程
- 这些 helper拖拽附件、等 user 节点、多信号合流等待 agent 完成)是
**所有 skill 都通用的**,不应该每个 skill 都内嵌一份
设计原则:
- 所有 helper 只接受 Playwright Page 作为入口(不依赖 SDK client 实例)
- 时间常量默认值留出宽容user message 60s / agent complete 1200s
- 失败一律抛 TimeoutError 或 AssertionError由调用方决定怎么截屏/上报
依赖匠厂 DOM 锚点docs/JIANGCHANG_CUSTOM_ANCHORS.md 守护):
- [data-testid="chat-composer-input"]
- [data-testid="chat-page"][data-sending]
- [data-role="user"] / [data-role="assistant"]
- [data-testid="chat-execution-graph"][data-collapsed]
- window.__jc_sending__
"""
from __future__ import annotations
import base64
import re
import time
from dataclasses import dataclass
from pathlib import Path
from typing import Iterator, Optional, Tuple
from playwright.sync_api import Page
__all__ = [
"SkillDataDir",
"resolve_skill_data_dir",
"require_logged_in_skill_data_dir",
"get_runtime_env",
"assert_skill_env_file",
"drop_file_into_composer",
"wait_for_attachment_ready",
"wait_for_composer_enabled",
"send_prompt_via_composer",
"wait_for_user_message",
"wait_for_agent_complete",
]
_NEWLINE = object()
_ANONYMOUS_USER_IDS = frozenset({"", "_anon", "anon", "anonymous"})
_DEFAULT_PLACEHOLDER_VALUES = frozenset({
"",
"xxx",
"sk-xxx",
"your-key",
"your_api_key",
"todo",
"changeme",
"replace-me",
"<your-key>",
})
@dataclass(frozen=True)
class SkillDataDir:
"""匠厂宿主解析出的 skill 用户数据目录上下文。"""
root: Path
path: Path
user_id: str
skill_slug: str
def _normalize_ws(text: str) -> str:
return re.sub(r"\s+", "", text)
def _invoke_ipc(page: Page, channel: str, payload: dict | None = None) -> dict:
"""通过匠厂 preload 暴露的 IPC 调用主进程 handler。"""
result = page.evaluate(
"""
async ({ channel, payload }) => {
const electron = window.electron;
if (!electron || !electron.ipcRenderer
|| typeof electron.ipcRenderer.invoke !== 'function') {
throw new Error(
'window.electron.ipcRenderer.invoke 不可用:请在匠厂桌面客户端页面内运行 E2E'
+ '且 preload 已暴露 jiangchang IPC'
);
}
try {
if (payload === null || payload === undefined) {
return await electron.ipcRenderer.invoke(channel);
}
return await electron.ipcRenderer.invoke(channel, payload);
} catch (err) {
const msg = err && err.message ? err.message : String(err);
throw new Error(channel + ' IPC 调用失败: ' + msg);
}
}
""",
{"channel": channel, "payload": payload},
)
if not isinstance(result, dict):
raise RuntimeError(
f"{channel} 返回非对象: {result!r}"
)
return result
def resolve_skill_data_dir(
page: Page,
skill_slug: str,
*,
create: bool = True,
) -> SkillDataDir:
"""通过宿主 IPC 解析当前登录用户下的 skill 数据目录(不拼路径、不用 _anon 默认值)。"""
raw = _invoke_ipc(
page,
"jiangchang:resolve-skill-data-dir",
{"skillSlug": skill_slug},
)
missing = [k for k in ("root", "path", "userId", "skillSlug") if not raw.get(k)]
if missing:
raise RuntimeError(
"jiangchang:resolve-skill-data-dir 返回结构不完整,"
f"缺少字段 {missing},实际: {raw!r}"
)
data_path = Path(str(raw["path"]))
if create:
data_path.mkdir(parents=True, exist_ok=True)
return SkillDataDir(
root=Path(str(raw["root"])),
path=data_path,
user_id=str(raw["userId"]),
skill_slug=str(raw["skillSlug"]),
)
def require_logged_in_skill_data_dir(page: Page, skill_slug: str) -> SkillDataDir:
"""解析 skill 数据目录,并断言当前宿主已同步真实登录用户(非匿名)。"""
ctx = resolve_skill_data_dir(page, skill_slug)
uid = (ctx.user_id or "").strip().lower()
if uid in _ANONYMOUS_USER_IDS:
raise AssertionError(
"当前宿主没有同步登录用户 ID收到 "
f"{ctx.user_id!r}),不能运行桌面 E2E。"
"请先在匠厂客户端登录,或检查 jiangchangUserId 是否已同步到设置。"
)
return ctx
def get_runtime_env(page: Page) -> dict:
"""诊断用:读取宿主主进程解析的运行时环境(勿用于拼路径)。"""
return _invoke_ipc(page, "jiangchang:get-runtime-env", None)
def _strip_env_value(raw: str) -> str:
value = raw.strip()
if len(value) >= 2 and value[0] == value[-1] and value[0] in ('"', "'"):
return value[1:-1]
return value
def _parse_env_file(env_path: Path) -> dict[str, str]:
"""解析简单 KEY=value .env无 shell 展开)。"""
parsed: dict[str, str] = {}
text = env_path.read_text(encoding="utf-8")
for line in text.splitlines():
stripped = line.strip()
if not stripped or stripped.startswith("#"):
continue
if "=" not in stripped:
continue
key, _, raw_value = stripped.partition("=")
key = key.strip()
if not key:
continue
parsed[key] = _strip_env_value(raw_value)
return parsed
def _is_placeholder_value(value: str, extra: set[str] | None) -> bool:
normalized = value.strip().lower()
placeholders = _DEFAULT_PLACEHOLDER_VALUES
if extra:
placeholders = placeholders | {p.strip().lower() for p in extra}
return normalized in placeholders
def assert_skill_env_file(
skill_data_dir: SkillDataDir,
required_keys: list[str] | tuple[str, ...],
*,
env_rel_path: str = "config/.env",
placeholder_values: set[str] | None = None,
) -> Path:
"""Preflight检查 skill 用户数据目录下 config/.env 是否已配置所需密钥。"""
env_path = skill_data_dir.path / env_rel_path
if not env_path.is_file():
raise AssertionError(
f"缺少配置文件: {env_path}\n"
"E2E 测试不会自动生成真实密钥,请先一次性配置用户数据目录 config/.env"
)
env_map = _parse_env_file(env_path)
invalid: list[str] = []
for key in required_keys:
if key not in env_map:
invalid.append(key)
continue
value = env_map[key]
if not value.strip() or _is_placeholder_value(value, placeholder_values):
invalid.append(key)
if invalid:
raise AssertionError(
f"config/.env 配置不完整或仍为占位符: {env_path}\n"
f"缺失/无效的 key: {', '.join(invalid)}\n"
"E2E 测试不会自动生成真实密钥,请先一次性配置用户数据目录 config/.env"
)
return env_path
def _iter_text_and_newlines(prompt: str) -> Iterator[str | object]:
"""按真实输入顺序产出文本片段与换行标记(\\r\\n / \\n / \\r 均视为单次换行)。"""
i = 0
n = len(prompt)
text_start = 0
while i < n:
if i + 1 < n and prompt[i] == "\r" and prompt[i + 1] == "\n":
if i > text_start:
yield prompt[text_start:i]
yield _NEWLINE
i += 2
text_start = i
elif prompt[i] in "\n\r":
if i > text_start:
yield prompt[text_start:i]
yield _NEWLINE
i += 1
text_start = i
else:
i += 1
if text_start < n:
yield prompt[text_start:]
# ─── 附件上传(绕开 SDK send_file 未实现) ──────────────────────────
def drop_file_into_composer(page: Page, file_path: Path) -> None:
"""通过 DragEvent + DataTransfer 把文件拖入聊天输入框。
匠厂 ChatInput 的 onDrop 挂在外层 div 上,但 React 17+ 事件代理到 root
所以在 textarea 上派发 bubbles:true 的 drop 事件能正常触发 stageBufferFiles
→ /api/files/stage-buffer → 真实附件链路。
"""
payload_b64 = base64.b64encode(file_path.read_bytes()).decode("ascii")
page.evaluate(
"""
({ base64, fileName, mimeType }) => {
const binary = atob(base64);
const bytes = new Uint8Array(binary.length);
for (let i = 0; i < binary.length; i++) {
bytes[i] = binary.charCodeAt(i);
}
const file = new File([bytes], fileName, { type: mimeType });
const dt = new DataTransfer();
dt.items.add(file);
const textarea = document.querySelector('[data-testid="chat-composer-input"]');
if (!textarea) {
throw new Error('chat-composer-input not found');
}
const opts = { bubbles: true, cancelable: true, dataTransfer: dt };
textarea.dispatchEvent(new DragEvent('dragenter', opts));
textarea.dispatchEvent(new DragEvent('dragover', opts));
textarea.dispatchEvent(new DragEvent('drop', opts));
}
""",
{
"base64": payload_b64,
"fileName": file_path.name,
"mimeType": _guess_mime(file_path),
},
)
def _guess_mime(file_path: Path) -> str:
suffix = file_path.suffix.lower()
return {
".xlsx": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
".xls": "application/vnd.ms-excel",
".csv": "text/csv",
".pdf": "application/pdf",
".png": "image/png",
".jpg": "image/jpeg",
".jpeg": "image/jpeg",
".txt": "text/plain",
".json": "application/json",
}.get(suffix, "application/octet-stream")
def wait_for_attachment_ready(page: Page, file_name: str, timeout_s: float = 30.0) -> None:
"""stage-buffer 成功后附件 chip 会展示 fileName轮询直到可见。"""
deadline = time.time() + timeout_s
name_loc = page.get_by_text(file_name, exact=False)
while time.time() < deadline:
try:
n = name_loc.count()
for i in range(n):
if name_loc.nth(i).is_visible():
return
except Exception:
pass
time.sleep(0.5)
raise TimeoutError(
f"附件 {file_name!r}{timeout_s}s 内未变为 ready"
)
# ─── 发送 prompt绕开 SDK ask 的 15s 锁死) ────────────────────────
def wait_for_composer_enabled(page: Page, timeout_s: float = 15.0) -> None:
"""等输入框可见且非 disabled避免 canSend=false 静默 no-op"""
page.wait_for_selector(
'[data-testid="chat-composer-input"]',
state="visible",
timeout=int(timeout_s * 1000),
)
textarea = page.locator('[data-testid="chat-composer-input"]')
deadline = time.time() + timeout_s
while time.time() < deadline:
if not textarea.evaluate("(el) => el.disabled"):
return
time.sleep(0.3)
raise TimeoutError(
f"chat-composer-input 在 {timeout_s}s 内仍为 disabledgateway 可能未就绪)"
)
def send_prompt_via_composer(page: Page, prompt: str, *, delay_ms: int = 25) -> None:
"""拟人逐字输入 + Shift+Enter 换行 + Enter 发送(禁止 DOM/剪贴板/伪造事件)。"""
textarea = page.locator('[data-testid="chat-composer-input"]')
textarea.click()
textarea.press("Control+A")
textarea.press("Backspace")
for segment in _iter_text_and_newlines(prompt):
if segment is _NEWLINE:
textarea.press("Shift+Enter")
else:
textarea.press_sequentially(str(segment), delay=delay_ms)
textarea.press("Enter")
def wait_for_user_message(page: Page, prompt: str, timeout_s: float = 60.0) -> int:
"""等我们这条 user 消息进入 DOM返回它在 [data-role] 列表里的 index。
绕开 SDK ask() 只给 15s 窗口的脆弱判据:用 prompt 文本反向匹配,
宽容到 60s。脏 sessionnew_task 后 gateway 载回历史)也能正确定位。
"""
prompt_norm = _normalize_ws(prompt)
deadline = time.time() + timeout_s
items = page.locator('[data-role]')
while time.time() < deadline:
total = items.count()
for i in range(total - 1, -1, -1):
node = items.nth(i)
role = (node.get_attribute("data-role") or "").lower()
if role != "user":
continue
text = node.inner_text(timeout=2000) or ""
text_norm = _normalize_ws(text)
if prompt_norm in text_norm or text_norm in prompt_norm:
return i
time.sleep(0.5)
raise TimeoutError(
f"{timeout_s}s 内未找到匹配 user 消息prompt 归一化前 80 字="
f"{prompt_norm[:80]!r})"
)
# ─── 等 agent 完成(多信号合流) ────────────────────────────────
def _chat_sending_false(page: Page) -> bool:
val = page.locator('[data-testid="chat-page"]').first.get_attribute("data-sending")
return val is not None and val.lower() == "false"
def _window_sending_not_true(page: Page) -> bool:
val = page.evaluate("() => window.__jc_sending__")
return val is not True
def _no_expanded_execution_graph(page: Page) -> bool:
return (
page.locator('[data-testid="chat-execution-graph"][data-collapsed="false"]').count()
== 0
)
def _latest_assistant_after(page: Page, user_idx: int) -> Optional[Tuple[int, str, bool]]:
items = page.locator('[data-role]')
total = items.count()
for i in range(total - 1, user_idx, -1):
node = items.nth(i)
role = (node.get_attribute("data-role") or "").lower()
if role != "assistant":
continue
body = node.inner_text(timeout=2000) or ""
streaming = (node.get_attribute("data-streaming") or "false").lower() == "true"
return i, body, streaming
return None
def wait_for_agent_complete(
page: Page,
user_idx: int,
*,
overall_timeout_s: float = 1200.0,
stable_seconds: float = 5.0,
poll_interval: float = 1.0,
) -> str:
"""多信号合流等本轮 agent 完成,返回最末 assistant 气泡文本。
完成条件(全部满足并稳定 stable_seconds 秒):
1) chat-page data-sending="false"
2) window.__jc_sending__ 不为 True
3) 不存在未折叠的 execution graph
4) user_idx 之后存在 assistant 节点,有非空 bodydata-streaming != "true"
5) 上面 assistant 节点的文本长度连续 stable_seconds 秒无变化
"""
deadline = time.time() + overall_timeout_s
last_body_len = -1
last_change_at = time.time()
while time.time() < deadline:
chat_ok = _chat_sending_false(page)
win_ok = _window_sending_not_true(page)
graph_ok = _no_expanded_execution_graph(page)
picked = _latest_assistant_after(page, user_idx)
body_text = ""
streaming = True
has_body = False
if picked is not None:
_, body_text, streaming = picked
has_body = len(body_text.strip()) > 0
all_ready = chat_ok and win_ok and graph_ok and has_body and not streaming
if not all_ready:
last_body_len = -1
last_change_at = time.time()
else:
cur_len = len(body_text)
if cur_len != last_body_len:
last_body_len = cur_len
last_change_at = time.time()
elif time.time() - last_change_at >= stable_seconds:
return body_text
time.sleep(poll_interval)
raise TimeoutError(
f"agent 在 {overall_timeout_s}s 内未完成user_idx={user_idx} "
f"last_body_len={last_body_len}"
)

View File

@@ -0,0 +1,85 @@
# -*- coding: utf-8 -*-
"""
Win32 窗口工具:跨 SDK / screencast 共享的"激活并最大化匠厂主窗口"实现。
为什么单独抽出来:
- SDK 的 bring_to_front 需要 OS 级窗口前置page.bring_to_front 只切 tab
- screencast 的 run_screencast 录屏前也要把匠厂浮到前面并铺满屏幕
- 两者用同一份 EnumWindows + ShowWindow(SW_MAXIMIZE) + SetForegroundWindow 实现,
避免代码重复维护
跨平台行为:
- Windows: 真正执行
- 其他系统: 直接返回 False调用方按需 fallback 到协议激活)
"""
from __future__ import annotations
import logging
import os
from typing import Optional, Tuple
_logger = logging.getLogger("jiangchang_desktop_sdk.window_win32")
_DEFAULT_KEYWORDS: Tuple[str, ...] = ("匠厂", "Jiangchang", "ClawX", "OpenClaw")
SW_MAXIMIZE = 3
def activate_and_maximize_jiangchang_window(
keywords: Optional[Tuple[str, ...]] = None,
) -> bool:
"""Windows: EnumWindows 找匠厂主窗口 → ShowWindow(SW_MAXIMIZE) + SetForegroundWindow。
Args:
keywords: 标题模糊匹配的关键词;不传则用默认
("匠厂", "Jiangchang", "ClawX", "OpenClaw")
Returns:
True: 找到窗口并执行了 ShowWindow + SetForegroundWindow
False: 非 Windows / ctypes 不可用 / 找不到匹配窗口 / 任何异常
"""
if os.name != "nt":
return False
try:
import ctypes
from ctypes import wintypes
except Exception as exc:
_logger.debug("ctypes 不可用:%s", exc)
return False
user32 = ctypes.windll.user32
kw = keywords or _DEFAULT_KEYWORDS
matching: list = []
@ctypes.WINFUNCTYPE(ctypes.c_bool, wintypes.HWND, wintypes.LPARAM)
def _enum_proc(hwnd, _lparam):
if not user32.IsWindowVisible(hwnd):
return True
length = user32.GetWindowTextLengthW(hwnd)
if length == 0:
return True
buf = ctypes.create_unicode_buffer(length + 1)
user32.GetWindowTextW(hwnd, buf, length + 1)
if any(k in buf.value for k in kw):
matching.append((hwnd, buf.value))
return True
try:
user32.EnumWindows(_enum_proc, 0)
except Exception as exc:
_logger.debug("EnumWindows 失败:%s", exc)
return False
if not matching:
_logger.debug("未找到匠厂主窗口keywords=%s", kw)
return False
hwnd, title = matching[0]
try:
user32.ShowWindow(hwnd, SW_MAXIMIZE)
user32.SetForegroundWindow(hwnd)
_logger.debug("已激活+最大化 hwnd=%s title=%s", hwnd, title)
return True
except Exception as exc:
_logger.debug("ShowWindow/SetForegroundWindow 失败:%s", exc)
return False

View File

@@ -1,9 +1,11 @@
from .client import EntitlementClient
from .config import ensure_env_file, get, get_bool, get_float, get_int
from .errors import EntitlementDeniedError, EntitlementError, EntitlementServiceError
from .guard import enforce_entitlement
from .models import EntitlementResult
from .runtime_env import (
apply_cli_local_defaults,
find_chrome_executable,
get_data_root,
get_sibling_skills_root,
get_skills_root,
@@ -20,6 +22,11 @@ from .unified_logging import (
subprocess_env_with_trace,
)
try:
from . import rpa
except ImportError:
rpa = None # type: ignore[assignment,misc]
__all__ = [
"EntitlementClient",
"EntitlementDeniedError",
@@ -28,9 +35,15 @@ __all__ = [
"EntitlementServiceError",
"apply_cli_local_defaults",
"attach_unified_file_handler",
"ensure_env_file",
"enforce_entitlement",
"ensure_trace_for_process",
"find_chrome_executable",
"get",
"get_bool",
"get_data_root",
"get_float",
"get_int",
"get_sibling_skills_root",
"get_skills_root",
"get_skill_log_file_path",
@@ -38,6 +51,7 @@ __all__ = [
"get_unified_logs_dir",
"get_user_id",
"platform_default_data_root",
"rpa",
"setup_skill_logging",
"subprocess_env_with_trace",
]

View File

@@ -0,0 +1,125 @@
"""三级优先级配置读取 + 首次 .env 落盘(进程 env > 数据目录 .env > .env.example"""
from __future__ import annotations
import os
import shutil
from typing import Any
from .runtime_env import get_data_root, get_user_id
_skill_slug: str | None = None
_example_path: str | None = None
_env_file_path: str | None = None
_user_env_cache: dict[str, str] | None = None
_example_defaults_cache: dict[str, str] | None = None
def _parse_env_file(path: str) -> dict[str, str]:
"""标准库手写 .env 解析(不引 python-dotenv"""
result: dict[str, str] = {}
if not path or not os.path.isfile(path):
return result
with open(path, encoding="utf-8") as f:
for raw_line in f:
line = raw_line.strip()
if not line or line.startswith("#"):
continue
if "=" not in line:
continue
key, _, value = line.partition("=")
key = key.strip()
value = value.strip()
if not key:
continue
if len(value) >= 2 and value[0] == value[-1] and value[0] in ('"', "'"):
value = value[1:-1]
elif " #" in value:
value = value.split(" #", 1)[0].strip()
result[key] = value
return result
def _get_user_env() -> dict[str, str]:
global _user_env_cache
if _user_env_cache is not None:
return _user_env_cache
if _env_file_path and os.path.isfile(_env_file_path):
_user_env_cache = _parse_env_file(_env_file_path)
else:
_user_env_cache = {}
return _user_env_cache
def _get_example_defaults() -> dict[str, str]:
global _example_defaults_cache
if _example_defaults_cache is not None:
return _example_defaults_cache
if _example_path and os.path.isfile(_example_path):
_example_defaults_cache = _parse_env_file(_example_path)
else:
_example_defaults_cache = {}
return _example_defaults_cache
def reset_cache() -> None:
"""测试用:清空解析缓存。"""
global _user_env_cache, _example_defaults_cache
_user_env_cache = None
_example_defaults_cache = None
def ensure_env_file(skill_slug: str, example_path: str) -> str:
"""首次把 .env.example copy 到 {data_root}/{user}/{slug}/.env已存在不覆盖返回路径。"""
global _skill_slug, _example_path, _env_file_path
_skill_slug = skill_slug
_example_path = os.path.abspath(example_path)
dest_dir = os.path.join(get_data_root(), get_user_id(), skill_slug)
os.makedirs(dest_dir, exist_ok=True)
dest = os.path.join(dest_dir, ".env")
_env_file_path = dest
if not os.path.isfile(dest) and os.path.isfile(_example_path):
shutil.copy2(_example_path, dest)
reset_cache()
return dest
def get(key: str, default: Any = None) -> str | None:
"""进程环境变量 > 数据目录 .env > .env.example 默认值。"""
env_val = os.environ.get(key)
if env_val is not None and env_val != "":
return env_val
user_val = _get_user_env().get(key)
if user_val is not None and user_val != "":
return user_val
example_val = _get_example_defaults().get(key)
if example_val is not None and example_val != "":
return example_val
return default
def get_bool(key: str, default: bool = False) -> bool:
val = get(key)
if val is None:
return default
return str(val).strip().lower() in ("1", "true", "yes", "on")
def get_float(key: str, default: float) -> float:
val = get(key)
if val is None:
return default
try:
return float(val)
except (TypeError, ValueError):
return default
def get_int(key: str, default: int) -> int:
val = get(key)
if val is None:
return default
try:
return int(val)
except (TypeError, ValueError):
return default

View File

@@ -0,0 +1,65 @@
"""浏览器 RPA 共享库stealth、拟人操作、启动封装、HITL 等待、失败存证。"""
from __future__ import annotations
from . import anti_detect, artifacts, errors, human_login, stealth
try:
from .browser import launch_persistent_browser
except ImportError:
launch_persistent_browser = None # type: ignore[misc, assignment]
try:
from .video_session import RpaVideoSession
except ImportError:
RpaVideoSession = None # type: ignore[misc, assignment]
__all__ = [
"anti_detect",
"artifacts",
"errors",
"human_login",
"stealth",
"launch_persistent_browser",
"RpaVideoSession",
# re-export common anti_detect helpers
"random_delay",
"human_delay_short",
"human_delay_page",
"human_delay_batch",
"human_mouse_move",
"human_mouse_wiggle",
"human_click",
"human_type",
"human_scroll",
# human_login
"wait_for_captcha_pass",
"wait_for_login",
"DEFAULT_CAPTCHA_MARKERS",
# artifacts
"artifact_path",
"capture_failure",
# stealth
"STEALTH_INIT_SCRIPT",
"stealth_enabled",
"persistent_context_launch_parts",
]
from .anti_detect import (
human_click,
human_delay_batch,
human_delay_page,
human_delay_short,
human_mouse_move,
human_mouse_wiggle,
human_scroll,
human_type,
random_delay,
)
from .artifacts import artifact_path, capture_failure
from .human_login import DEFAULT_CAPTCHA_MARKERS, wait_for_captcha_pass, wait_for_login
from .stealth import (
STEALTH_INIT_SCRIPT,
persistent_context_launch_parts,
stealth_enabled,
)

View File

@@ -0,0 +1,215 @@
"""防反爬:随机延迟、人工鼠标轨迹模拟、拟人滚动。
所有延迟使用 random.uniform() + asyncio.sleep()。
鼠标移动使用三次贝塞尔曲线 + 随机抖动,避免机械直线轨迹。
"""
from __future__ import annotations
import asyncio
import math
import random
from typing import Optional
# ── 随机延迟 ────────────────────────────────────────────────
async def random_delay(min_s: float, max_s: float) -> float:
"""随机延迟 min_s ~ max_s 秒,返回实际延迟秒数。"""
delay = random.uniform(min_s, max_s)
await asyncio.sleep(delay)
return delay
async def human_delay_short() -> float:
"""页面内操作间短延迟 0.5~2 秒(点击/滚动)。"""
return await random_delay(0.5, 2.0)
async def human_delay_page(min_s: float = 2, max_s: float = 8) -> float:
"""页面间/店铺间切换延迟(可配置范围)。"""
return await random_delay(min_s, max_s)
async def human_delay_batch(min_s: float = 5, max_s: float = 15) -> float:
"""批次间/翻页间长延迟。"""
return await random_delay(min_s, max_s)
# ── 鼠标轨迹 ────────────────────────────────────────────────
async def human_mouse_move(
page,
target_x: float,
target_y: float,
steps: Optional[int] = None,
) -> None:
"""模拟人工鼠标移动到目标坐标。
使用三次贝塞尔曲线生成 S/C 形路径,中间控制点随机偏移,
每步添加微小随机抖动,避免完美直线移动。
Args:
page: Playwright Page 对象。
target_x: 目标 X 坐标。
target_y: 目标 Y 坐标。
steps: 移动步数None 时按距离自动计算)。
"""
if steps is None:
dist = math.hypot(target_x, target_y)
steps = max(8, int(dist / 15))
# 随机起始点(模拟鼠标从页面其他位置移动过来)
start_x = target_x + random.uniform(-300, 300)
start_y = target_y + random.uniform(-200, 200)
# 两个贝塞尔控制点(制造弯曲路径)
dx = target_x - start_x
dy = target_y - start_y
cp1_x = start_x + dx * random.uniform(0.2, 0.5) + random.uniform(-120, 120)
cp1_y = start_y + dy * random.uniform(0.1, 0.4) + random.uniform(-100, 100)
cp2_x = start_x + dx * random.uniform(0.5, 0.8) + random.uniform(-100, 100)
cp2_y = start_y + dy * random.uniform(0.4, 0.7) + random.uniform(-80, 80)
for i in range(steps + 1):
t = i / steps
# 三次贝塞尔: B(t) = (1-t)³·P0 + 3(1-t)²·t·P1 + 3(1-t)·t²·P2 + t³·P3
u = 1 - t
x = (
(u ** 3) * start_x
+ 3 * (u ** 2) * t * cp1_x
+ 3 * u * (t ** 2) * cp2_x
+ (t ** 3) * target_x
)
y = (
(u ** 3) * start_y
+ 3 * (u ** 2) * t * cp1_y
+ 3 * u * (t ** 2) * cp2_y
+ (t ** 3) * target_y
)
# 微小随机抖动
x += random.uniform(-1.5, 1.5)
y += random.uniform(-1.5, 1.5)
await page.mouse.move(x, y)
await asyncio.sleep(random.uniform(0.003, 0.015))
# 最终精确到位
await page.mouse.move(target_x, target_y)
async def human_click(
page,
selector: Optional[str] = None,
*,
x: Optional[float] = None,
y: Optional[float] = None,
) -> None:
"""拟人点击:贝塞尔轨迹移动 → 微延迟 → 点击。
支持两种定位方式:
- selector: CSS 选择器,自动取元素中心附近随机偏移
- x, y: 直接指定坐标(自动加微小随机偏移)
Args:
page: Playwright Page 对象。
selector: CSS 选择器(与 x/y 二选一)。
x: 目标 X 坐标。
y: 目标 Y 坐标。
"""
if selector:
locator = page.locator(selector).first
box = await locator.bounding_box()
if box is None:
raise ValueError(f"无法获取元素边界: {selector}")
target_x = box["x"] + box["width"] * random.uniform(0.3, 0.7)
target_y = box["y"] + box["height"] * random.uniform(0.3, 0.7)
elif x is not None and y is not None:
target_x = x + random.uniform(-3, 3)
target_y = y + random.uniform(-3, 3)
else:
raise ValueError("必须提供 selector 或 (x, y) 坐标")
await human_mouse_move(page, target_x, target_y)
await human_delay_short()
await page.mouse.click(target_x, target_y)
# ── 滚动模拟 ────────────────────────────────────────────────
async def human_scroll(
page, direction: str = "down", amount: Optional[int] = None
) -> None:
"""模拟人工滚动(分段 + 变速 + 微延迟)。
Args:
page: Playwright Page 对象。
direction: "up""down"
amount: 滚动像素总量,默认随机 300~800。
"""
if amount is None:
amount = random.randint(300, 800)
sign = -1 if direction == "up" else 1
remaining = amount
step_count = random.randint(3, 7)
for i in range(step_count):
step = remaining // (step_count - i) + random.randint(-30, 30)
step = max(10, min(step, remaining))
remaining -= step
await page.mouse.wheel(0, sign * step)
await asyncio.sleep(random.uniform(0.05, 0.2))
if remaining > 0:
await page.mouse.wheel(0, sign * remaining)
await human_delay_short()
# ── 进场鼠标随机晃动 ────────────────────────────────────────
async def human_mouse_wiggle(page) -> None:
"""页面进入后随机移动鼠标 2~4 次(贝塞尔轨迹),模拟真人到场环顾。"""
try:
vp = page.viewport_size or {"width": 1280, "height": 800}
except Exception:
vp = {"width": 1280, "height": 800}
w, h = vp.get("width", 1280), vp.get("height", 800)
for _ in range(random.randint(2, 4)):
tx = random.uniform(w * 0.2, w * 0.8)
ty = random.uniform(h * 0.2, h * 0.7)
await human_mouse_move(page, tx, ty)
await asyncio.sleep(random.uniform(0.2, 0.8))
# ── 拟人逐字输入 ────────────────────────────────────────────
async def human_type(page, locator, text: str) -> None:
"""真人式输入:贝塞尔移动到输入框 → 真实点击聚焦 → 逐字符 keyboard.type。
全程使用 isTrusted=true 的可信事件,绝不用 JS 设置 value。
"""
box = await locator.bounding_box()
if box is None:
raise ValueError("human_type: 无法获取输入框边界")
tx = box["x"] + box["width"] * random.uniform(0.3, 0.7)
ty = box["y"] + box["height"] * random.uniform(0.3, 0.7)
await human_mouse_move(page, tx, ty)
await asyncio.sleep(random.uniform(0.15, 0.5))
await page.mouse.click(tx, ty)
await asyncio.sleep(random.uniform(0.2, 0.6))
# 清空已有内容(用键盘,不用 JS
await page.keyboard.press("Control+A")
await page.keyboard.press("Delete")
await asyncio.sleep(random.uniform(0.2, 0.5))
for ch in text:
await page.keyboard.type(ch, delay=random.uniform(90, 240))

View File

@@ -0,0 +1,28 @@
"""RPA 失败存证路径与截图。"""
from __future__ import annotations
import os
from datetime import datetime
def _artifacts_enabled() -> bool:
v = (os.environ.get("OPENCLAW_ARTIFACTS_ON_FAILURE") or "1").strip().lower()
return v not in ("0", "false", "no", "off")
def artifact_path(data_dir: str, batch_id: str, tag: str, ext: str = "png") -> str:
"""返回 {data_dir}/rpa-artifacts/{batch_id}/{tag}_{timestamp}.{ext}"""
ts = datetime.now().strftime("%Y%m%d_%H%M%S")
directory = os.path.join(data_dir, "rpa-artifacts", batch_id)
os.makedirs(directory, exist_ok=True)
return os.path.join(directory, f"{tag}_{ts}.{ext}")
async def capture_failure(page, data_dir: str, batch_id: str, tag: str) -> str:
"""受 OPENCLAW_ARTIFACTS_ON_FAILURE 控制,截图并返回路径;禁用时返回空字符串。"""
if not _artifacts_enabled():
return ""
path = artifact_path(data_dir, batch_id, tag, ext="png")
await page.screenshot(path=path, full_page=True)
return path

View File

@@ -0,0 +1,58 @@
"""统一 persistent context 启动封装(仅浏览器自动化,不负责录屏)。"""
from __future__ import annotations
import os
from typing import TYPE_CHECKING
from ..runtime_env import find_chrome_executable
from .stealth import (
STEALTH_INIT_SCRIPT,
persistent_context_launch_parts,
stealth_enabled,
)
if TYPE_CHECKING:
from playwright.async_api import BrowserContext
async def launch_persistent_browser(
playwright,
*,
profile_dir: str,
channel: str = "chrome",
executable_path: str | None = None,
headless: bool | None = None,
extra_args: list[str] | None = None,
) -> "BrowserContext":
"""用 stealth 参数启动 persistent context + 注入 STEALTH_INIT_SCRIPT。
headless=None 时读 OPENCLAW_BROWSER_HEADLESS。
录屏由 RpaVideoSessionffmpeg负责本函数不向 Playwright 传递任何录屏参数。
"""
if headless is None:
v = (os.environ.get("OPENCLAW_BROWSER_HEADLESS") or "0").strip().lower()
headless = v in ("1", "true", "yes", "on")
chrome = executable_path or find_chrome_executable()
if not chrome:
raise RuntimeError(
"ERROR:MISSING_BROWSER 未检测到 Chrome 或 Edge 浏览器,请安装后重试。"
)
args, ignore = persistent_context_launch_parts(extra_args=extra_args)
launch_kwargs: dict = dict(
user_data_dir=profile_dir,
headless=headless,
executable_path=chrome,
locale="zh-CN",
no_viewport=True,
args=args,
)
if ignore is not None:
launch_kwargs["ignore_default_args"] = ignore
context = await playwright.chromium.launch_persistent_context(**launch_kwargs)
if stealth_enabled():
await context.add_init_script(STEALTH_INIT_SCRIPT)
return context

View File

@@ -0,0 +1,19 @@
"""RPA 场景统一错误码stdout / 异常消息前缀)。"""
ERR_REQUIRE_LOGIN = "ERROR:REQUIRE_LOGIN"
ERR_LOGIN_TIMEOUT = "ERROR:LOGIN_TIMEOUT"
ERR_CAPTCHA_NEED_HUMAN = "ERROR:CAPTCHA_NEED_HUMAN"
ERR_RATE_LIMITED = "ERROR:RATE_LIMITED"
ERR_MISSING_BROWSER = "ERROR:MISSING_BROWSER"
ERR_DEVICE_NOT_READY = "ERROR:DEVICE_NOT_READY"
ERR_WINDOW_NOT_FOUND = "ERROR:WINDOW_NOT_FOUND"
__all__ = [
"ERR_REQUIRE_LOGIN",
"ERR_LOGIN_TIMEOUT",
"ERR_CAPTCHA_NEED_HUMAN",
"ERR_RATE_LIMITED",
"ERR_MISSING_BROWSER",
"ERR_DEVICE_NOT_READY",
"ERR_WINDOW_NOT_FOUND",
]

View File

@@ -0,0 +1,96 @@
"""验证码/登录 HITL 等待(站点无关通用部分)。"""
from __future__ import annotations
import asyncio
DEFAULT_CAPTCHA_MARKERS = (
"punish",
"baxia",
"nc_login",
"tmd",
"x5sec",
"安全验证",
"请按住滑块",
"拖动到最右边",
)
async def wait_for_captcha_pass(
page,
captcha_markers: tuple[str, ...] = DEFAULT_CAPTCHA_MARKERS,
timeout_sec: int = 120,
) -> bool:
"""
检测到验证码/拦截页时,暂停等待用户手工完成验证。
每 3 秒轮询一次页面状态,直到页面不再是拦截状态或超时。
Returns:
True — 验证码已通过,可以继续
False — 超时,用户未在规定时间内完成验证
"""
print(
f"[验证码] 检测到安全拦截,请在浏览器中手工完成验证。"
f"等待最多 {timeout_sec} 秒..."
)
elapsed = 0
while elapsed < timeout_sec:
await asyncio.sleep(3)
elapsed += 3
try:
url = page.url or ""
text = ""
try:
text = await page.locator("body").inner_text(timeout=2_000)
except Exception:
pass
haystack = (url + " " + text[:1000]).lower()
still_blocked = any(m.lower() in haystack for m in captcha_markers)
if not still_blocked:
print(f"[验证码] 已通过!({elapsed}s 后检测到正常页面)")
return True
except Exception:
pass
print(f"[验证码] 超时 {timeout_sec}s用户未完成验证")
return False
async def wait_for_login(
page,
success_selectors: list[str],
timeout_sec: int = 180,
) -> bool:
"""
轮询等待登录成功标志出现(站点专属选择器由调用方传入)。
Args:
page: Playwright Page 对象。
success_selectors: 登录成功后应出现的 CSS 选择器列表。
timeout_sec: 最长等待秒数。
Returns:
True — 检测到登录成功标志
False — 超时
"""
print(
f"[登录] 请在浏览器中完成登录。等待最多 {timeout_sec} 秒,"
"期间程序不会操作页面..."
)
elapsed = 0
while elapsed < timeout_sec:
await asyncio.sleep(2)
elapsed += 2
for selector in success_selectors:
try:
el = await page.query_selector(selector)
if el is not None:
print("[登录] 检测到登录成功!")
return True
except Exception:
pass
print(f"[登录] 超时 {timeout_sec}s未完成登录")
return False

View File

@@ -0,0 +1,66 @@
"""淡化 Playwright 自动化指纹,降低 baxia/x5sec 风控命中率。
关闭:环境变量 OPENCLAW_PLAYWRIGHT_STEALTH=0或 false/off/no
"""
from __future__ import annotations
import os
STEALTH_INIT_SCRIPT = """
(() => {
try { Object.defineProperty(navigator, 'webdriver', { get: () => undefined }); } catch (e) {}
try {
window.chrome = window.chrome || {};
window.chrome.runtime = window.chrome.runtime || {};
} catch (e) {}
try {
const orig = navigator.permissions && navigator.permissions.query;
if (orig) {
navigator.permissions.query = (p) =>
p && p.name === 'notifications'
? Promise.resolve({ state: Notification.permission })
: orig(p);
}
} catch (e) {}
try {
Object.defineProperty(navigator, 'plugins', { get: () => [1, 2, 3, 4, 5] });
} catch (e) {}
try {
Object.defineProperty(navigator, 'languages', { get: () => ['zh-CN', 'zh'] });
} catch (e) {}
try {
Object.defineProperty(navigator, 'hardwareConcurrency', { get: () => 8 });
} catch (e) {}
try {
Object.defineProperty(navigator, 'deviceMemory', { get: () => 8 });
} catch (e) {}
try {
const getParameter = WebGLRenderingContext.prototype.getParameter;
WebGLRenderingContext.prototype.getParameter = function (p) {
if (p === 37445) return 'Intel Inc.';
if (p === 37446) return 'Intel Iris OpenGL Engine';
return getParameter.call(this, p);
};
} catch (e) {}
})();
"""
def stealth_enabled() -> bool:
v = (os.environ.get("OPENCLAW_PLAYWRIGHT_STEALTH") or "1").strip().lower()
return v not in ("0", "false", "no", "off")
def persistent_context_launch_parts(
*, extra_args: list[str] | None = None
) -> tuple[list[str], list[str] | None]:
"""返回 (args, ignore_default_args)。ignore_default_args 为 None 时不要传给 launch。"""
args = ["--start-maximized"]
if extra_args:
args = args + list(extra_args)
if not stealth_enabled():
return args, None
if "--disable-blink-features=AutomationControlled" not in args:
args.append("--disable-blink-features=AutomationControlled")
return args, ["--enable-automation"]

View File

@@ -0,0 +1,420 @@
"""RPA 运行录屏会话ffmpeg 桌面录制 → 字幕 + 背景音乐 → 最终 MP4。
Playwright 仅用于浏览器自动化,不负责录屏。
"""
from __future__ import annotations
import asyncio
import os
import random
import shutil
import subprocess
import sys
import time
from dataclasses import dataclass
from datetime import datetime
from pathlib import Path
from typing import Any, Dict, List, Optional
from .. import config
_JIANGCHANG_SUBTITLE_STYLE = (
"FontName=Arial,FontSize=14,"
"PrimaryColour=&H00FFFFFF,OutlineColour=&H00000000,"
"Outline=2,Shadow=1,Alignment=2"
)
_DEFAULT_MUSIC_ROOT = os.environ.get(
"MEDIA_ASSETS_ROOT",
r"D:\OpenClaw\client-commons\media-assets",
)
_CAPTURE_FRAMERATE = 15
def _record_video_enabled() -> bool:
return config.get_bool("OPENCLAW_RECORD_VIDEO", default=False)
def _resolve_music_file() -> Optional[Path]:
music_dir = Path(_DEFAULT_MUSIC_ROOT) / "music"
if not music_dir.is_dir():
return None
files = list(music_dir.rglob("*.mp3"))
if not files:
return None
return random.choice(files)
@dataclass
class _StepEntry:
start: float
text: str
duration: float = 4.0
def _generate_srt(entries: List[_StepEntry], output_path: str) -> None:
MIN_DURATION = 1.0
GAP = 0.1
steps = sorted(entries, key=lambda e: e.start)
if not steps:
steps = [_StepEntry(0.0, "任务执行中", 4.0)]
for i in range(len(steps) - 1):
cur, nxt = steps[i], steps[i + 1]
max_end = nxt.start - GAP
new_dur = max_end - cur.start
if new_dur < MIN_DURATION:
cur.duration = MIN_DURATION
nxt.start = cur.start + MIN_DURATION + GAP
else:
cur.duration = min(cur.duration, new_dur)
def _fmt(s: float) -> str:
h = int(s // 3600)
m = int((s % 3600) // 60)
sec = int(s % 60)
ms = int((s % 1) * 1000)
return f"{h:02d}:{m:02d}:{sec:02d},{ms:03d}"
Path(output_path).parent.mkdir(parents=True, exist_ok=True)
with open(output_path, "w", encoding="utf-8") as f:
for i, e in enumerate(steps, 1):
f.write(f"{i}\n")
f.write(f"{_fmt(e.start)} --> {_fmt(e.start + e.duration)}\n")
f.write(f"{e.text}\n\n")
def _escape_srt_path(path: str) -> str:
return path.replace("\\", "/").replace(":", "\\:")
def _run_ffmpeg(cmd: List[str]) -> tuple[bool, str]:
try:
result = subprocess.run(
cmd,
capture_output=True,
text=True,
encoding="utf-8",
errors="replace",
)
except FileNotFoundError:
return False, "ffmpeg not found"
if result.returncode != 0:
err = (result.stderr or result.stdout or "").strip()
return False, err[:500] if err else f"exit {result.returncode}"
return True, ""
def _build_desktop_record_cmd(capture_path: str) -> Optional[List[str]]:
"""Windows gdigrab 全桌面录制;非 Windows 返回 None。"""
if sys.platform != "win32":
return None
return [
"ffmpeg",
"-y",
"-f",
"gdigrab",
"-framerate",
str(_CAPTURE_FRAMERATE),
"-i",
"desktop",
"-c:v",
"libx264",
"-preset",
"ultrafast",
"-pix_fmt",
"yuv420p",
capture_path,
]
def _compose_capture_mp4(
capture_path: Path,
srt_path: Path,
output_path: Path,
*,
music_file: Optional[Path] = None,
music_volume: float = 0.15,
) -> tuple[bool, str]:
output_path.parent.mkdir(parents=True, exist_ok=True)
srt_esc = _escape_srt_path(str(srt_path.resolve()))
subtitle_filter = (
f"subtitles='{srt_esc}'"
f":force_style='{_JIANGCHANG_SUBTITLE_STYLE}'"
)
if music_file and music_file.is_file():
filter_complex = (
f"[0:v]{subtitle_filter}[vout];"
f"[1:a]volume={music_volume},apad[aout]"
)
cmd = [
"ffmpeg",
"-y",
"-i",
str(capture_path),
"-i",
str(music_file),
"-filter_complex",
filter_complex,
"-map",
"[vout]",
"-map",
"[aout]",
"-c:v",
"libx264",
"-preset",
"fast",
"-crf",
"23",
"-c:a",
"aac",
"-b:a",
"128k",
"-shortest",
"-pix_fmt",
"yuv420p",
str(output_path),
]
else:
cmd = [
"ffmpeg",
"-y",
"-i",
str(capture_path),
"-vf",
subtitle_filter,
"-c:v",
"libx264",
"-preset",
"fast",
"-crf",
"23",
"-c:a",
"aac",
"-b:a",
"128k",
"-pix_fmt",
"yuv420p",
str(output_path),
]
return _run_ffmpeg(cmd)
class RpaVideoSession:
"""Service/RPA 层统一 ffmpeg 桌面录屏OPENCLAW_RECORD_VIDEO=1 时启用成片流程。"""
def __init__(
self,
*,
skill_slug: str,
skill_data_dir: str,
batch_id: str,
title: str = "",
) -> None:
self.skill_slug = skill_slug
self.skill_data_dir = os.path.abspath(skill_data_dir)
self.batch_id = batch_id
self.title = title
self.enabled = _record_video_enabled()
self.warnings: List[str] = []
self.output_video_path: Optional[str] = None
self.capture_path: Optional[str] = None
# 已废弃:勿用于 Playwright始终为 None。
self.record_video_dir: Optional[str] = None
self.artifacts: Dict[str, Any] = {}
ts = datetime.now().strftime("%Y%m%d_%H%M%S")
safe_batch = "".join(c if c.isalnum() or c in "-_" else "_" for c in batch_id)
self._video_stem = f"{skill_slug}_{ts}_{safe_batch}"
self._videos_dir = os.path.join(self.skill_data_dir, "videos")
self._artifact_root = os.path.join(
self.skill_data_dir, "rpa-artifacts", batch_id
)
self._subtitles_dir = os.path.join(self._artifact_root, "subtitles")
self._logs_dir = os.path.join(self._artifact_root, "logs")
self._capture_path = os.path.join(self._artifact_root, "capture.mp4")
self._srt_path = os.path.join(self._subtitles_dir, f"{self._video_stem}.srt")
self._planned_output = os.path.join(self._videos_dir, f"{self._video_stem}.mp4")
self._steps: List[_StepEntry] = []
self._t0: Optional[float] = None
self._ffmpeg_proc: Optional[subprocess.Popen] = None
if self.enabled:
os.makedirs(self._videos_dir, exist_ok=True)
os.makedirs(self._subtitles_dir, exist_ok=True)
os.makedirs(self._logs_dir, exist_ok=True)
os.makedirs(self._artifact_root, exist_ok=True)
def step(self, text: str, *, duration: float = 4.0) -> None:
self.add_step(text, duration=duration)
def add_step(self, text: str, *, duration: float = 4.0) -> None:
"""记录一步骤字幕(相对录屏开始时间)。"""
if not self.enabled:
return
if self._t0 is None:
self._t0 = time.monotonic()
elapsed = time.monotonic() - self._t0
self._steps.append(_StepEntry(start=elapsed, text=text.strip(), duration=duration))
def _start_ffmpeg_capture(self) -> None:
if not self.enabled:
return
if not shutil.which("ffmpeg"):
self.warnings.append("ffmpeg_not_found")
return
cmd = _build_desktop_record_cmd(self._capture_path)
if cmd is None:
self.warnings.append(
"ffmpeg_record_start_failed: desktop capture only supported on Windows"
)
return
try:
self._ffmpeg_proc = subprocess.Popen(
cmd,
stdin=subprocess.PIPE,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
self.capture_path = self._capture_path
except Exception as exc:
self.warnings.append(f"ffmpeg_record_start_failed: {exc}")
self._ffmpeg_proc = None
def _stop_ffmpeg_capture(self) -> None:
proc = self._ffmpeg_proc
if proc is None:
return
if proc.poll() is None:
try:
if proc.stdin:
proc.stdin.write(b"q")
proc.stdin.flush()
except Exception:
try:
proc.terminate()
except Exception:
pass
try:
proc.wait(timeout=20)
except subprocess.TimeoutExpired:
try:
proc.kill()
except Exception:
pass
try:
proc.wait(timeout=5)
except Exception:
pass
self._ffmpeg_proc = None
await_sleep = 0.3
time.sleep(await_sleep)
async def __aenter__(self) -> "RpaVideoSession":
if self.enabled:
self._t0 = time.monotonic()
if self.title:
self.add_step(self.title, duration=3.0)
self._start_ffmpeg_capture()
return self
async def __aexit__(self, exc_type, exc, tb) -> None:
if self.enabled:
self._stop_ffmpeg_capture()
await self.finalize()
async def finalize(self) -> None:
"""停止录制后合成最终 MP4失败只记 warning不抛异常。"""
if not self.enabled:
self.artifacts = self.summary()
return
if not self._steps and self.title:
self.add_step(self.title)
try:
capture = Path(self._capture_path)
if not capture.is_file() or capture.stat().st_size <= 0:
self.warnings.append("ffmpeg_capture_missing")
self.artifacts = self.summary()
return
self.capture_path = str(capture.resolve())
entries = list(self._steps)
if entries:
entries[-1].duration = max(entries[-1].duration, 5.0)
try:
_generate_srt(entries, self._srt_path)
except Exception as exc:
self.warnings.append(f"subtitle_write_failed: {exc}")
music = _resolve_music_file()
if music is None:
self.warnings.append("background_music_missing")
srt_file = Path(self._srt_path)
use_srt = srt_file.is_file() and srt_file.stat().st_size > 0
ok, err = _compose_capture_mp4(
capture,
srt_file if use_srt else Path(self._srt_path),
Path(self._planned_output),
music_file=music,
)
if not ok and music is not None:
self.warnings.append(f"ffmpeg_with_music_failed: {err}")
ok, err = _compose_capture_mp4(
capture,
srt_file if use_srt else Path(self._srt_path),
Path(self._planned_output),
music_file=None,
)
if not ok and not use_srt:
ok, err = _run_ffmpeg(
[
"ffmpeg",
"-y",
"-i",
str(capture),
"-c:v",
"libx264",
"-preset",
"fast",
"-crf",
"23",
"-pix_fmt",
"yuv420p",
str(self._planned_output),
]
)
if not ok:
self.warnings.append(f"ffmpeg_compose_failed: {err}")
self.artifacts = self.summary()
return
self.output_video_path = str(Path(self._planned_output).resolve())
except Exception as exc:
self.warnings.append(f"video_finalize_error: {exc}")
self.artifacts = self.summary()
def summary(self) -> Dict[str, Any]:
return {
"enabled": self.enabled,
"path": self.output_video_path,
"capture_path": self.capture_path,
"warnings": list(self.warnings),
}
def to_artifact_dict(self) -> Dict[str, Any]:
data = self.summary()
data["batch_id"] = self.batch_id
return data

View File

@@ -134,3 +134,74 @@ def apply_cli_local_defaults() -> None:
os.environ["JIANGCHANG_DATA_ROOT"] = platform_default_data_root()
if not (os.getenv("JIANGCHANG_USER_ID") or "").strip():
os.environ["JIANGCHANG_USER_ID"] = DEFAULT_LOCAL_USER_ID.strip()
def find_chrome_executable() -> "str | None":
"""查找系统已安装的 Chrome 或 Edge 可执行文件路径。
查找顺序:
1. 常见文件路径PROGRAMFILES、LOCALAPPDATA、macOS、Linux
2. Windows 注册表 App PathsHKLM 优先HKCU 其次)
3. PATH shutil.which 兜底
返回找到的路径字符串,找不到返回 None。
宿主/技能调用 Playwright launch 时应传 executable_path=find_chrome_executable()
而非 channel="chrome",以避免 Playwright 内部检测在系统级安装时失效。
"""
import shutil
if sys.platform == "win32":
local_app_data = os.environ.get("LOCALAPPDATA", "")
prog_files = os.environ.get("PROGRAMFILES", r"C:\Program Files")
prog_files_x86 = os.environ.get("PROGRAMFILES(X86)", r"C:\Program Files (x86)")
candidates = [
os.path.join(prog_files, r"Google\Chrome\Application\chrome.exe"),
os.path.join(prog_files_x86, r"Google\Chrome\Application\chrome.exe"),
os.path.join(local_app_data, r"Google\Chrome\Application\chrome.exe"),
os.path.join(prog_files, r"Microsoft\Edge\Application\msedge.exe"),
os.path.join(prog_files_x86, r"Microsoft\Edge\Application\msedge.exe"),
os.path.join(local_app_data, r"Microsoft\Edge\Application\msedge.exe"),
]
elif sys.platform == "darwin":
candidates = [
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
"/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge",
"/Applications/Chromium.app/Contents/MacOS/Chromium",
]
else:
candidates = [
"/usr/bin/google-chrome-stable",
"/usr/bin/google-chrome",
"/usr/bin/chromium-browser",
"/usr/bin/chromium",
"/usr/bin/microsoft-edge",
]
for path in candidates:
if path and os.path.isfile(path):
return path
# Windows 注册表兜底(覆盖系统级 HKLM 安装)
if sys.platform == "win32":
try:
import winreg
for hive in (winreg.HKEY_LOCAL_MACHINE, winreg.HKEY_CURRENT_USER):
for reg_path in (
r"SOFTWARE\Microsoft\Windows\CurrentVersion\App Paths\chrome.exe",
r"SOFTWARE\Microsoft\Windows\CurrentVersion\App Paths\msedge.exe",
):
try:
with winreg.OpenKey(hive, reg_path) as key:
val, _ = winreg.QueryValueEx(key, "")
if val and os.path.isfile(val):
return val
except OSError:
continue
except ImportError:
pass
# PATH 兜底
for name in ("google-chrome-stable", "google-chrome", "chrome", "msedge", "chromium"):
found = shutil.which(name)
if found:
return found
return None

View File

@@ -0,0 +1,3 @@
from .runner import run_screencast
__all__ = ["run_screencast"]

View File

@@ -0,0 +1,80 @@
"""FFmpeg 合成:帧序列 + SRT 字幕 + 背景音乐 → MP4。"""
from __future__ import annotations
import random
import subprocess
import sys
from pathlib import Path
# 匠厂统一字幕样式 —— 所有 skill 录屏共用,做品牌一致性
# FontSize 141080p 屏幕约占 1.3% 高度,演示视频留足画面呼吸感
# Outline 214 号字配 Outline 3 会显胖2 已足够在浅色背景上保持可读
# 修改请评估对所有 skill 录屏的影响。
_JIANGCHANG_SUBTITLE_STYLE = (
"FontName=Arial,FontSize=14,"
"PrimaryColour=&H00FFFFFF,OutlineColour=&H00000000,"
"Outline=2,Shadow=1,Alignment=2"
)
def compose_video(
frames_dir: str,
fps: int,
subtitle_path: str,
music_dir: str,
output_path: str,
music_volume: float = 0.15,
) -> str:
frames_dir = Path(frames_dir)
subtitle_path = Path(subtitle_path)
output_path = Path(output_path)
output_path.parent.mkdir(parents=True, exist_ok=True)
# 随机选一首背景音乐
music_files = list(Path(music_dir).rglob("*.mp3"))
if not music_files:
raise FileNotFoundError(f"music_dir 下没有找到 MP3 文件: {music_dir}")
music_file = random.choice(music_files)
print(f"[screencast] 背景音乐: {music_file.name}")
# FFmpeg 字幕路径在 Windows 下需要转义冒号subtitles 滤镜内部语法)
srt_path_str = str(subtitle_path).replace("\\", "/").replace(":", "\\:")
subtitle_filter = (
f"subtitles='{srt_path_str}'"
f":force_style='{_JIANGCHANG_SUBTITLE_STYLE}'"
)
# 视频和音频都放进 filter_complex避免 -vf 与 -filter_complex 混用报错
filter_complex = (
f"[0:v]{subtitle_filter}[vout];"
f"[1:a]volume={music_volume},apad[aout]"
)
cmd = [
"ffmpeg", "-y",
"-framerate", str(fps),
"-i", str(frames_dir / "frame_%06d.png"),
"-i", str(music_file),
"-filter_complex", filter_complex,
"-map", "[vout]",
"-map", "[aout]",
"-c:v", "libx264",
"-preset", "fast",
"-crf", "23",
"-c:a", "aac",
"-b:a", "128k",
"-shortest",
"-pix_fmt", "yuv420p",
str(output_path),
]
print(f"[screencast] 运行 FFmpeg 合成...")
result = subprocess.run(cmd, capture_output=True, text=True, encoding="utf-8", errors="replace")
if result.returncode != 0:
print(result.stderr, file=sys.stderr)
raise RuntimeError(f"FFmpeg 合成失败,退出码 {result.returncode}")
print(f"[screencast] 输出: {output_path}")
return str(output_path)

View File

@@ -0,0 +1,63 @@
"""全屏截帧引擎,后台线程持续截图存为 PNG 帧序列。"""
from __future__ import annotations
import threading
import time
from pathlib import Path
from typing import Optional
import mss
import mss.tools
class ScreenRecorder:
def __init__(
self,
frames_dir: str,
fps: int = 10,
monitor_index: Optional[int] = None,
):
"""
Args:
monitor_index: mss 显示器索引None=monitors[0](所有屏合并区域,
旧行为1=主屏2+=第 N 屏。多显示器场景推荐显式传 1。
"""
self._frames_dir = Path(frames_dir)
self._fps = fps
self._monitor_index = monitor_index
self._stop_event = threading.Event()
self._thread: threading.Thread | None = None
self._frame_count = 0
def start(self) -> None:
self._frames_dir.mkdir(parents=True, exist_ok=True)
self._stop_event.clear()
self._frame_count = 0
self._thread = threading.Thread(target=self._capture_loop, daemon=True)
self._thread.start()
def stop(self) -> None:
self._stop_event.set()
if self._thread:
self._thread.join(timeout=10)
@property
def frame_count(self) -> int:
return self._frame_count
def _capture_loop(self) -> None:
interval = 1.0 / self._fps
with mss.mss() as sct:
idx = self._monitor_index if self._monitor_index is not None else 0
if idx < 0 or idx >= len(sct.monitors):
idx = 0 # 越界兜底
monitor = sct.monitors[idx]
while not self._stop_event.is_set():
t0 = time.perf_counter()
img = sct.grab(monitor)
path = self._frames_dir / f"frame_{self._frame_count:06d}.png"
mss.tools.to_png(img.rgb, img.size, output=str(path))
self._frame_count += 1
elapsed = time.perf_counter() - t0
sleep = max(0.0, interval - elapsed)
time.sleep(sleep)

142
src/screencast/runner.py Normal file
View File

@@ -0,0 +1,142 @@
"""主编排器:录屏 + 运行 pytest + 生成字幕 + FFmpeg 合成。"""
from __future__ import annotations
import os
import subprocess
import sys
import tempfile
import time
from datetime import datetime
from pathlib import Path
from typing import List, Optional, Tuple
def _ensure_sdk_on_path() -> None:
"""pip 安装后应能直接 import仅源码开发态才把 src 加入 sys.path。"""
import importlib.util
if importlib.util.find_spec("jiangchang_desktop_sdk") is not None:
return
# src/screencast/runner.py → src/screencast → src
here = Path(__file__).resolve()
src = here.parent.parent
sdk_dir = src / "jiangchang_desktop_sdk"
if sdk_dir.is_dir():
src_str = str(src)
if src_str not in sys.path:
sys.path.insert(0, src_str)
_ensure_sdk_on_path()
from jiangchang_desktop_sdk.window_win32 import activate_and_maximize_jiangchang_window # noqa: E402
from .composer import compose_video
from .recorder import ScreenRecorder
from .subtitle import SubtitleEngine
def run_screencast(
skill_slug: str,
subtitle_script: List[Tuple[str, str]],
pytest_args: List[str],
output_dir: str,
media_assets_root: Optional[str] = None,
music_subdir: str = "music",
fps: int = 10,
activate_window: bool = True,
monitor_index: Optional[int] = None,
) -> str:
"""
录制一次技能桌面 E2E 演示视频。
Args:
skill_slug: 技能唯一标识,如 "query-balance-icbc"
subtitle_script: [(关键词, 字幕文案), ...] 按顺序匹配 pytest stdout
pytest_args: 传递给 pytest 的参数列表
output_dir: 最终 MP4 输出目录
media_assets_root: media-assets 仓库本地路径;
不传时读 MEDIA_ASSETS_ROOT 环境变量,
再不存在则用默认值 D:\\OpenClaw\\client-commons\\media-assets
music_subdir: music_assets_root 下音乐子目录,默认 "music"
fps: 截帧帧率,默认 10
activate_window: 录屏开始前是否自动激活+最大化匠厂窗口(默认 True
Windows 走 ctypes非 Windows 走 jiangchang:// 协议)
monitor_index: 指定录哪个显示器None=所有显示器合并(旧行为),
1=主屏2+=第 N 屏(推荐多显示器场景显式传 1
Returns:
输出 MP4 的绝对路径
"""
if media_assets_root is None:
media_assets_root = os.environ.get(
"MEDIA_ASSETS_ROOT",
r"D:\OpenClaw\client-commons\media-assets",
)
music_dir = str(Path(media_assets_root) / music_subdir)
output_dir_path = Path(output_dir)
output_dir_path.mkdir(parents=True, exist_ok=True)
date_str = datetime.now().strftime("%Y%m%d_%H%M%S")
base_name = f"{skill_slug}_{date_str}"
output_mp4 = output_dir_path / f"{base_name}.mp4"
subtitle_file = output_dir_path / f"{base_name}.srt"
engine = SubtitleEngine(subtitle_script)
with tempfile.TemporaryDirectory(prefix="screencast_frames_") as frames_dir:
recorder = ScreenRecorder(frames_dir, fps=fps, monitor_index=monitor_index)
if activate_window:
ok = activate_and_maximize_jiangchang_window()
if ok:
print("[screencast] 已激活+最大化匠厂窗口")
time.sleep(2) # 等窗口浮起稳定
else:
print("[screencast] 未能激活匠厂窗口,继续录制(可能录到桌面边角)")
print(f"[screencast] 启动录制 → {output_mp4.name}")
recorder.start()
engine.set_start_time(time.time())
try:
# Windows 子进程 stdout 默认走系统 ANSI code page中文 Windows 是 cp936/GBK
# 这边用 encoding="utf-8" 解码会乱码。注入 PYTHONIOENCODING + PYTHONUTF8 让
# 子进程 Python 强制以 UTF-8 输出,两端编码对齐。非 Windows 默认就 UTF-8无副作用。
env = os.environ.copy()
env["PYTHONIOENCODING"] = "utf-8"
env["PYTHONUTF8"] = "1"
proc = subprocess.Popen(
[sys.executable, "-m", "pytest", *pytest_args],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
encoding="utf-8",
errors="replace",
env=env,
)
assert proc.stdout is not None
for line in proc.stdout:
sys.stdout.write(line)
sys.stdout.flush()
engine.process_line(line)
exit_code = proc.wait()
if exit_code != 0:
print(f"[screencast] pytest 退出码 {exit_code},录制仍继续合成")
finally:
recorder.stop()
print(f"[screencast] 录制停止,共 {recorder.frame_count}")
engine.generate_srt(str(subtitle_file))
print(f"[screencast] 字幕 → {subtitle_file.name}")
compose_video(
frames_dir=frames_dir,
fps=fps,
subtitle_path=str(subtitle_file),
music_dir=music_dir,
output_path=str(output_mp4),
)
return str(output_mp4)

View File

@@ -0,0 +1,94 @@
"""字幕引擎:根据 stdout 关键词打时间戳,生成 SRT 字幕文件。"""
from __future__ import annotations
import time
from dataclasses import dataclass
from pathlib import Path
from typing import List, Tuple
@dataclass
class _Entry:
start: float # 距录制开始的秒数
text: str
duration: float = 5.0 # 每条字幕默认显示 5 秒
class SubtitleEngine:
def __init__(
self,
script: List[Tuple],
default_duration: float = 5.0,
):
"""
script: [(关键词, 字幕文案) | (关键词, 字幕文案, 显示时长秒), ...]
- 二元组:用 default_duration向后兼容
- 三元组:本条字幕独立指定显示秒数(用于长等待场景延长字幕停留)
关键词大小写不敏感,每个关键词只触发一次。
"""
self._script = script
self._default_duration = default_duration
self._entries: List[_Entry] = []
self._matched: set[str] = set()
self._t0: float = 0.0
def set_start_time(self, t: float) -> None:
self._t0 = t
def process_line(self, line: str) -> None:
lower = line.lower()
for entry_tuple in self._script:
if len(entry_tuple) == 2:
keyword, text = entry_tuple
duration = self._default_duration
elif len(entry_tuple) == 3:
keyword, text, duration = entry_tuple
else:
continue
if keyword in self._matched:
continue
if keyword.lower() in lower:
elapsed = time.time() - self._t0
self._entries.append(_Entry(start=elapsed, text=text, duration=float(duration)))
self._matched.add(keyword)
def generate_srt(self, output_path: str) -> None:
"""生成 SRT 文件,自动处理字幕重叠。
字幕重叠修复策略(避免屏幕上同时显示 2+ 条叠成两行):
- 字幕按 start 时间升序排序后;
- 相邻字幕保留 GAP=0.1s 间隔;
- 每条字幕保证至少 MIN_DURATION=1.0s 显示时间(短到这个值仍读不完是字幕脚本设计问题);
- 若当前条让位给下一条后剩余时间 < MIN_DURATION则把下一条 start 往后推。
这样任意时刻屏幕上最多 1 条字幕。
"""
def _fmt(s: float) -> str:
h = int(s // 3600)
m = int((s % 3600) // 60)
sec = int(s % 60)
ms = int((s % 1) * 1000)
return f"{h:02d}:{m:02d}:{sec:02d},{ms:03d}"
# 防重叠 post-process
MIN_DURATION = 1.0
GAP = 0.1
entries = sorted(self._entries, key=lambda e: e.start)
for i in range(len(entries) - 1):
cur, nxt = entries[i], entries[i + 1]
max_end = nxt.start - GAP
new_dur = max_end - cur.start
if new_dur < MIN_DURATION:
# 太挤cur 至少撑 MIN_DURATION 秒,把 nxt 推迟
cur.duration = MIN_DURATION
nxt.start = cur.start + MIN_DURATION + GAP
else:
# 正常让位cur 显示到 (nxt.start - GAP)
cur.duration = min(cur.duration, new_dur)
# 最后一条不需要让位,保持原 duration
Path(output_path).parent.mkdir(parents=True, exist_ok=True)
with open(output_path, "w", encoding="utf-8") as f:
for i, e in enumerate(entries, 1):
f.write(f"{i}\n")
f.write(f"{_fmt(e.start)} --> {_fmt(e.start + e.duration)}\n")
f.write(f"{e.text}\n\n")

1
tests/__init__.py Normal file
View File

@@ -0,0 +1 @@
# Package marker for unittest discovery.

View File

@@ -0,0 +1,26 @@
# -*- coding: utf-8 -*-
"""anti_detect / rpa 子包导入冒烟。"""
from __future__ import annotations
import importlib
import unittest
class TestAntiDetectImport(unittest.TestCase):
def test_import_anti_detect(self) -> None:
mod = importlib.import_module("jiangchang_skill_core.rpa.anti_detect")
self.assertTrue(callable(mod.human_click))
self.assertTrue(callable(mod.random_delay))
def test_import_rpa_package(self) -> None:
mod = importlib.import_module("jiangchang_skill_core.rpa")
self.assertTrue(callable(mod.wait_for_captcha_pass))
self.assertIsNotNone(mod.stealth_enabled)
def test_top_level_import_with_rpa(self) -> None:
pkg = importlib.import_module("jiangchang_skill_core")
self.assertIsNotNone(getattr(pkg, "config", None))
if __name__ == "__main__":
unittest.main()

86
tests/test_config.py Normal file
View File

@@ -0,0 +1,86 @@
# -*- coding: utf-8 -*-
"""config 三级优先级与 ensure_env_file 测试。"""
from __future__ import annotations
import os
import tempfile
import unittest
from jiangchang_skill_core import config
class TestConfig(unittest.TestCase):
def setUp(self) -> None:
config.reset_cache()
self._saved_env = dict(os.environ)
def tearDown(self) -> None:
os.environ.clear()
os.environ.update(self._saved_env)
config.reset_cache()
def test_ensure_env_file_copies_once(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
example = os.path.join(tmp, ".env.example")
with open(example, "w", encoding="utf-8") as f:
f.write("FOO=from_example\nBAR=2\n")
os.environ["JIANGCHANG_DATA_ROOT"] = tmp
os.environ["JIANGCHANG_USER_ID"] = "testuser"
dest = config.ensure_env_file("demo-skill", example)
self.assertTrue(os.path.isfile(dest))
with open(dest, encoding="utf-8") as f:
content = f.read()
self.assertIn("FOO=from_example", content)
with open(dest, "w", encoding="utf-8") as f:
f.write("FOO=user_edited\n")
dest2 = config.ensure_env_file("demo-skill", example)
self.assertEqual(dest, dest2)
with open(dest2, encoding="utf-8") as f:
self.assertIn("user_edited", f.read())
def test_three_level_priority(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
example = os.path.join(tmp, ".env.example")
with open(example, "w", encoding="utf-8") as f:
f.write("KEY_A=example\nKEY_B=example\nKEY_C=example\n")
os.environ["JIANGCHANG_DATA_ROOT"] = tmp
os.environ["JIANGCHANG_USER_ID"] = "u1"
dest = config.ensure_env_file("sk", example)
with open(dest, "w", encoding="utf-8") as f:
f.write("KEY_B=user_env\n")
config.reset_cache()
self.assertEqual(config.get("KEY_A"), "example")
self.assertEqual(config.get("KEY_B"), "user_env")
self.assertEqual(config.get("KEY_C"), "example")
self.assertIsNone(config.get("MISSING"))
os.environ["KEY_C"] = "process_env"
self.assertEqual(config.get("KEY_C"), "process_env")
def test_get_bool_float_int(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
example = os.path.join(tmp, ".env.example")
with open(example, "w", encoding="utf-8") as f:
f.write("FLAG=1\nRATE=2.5\nCOUNT=42\nBAD=x\n")
os.environ["JIANGCHANG_DATA_ROOT"] = tmp
os.environ["JIANGCHANG_USER_ID"] = "u1"
config.ensure_env_file("sk", example)
self.assertTrue(config.get_bool("FLAG"))
self.assertFalse(config.get_bool("MISSING", default=False))
self.assertAlmostEqual(config.get_float("RATE", 0.0), 2.5)
self.assertEqual(config.get_int("COUNT", 0), 42)
self.assertEqual(config.get_float("BAD", 9.9), 9.9)
self.assertEqual(config.get_int("BAD", 7), 7)
if __name__ == "__main__":
unittest.main()

125
tests/test_e2e_helpers.py Normal file
View File

@@ -0,0 +1,125 @@
# -*- coding: utf-8 -*-
"""e2e_helpers 轻量单元测试(不依赖匠厂桌面 / IPC"""
from __future__ import annotations
import tempfile
import unittest
from pathlib import Path
from jiangchang_desktop_sdk.e2e_helpers import (
SkillDataDir,
_iter_text_and_newlines,
_is_placeholder_value,
_parse_env_file,
assert_skill_env_file,
)
def _segments(prompt: str) -> list[str | None]:
out: list[str | None] = []
for seg in _iter_text_and_newlines(prompt):
if seg is not None and not isinstance(seg, str):
out.append(None)
else:
out.append(seg) # type: ignore[arg-type]
return out
class TestIterTextAndNewlines(unittest.TestCase):
def test_lf(self) -> None:
self.assertEqual(_segments("a\nb"), ["a", None, "b"])
def test_crlf(self) -> None:
self.assertEqual(_segments("a\r\nb"), ["a", None, "b"])
def test_cr(self) -> None:
self.assertEqual(_segments("a\rb"), ["a", None, "b"])
def test_multiple_newlines(self) -> None:
self.assertEqual(_segments("x\n\ny"), ["x", None, None, "y"])
class TestParseEnvFile(unittest.TestCase):
def _write(self, path: Path, content: str) -> None:
path.write_text(content, encoding="utf-8")
def test_basic_and_comments(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
env_path = Path(tmp) / ".env"
self._write(
env_path,
"# comment\n\nAPI_KEY=real-secret\n# KEY=ignored\n",
)
parsed = _parse_env_file(env_path)
self.assertEqual(parsed["API_KEY"], "real-secret")
self.assertNotIn("KEY", parsed)
def test_quoted_values(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
env_path = Path(tmp) / ".env"
self._write(env_path, 'A="double"\nB=\'single\'\n')
parsed = _parse_env_file(env_path)
self.assertEqual(parsed["A"], "double")
self.assertEqual(parsed["B"], "single")
def test_placeholder_detection(self) -> None:
self.assertTrue(_is_placeholder_value("TODO", None))
self.assertTrue(_is_placeholder_value("sk-xxx", None))
self.assertTrue(_is_placeholder_value(" Changeme ", None))
self.assertFalse(_is_placeholder_value("sk-live-abc", None))
class TestAssertSkillEnvFile(unittest.TestCase):
def test_missing_file(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
ctx = SkillDataDir(
root=Path(tmp),
path=Path(tmp) / "user" / "demo-skill",
user_id="12428",
skill_slug="demo-skill",
)
with self.assertRaises(AssertionError) as cm:
assert_skill_env_file(ctx, ["API_KEY"])
self.assertIn("config/.env", str(cm.exception))
def test_missing_and_placeholder_keys(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
skill_dir = Path(tmp) / "12428" / "demo-skill"
env_dir = skill_dir / "config"
env_dir.mkdir(parents=True)
(env_dir / ".env").write_text(
"API_KEY=your-key\nOTHER=real\n",
encoding="utf-8",
)
ctx = SkillDataDir(
root=Path(tmp),
path=skill_dir,
user_id="12428",
skill_slug="demo-skill",
)
with self.assertRaises(AssertionError) as cm:
assert_skill_env_file(ctx, ["API_KEY", "MISSING", "OTHER"])
msg = str(cm.exception)
self.assertIn("API_KEY", msg)
self.assertIn("MISSING", msg)
self.assertNotIn("OTHER", msg)
def test_pass_returns_path(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
skill_dir = Path(tmp) / "1" / "demo-skill"
env_dir = skill_dir / "config"
env_dir.mkdir(parents=True)
env_file = env_dir / ".env"
env_file.write_text("API_KEY=sk-live-ok\n", encoding="utf-8")
ctx = SkillDataDir(
root=Path(tmp),
path=skill_dir,
user_id="1",
skill_slug="demo-skill",
)
got = assert_skill_env_file(ctx, ["API_KEY"])
self.assertEqual(got, env_file)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,42 @@
"""Text-level checks that reusable-release-skill packages .env.example at package root."""
from __future__ import annotations
import os
import re
import pytest
_REPO_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
_WORKFLOW_PATH = os.path.join(
_REPO_ROOT, ".github", "workflows", "reusable-release-skill.yaml"
)
@pytest.fixture(scope="module")
def workflow_text() -> str:
with open(_WORKFLOW_PATH, encoding="utf-8") as f:
return f.read()
def test_workflow_packages_env_example_at_root(workflow_text: str) -> None:
assert ".env.example" in workflow_text
assert "Copied .env.example" in workflow_text
assert "env_example_src = os.path.abspath('.env.example')" in workflow_text
assert "env_example_dst = os.path.join(PACKAGE, '.env.example')" in workflow_text
assert (
"raise RuntimeError('.env.example exists in skill root but was not packaged')"
in workflow_text
)
def test_workflow_does_not_package_dot_env(workflow_text: str) -> None:
# Must not copy live .env to package root (only .env.example).
assert not re.search(
r"shutil\.copy2\([^)]*['\"]\.env['\"]",
workflow_text,
)
assert not re.search(
r"os\.path\.join\(PACKAGE,\s*['\"]\.env['\"]\)",
workflow_text,
)

54
tests/test_stealth.py Normal file
View File

@@ -0,0 +1,54 @@
# -*- coding: utf-8 -*-
"""stealth 启动参数与开关测试。"""
from __future__ import annotations
import os
import unittest
from jiangchang_skill_core.rpa.stealth import (
persistent_context_launch_parts,
stealth_enabled,
)
class TestStealth(unittest.TestCase):
def setUp(self) -> None:
self._saved = os.environ.get("OPENCLAW_PLAYWRIGHT_STEALTH")
def tearDown(self) -> None:
if self._saved is None:
os.environ.pop("OPENCLAW_PLAYWRIGHT_STEALTH", None)
else:
os.environ["OPENCLAW_PLAYWRIGHT_STEALTH"] = self._saved
def test_stealth_enabled_default_on(self) -> None:
os.environ.pop("OPENCLAW_PLAYWRIGHT_STEALTH", None)
self.assertTrue(stealth_enabled())
def test_stealth_enabled_off_values(self) -> None:
for off in ("0", "false", "no", "off", "FALSE"):
os.environ["OPENCLAW_PLAYWRIGHT_STEALTH"] = off
self.assertFalse(stealth_enabled(), msg=off)
def test_launch_parts_stealth_on(self) -> None:
os.environ["OPENCLAW_PLAYWRIGHT_STEALTH"] = "1"
args, ignore = persistent_context_launch_parts()
self.assertIn("--start-maximized", args)
self.assertIn("--disable-blink-features=AutomationControlled", args)
self.assertEqual(ignore, ["--enable-automation"])
def test_launch_parts_stealth_off(self) -> None:
os.environ["OPENCLAW_PLAYWRIGHT_STEALTH"] = "0"
args, ignore = persistent_context_launch_parts()
self.assertIn("--start-maximized", args)
self.assertNotIn("--disable-blink-features=AutomationControlled", args)
self.assertIsNone(ignore)
def test_launch_parts_extra_args(self) -> None:
os.environ["OPENCLAW_PLAYWRIGHT_STEALTH"] = "1"
args, _ = persistent_context_launch_parts(extra_args=["--foo"])
self.assertIn("--foo", args)
if __name__ == "__main__":
unittest.main()

151
tests/test_video_session.py Normal file
View File

@@ -0,0 +1,151 @@
# -*- coding: utf-8 -*-
"""RpaVideoSessionffmpeg 桌面录屏,无 Playwright record_video_dir。"""
from __future__ import annotations
import asyncio
import os
import sys
import tempfile
import unittest
from unittest.mock import MagicMock, patch
from jiangchang_skill_core import config
from jiangchang_skill_core.rpa.video_session import RpaVideoSession
class TestRpaVideoSession(unittest.TestCase):
def setUp(self) -> None:
config.reset_cache()
def test_disabled_is_noop(self) -> None:
old = os.environ.pop("OPENCLAW_RECORD_VIDEO", None)
try:
with tempfile.TemporaryDirectory() as tmp:
config.reset_cache()
session = RpaVideoSession(
skill_slug="receive-order",
skill_data_dir=tmp,
batch_id="batch_test",
)
self.assertFalse(session.enabled)
self.assertIsNone(session.record_video_dir)
self.assertIsNone(session.output_video_path)
self.assertIsNone(session.capture_path)
summ = session.summary()
self.assertFalse(summ["enabled"])
self.assertIsNone(summ["path"])
finally:
if old is not None:
os.environ["OPENCLAW_RECORD_VIDEO"] = old
config.reset_cache()
def test_enabled_paths_no_raw_video(self) -> None:
os.environ["OPENCLAW_RECORD_VIDEO"] = "1"
try:
with tempfile.TemporaryDirectory() as tmp:
config.reset_cache()
session = RpaVideoSession(
skill_slug="receive-order",
skill_data_dir=tmp,
batch_id="exmail_20260602_120000",
title="接收订单",
)
self.assertTrue(session.enabled)
self.assertIsNone(session.record_video_dir)
self.assertTrue(
session._capture_path.replace("\\", "/").endswith(
"/rpa-artifacts/exmail_20260602_120000/capture.mp4"
)
)
self.assertTrue(
session._planned_output.replace("\\", "/").startswith(
tmp.replace("\\", "/") + "/videos/"
)
)
finally:
os.environ.pop("OPENCLAW_RECORD_VIDEO", None)
config.reset_cache()
@unittest.skipUnless(sys.platform == "win32", "gdigrab desktop capture")
@patch("jiangchang_skill_core.rpa.video_session.shutil.which", return_value="ffmpeg")
@patch("jiangchang_skill_core.rpa.video_session.subprocess.Popen")
def test_enter_starts_ffmpeg_gdigrab(self, mock_popen: MagicMock, _which: MagicMock) -> None:
proc = MagicMock()
proc.poll.return_value = None
proc.stdin = MagicMock()
mock_popen.return_value = proc
os.environ["OPENCLAW_RECORD_VIDEO"] = "1"
try:
with tempfile.TemporaryDirectory() as tmp:
config.reset_cache()
async def _run() -> None:
async with RpaVideoSession(
skill_slug="receive-order",
skill_data_dir=tmp,
batch_id="batch_ff",
) as video:
video.add_step("测试步骤")
with patch.object(video, "finalize", return_value=None):
pass
asyncio.run(_run())
self.assertTrue(mock_popen.called)
cmd = mock_popen.call_args[0][0]
self.assertIn("ffmpeg", cmd)
self.assertIn("gdigrab", cmd)
self.assertIn("desktop", cmd)
finally:
os.environ.pop("OPENCLAW_RECORD_VIDEO", None)
config.reset_cache()
@patch("jiangchang_skill_core.rpa.video_session.shutil.which", return_value=None)
def test_ffmpeg_not_found_warning(self, _which: MagicMock) -> None:
os.environ["OPENCLAW_RECORD_VIDEO"] = "1"
try:
with tempfile.TemporaryDirectory() as tmp:
config.reset_cache()
async def _run() -> None:
async with RpaVideoSession(
skill_slug="receive-order",
skill_data_dir=tmp,
batch_id="batch_no_ff",
):
pass
asyncio.run(_run())
session = RpaVideoSession(
skill_slug="receive-order",
skill_data_dir=tmp,
batch_id="batch_no_ff",
)
session._start_ffmpeg_capture()
self.assertIn("ffmpeg_not_found", session.warnings)
finally:
os.environ.pop("OPENCLAW_RECORD_VIDEO", None)
config.reset_cache()
def test_finalize_missing_capture_warning(self) -> None:
os.environ["OPENCLAW_RECORD_VIDEO"] = "1"
try:
with tempfile.TemporaryDirectory() as tmp:
config.reset_cache()
session = RpaVideoSession(
skill_slug="receive-order",
skill_data_dir=tmp,
batch_id="batch_warn",
)
asyncio.run(session.finalize())
self.assertIn("ffmpeg_capture_missing", session.warnings)
self.assertIsNone(session.output_video_path)
summ = session.summary()
self.assertIsNone(summ["path"])
finally:
os.environ.pop("OPENCLAW_RECORD_VIDEO", None)
config.reset_cache()
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,79 @@
"""Patch pyproject.toml and __init__.py version for CI builds (not committed)."""
from __future__ import annotations
import os
import re
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
def _read_base_version() -> str:
text = (ROOT / "pyproject.toml").read_text(encoding="utf-8")
match = re.search(r'^version\s*=\s*"([^"]+)"', text, re.MULTILINE)
if not match:
raise SystemExit("ERROR: version not found in pyproject.toml")
base = match.group(1)
return re.sub(r"\.post\d+$", "", base)
def resolve_version() -> str:
ref = os.environ.get("GITHUB_REF", "")
if ref.startswith("refs/tags/v"):
return ref.removeprefix("refs/tags/v")
if ref.startswith("refs/tags/"):
return ref.removeprefix("refs/tags/")
run_number = (
os.environ.get("GITHUB_RUN_NUMBER")
or os.environ.get("GITHUB_RUN_ID")
or "0"
)
base = _read_base_version()
return f"{base}.post{run_number}"
def patch_version(version: str) -> None:
pyproject = ROOT / "pyproject.toml"
py_text = pyproject.read_text(encoding="utf-8")
py_text, n = re.subn(
r'^version\s*=\s*"[^"]*"',
f'version = "{version}"',
py_text,
count=1,
flags=re.MULTILINE,
)
if n != 1:
raise SystemExit("ERROR: failed to patch version in pyproject.toml")
pyproject.write_text(py_text, encoding="utf-8")
init_py = ROOT / "src" / "jiangchang_desktop_sdk" / "__init__.py"
init_text = init_py.read_text(encoding="utf-8")
init_text, n = re.subn(
r'^__version__\s*=\s*"[^"]*"',
f'__version__ = "{version}"',
init_text,
count=1,
flags=re.MULTILINE,
)
if n != 1:
raise SystemExit("ERROR: failed to patch __version__ in __init__.py")
init_py.write_text(init_text, encoding="utf-8")
def main() -> None:
version = resolve_version()
print(f"Resolved package version: {version}")
patch_version(version)
if os.environ.get("GITHUB_OUTPUT"):
with open(os.environ["GITHUB_OUTPUT"], "a", encoding="utf-8") as fh:
fh.write(f"package_version={version}\n")
if __name__ == "__main__":
try:
main()
except SystemExit as exc:
print(exc, file=sys.stderr)
raise

View File

@@ -32,6 +32,8 @@
- 复制 assets/
- 复制 tests/
- 复制 evals/(除 scripts 外均为明文;若目录不存在则跳过;复制时排除常见缓存与运行产物)
- 若根目录存在 requirements.txt则复制到包根明文不 PyArmor
- 若根目录存在 .env.example则复制到包根明文不 PyArmor用于首次运行时落盘用户 .env
本脚本在打 tag 前会做一次 scripts/ 结构自检,避免子目录未提交却仍发布。
#>

View File

@@ -0,0 +1,6 @@
from ._bootstrap import bootstrap_src
bootstrap_src()
from screencast import run_screencast
__all__ = ["run_screencast"]

View File

@@ -0,0 +1,15 @@
"""源码仓库内运行 tools/screencast 时,确保 src 在 sys.pathpip 安装后无需)。"""
from __future__ import annotations
import sys
from pathlib import Path
def bootstrap_src() -> None:
try:
import screencast # noqa: F401
except ImportError:
src = Path(__file__).resolve().parent.parent.parent / "src"
src_str = str(src)
if src.is_dir() and src_str not in sys.path:
sys.path.insert(0, src_str)

View File

@@ -0,0 +1,6 @@
from ._bootstrap import bootstrap_src
bootstrap_src()
from screencast.composer import compose_video
__all__ = ["compose_video"]

View File

@@ -0,0 +1,6 @@
from ._bootstrap import bootstrap_src
bootstrap_src()
from screencast.recorder import ScreenRecorder
__all__ = ["ScreenRecorder"]

View File

@@ -0,0 +1 @@
mss>=9.0.1

View File

@@ -0,0 +1,6 @@
from ._bootstrap import bootstrap_src
bootstrap_src()
from screencast.runner import run_screencast
__all__ = ["run_screencast"]

View File

@@ -0,0 +1,6 @@
from ._bootstrap import bootstrap_src
bootstrap_src()
from screencast.subtitle import SubtitleEngine
__all__ = ["SubtitleEngine"]