feat: 票据合规预审助手首版 v1.0.0(verify-invoice-default)
This commit is contained in:
152
examples/real_desktop_rpa/README.md
Normal file
152
examples/real_desktop_rpa/README.md
Normal file
@@ -0,0 +1,152 @@
|
||||
# Real Windows Desktop RPA Example
|
||||
|
||||
这是从已跑通业务技能 `D:\OpenClaw\client-marketing\add-friend-wecom` 提炼的 Windows UIA 参考案例。它用于教技术人员和 AI 编程工具如何从真实拾取结果得到稳定定位器,并完成窗口、控件、动作、等待和诊断闭环。
|
||||
|
||||
## 状态
|
||||
|
||||
- **技术路线已真实验证**:Windows UIA + Python `uiautomation`
|
||||
- **来源业务已真实跑通**:企业微信添加客户(通讯录 → 新的客户 → 添加 → 搜索 → 发送)
|
||||
- **本目录默认安全模式**:只提供 probe、dry-run、assisted 范式,不默认发送真实邀请
|
||||
- **不是完整 skill 仓库**:按 Copy map 选择性复制
|
||||
|
||||
## 先读
|
||||
|
||||
1. [`../../development/RPA_ELEMENT_LOCATING.md`](../../development/RPA_ELEMENT_LOCATING.md)
|
||||
2. [`../../development/RPA_DESKTOP.md`](../../development/RPA_DESKTOP.md)(含 Name/Value、最大化激活、同名叶子)
|
||||
3. 本 README 的“原始拾取到生产定位器”
|
||||
|
||||
## 成功案例验证了什么
|
||||
|
||||
```text
|
||||
枚举目标顶层窗口
|
||||
→ 激活并最大化主窗(勿无条件 SW_RESTORE)
|
||||
→ 从 HWND 获取 UIA root / 业务子 pane
|
||||
→ 在父容器中定位(Name 与 Value 分开看)
|
||||
→ 真实点击并校验后置状态
|
||||
→ 等待子窗口或业务结果
|
||||
→ 处理独立/嵌套弹窗
|
||||
→ probe-ui 诊断
|
||||
```
|
||||
|
||||
来源案例还提供 `dry-run / assisted / auto` 三档、频率控制和业务记忆。本 example 只提炼 UIA 元素工程;账号、风控和数据库仍由具体业务技能实现。
|
||||
|
||||
本目录 `probe()` 是**最小探针**(主窗 + 通讯录入口,可选尝试新的客户行),**不等于**生产技能完整 probe。
|
||||
|
||||
## 目录结构
|
||||
|
||||
```text
|
||||
real_desktop_rpa/
|
||||
README.md
|
||||
assets/
|
||||
element-map.sample.yaml
|
||||
selector-capture.sample.xml
|
||||
scripts/
|
||||
service/
|
||||
desktop_session.py
|
||||
uia_locator.py
|
||||
wecom_reference_flow.py
|
||||
util/
|
||||
selectors.py
|
||||
tests/
|
||||
test_selector_contract.py
|
||||
```
|
||||
|
||||
## 原始拾取到生产定位器
|
||||
|
||||
### 搜索框(添加客户窗)
|
||||
|
||||
```text
|
||||
原始:Window(SearchExternalsWnd) → 多层 Layout/index → search_layout → searchedit
|
||||
提炼:窗口 → search_layout → Name=searchedit
|
||||
```
|
||||
|
||||
### 新的客户(易错)
|
||||
|
||||
```text
|
||||
原始:通讯录 pane → wlist_view → item[2] → status_layout → main_chattext
|
||||
提炼:wlist_view 内找 子控件 main_chattext.Value=新的客户 的那一行,再点该行 status_layout
|
||||
禁止:全树第一个 Name=status_layout(常为「群聊」)
|
||||
禁止:把选择器标题「新的客户」当成 UIA Name 去搜
|
||||
```
|
||||
|
||||
### 添加按钮
|
||||
|
||||
```text
|
||||
提炼:search_external → wbtn_container(子文案 Value=添加)
|
||||
禁止:搜索旁 plus_button 下的同名 wbtn_container
|
||||
```
|
||||
|
||||
正确提炼方式:
|
||||
|
||||
1. 保留稳定顶层窗口:进程、Window Class、标题;需要时加上业务子 pane。
|
||||
2. 保留最小业务父容器。
|
||||
3. 叶子用稳定 Name **和/或** Value / 子控件 Value。
|
||||
4. 删除只反映当前布局的深层 index;index 仅作已验证兜底并写清含义。
|
||||
5. 验证父容器内候选数量为 `1`。
|
||||
6. 重启应用后再次验证。
|
||||
|
||||
不要把完整 XML 路径或未校验的 `item[0]` 原样变成生产定位器。
|
||||
|
||||
## Element map
|
||||
|
||||
本目录的 [`assets/element-map.sample.yaml`](assets/element-map.sample.yaml) 只展示来源案例的元素登记方法。复制后必须重新拾取目标软件并全部替换。
|
||||
|
||||
关键规则:
|
||||
|
||||
- 窗口不唯一时失败,不取 `hwnds[0]`。
|
||||
- 控件不唯一时失败,不取第一个匹配。
|
||||
- `item`、`status_layout`、`wbtn_container` 等通用 Name 必须放进稳定父容器,并用文案校验。
|
||||
- 行顺序不是身份;用业务文案(Value)选择结果行。
|
||||
- 主窗依赖完整侧栏时先 maximize;`activate(maximize=False)` 不得冲掉已最大化状态。
|
||||
- 只有主定位成功、窗口前台且矩形有效时,才允许从 UIA BoundingRectangle 降级坐标点击。
|
||||
|
||||
## 三种实施模式
|
||||
|
||||
| 模式 | 行为 | 用途 |
|
||||
|---|---|---|
|
||||
| `probe` | 只检查依赖、窗口和关键控件 | 安装、自检、版本变更排障 |
|
||||
| `dry-run` | 导航并填写,提交前停止 | 开发和验收默认 |
|
||||
| `assisted` | 填写后等待用户人工确认 | 不可逆动作或现场联调 |
|
||||
|
||||
业务技能若增加 `auto`,必须实现明确确认策略、频率限制、业务后置状态和 `RESULT_UNCONFIRMED`。
|
||||
|
||||
## Copy map
|
||||
|
||||
| 来源 | 目标 | 说明 |
|
||||
|---|---|---|
|
||||
| `scripts/service/desktop_session.py` | `scripts/service/desktop_session.py` | 顶层窗口枚举、唯一性、激活/最大化 |
|
||||
| `scripts/service/uia_locator.py` | `scripts/service/uia_locator.py` | 作用域内唯一定位(含 Value)、点击、输入 |
|
||||
| `scripts/util/selectors.py` | `scripts/util/<platform>_selectors.py` | element spec;必须全部替换 |
|
||||
| `scripts/service/wecom_reference_flow.py` | `scripts/service/<business>_flow.py` | 只复制流程分层,不复制企微 selector |
|
||||
| `assets/element-map.sample.yaml` | `assets/<platform>-element-map.yaml` | 真实拾取后填写 |
|
||||
| `tests/test_selector_contract.py` | `tests/test_selector_contract.py` | 定位器契约测试思路 |
|
||||
|
||||
## 必须补齐的业务能力
|
||||
|
||||
- 目标应用安装/版本 doctor
|
||||
- 真实业务元素清单
|
||||
- 失败截图、UIA树和顶层窗口快照
|
||||
- Activity/RpaVideoSession 步骤
|
||||
- 不可逆动作确认
|
||||
- 业务成功/失败/结果不明确状态
|
||||
- 频率限制、幂等和断点续跑
|
||||
- 真实集成测试(默认不进 CI)
|
||||
|
||||
## 禁止照抄
|
||||
|
||||
- `WXWork`、`WeWorkWindow`、`SearchExternalsWnd`
|
||||
- `org_op_new`、`searchedit`、`wbtn_container`、`main_chattext`
|
||||
- 企业微信列表第 0/1/2 行含义或固定 index
|
||||
- 来源案例的本机路径、账号和业务数据
|
||||
- 任何未经目标软件真实拾取验证的 fallback
|
||||
|
||||
## 验收
|
||||
|
||||
- `probe` 能明确报告依赖、窗口和关键元素(生产技能覆盖关键跳转节点)
|
||||
- 每个关键元素在目标作用域内唯一
|
||||
- 关闭重开软件后仍能定位
|
||||
- 多窗口时不会误选
|
||||
- UI变化时停止并保存截图/UIA树
|
||||
- dry-run 不执行不可逆提交
|
||||
- assisted 清楚提示用户确认
|
||||
- 提交后仅在业务后置状态成立时报告成功
|
||||
117
examples/real_desktop_rpa/assets/element-map.sample.yaml
Normal file
117
examples/real_desktop_rpa/assets/element-map.sample.yaml
Normal file
@@ -0,0 +1,117 @@
|
||||
source_case: add-friend-wecom
|
||||
status: verified_source_redacted_example
|
||||
surface: windows
|
||||
driver: uiautomation
|
||||
notes:
|
||||
- 选择器显示名 ≠ UIA Name;可见中文常在 Value / 子 Label
|
||||
- 禁止全树第一个 status_layout / wbtn_container
|
||||
- 主窗流程先 maximize;activate 勿无条件 SW_RESTORE
|
||||
elements:
|
||||
- logical_name: main_window
|
||||
purpose: 企业微信主窗口
|
||||
scope:
|
||||
process_name: WXWork
|
||||
primary_locator:
|
||||
window_class: WeWorkWindow
|
||||
title: 企业微信
|
||||
expected_count: 1
|
||||
postcondition: UIA root 可读取;业务需要时已最大化
|
||||
replace_in_new_skill: true
|
||||
|
||||
- logical_name: contacts_entry
|
||||
purpose: 打开通讯录
|
||||
scope:
|
||||
logical_parent: main_window
|
||||
primary_locator:
|
||||
name: org_op_new
|
||||
control_type: TreeItem
|
||||
value_hint: 通讯录
|
||||
expected_count: 1
|
||||
postcondition: 出现企业微信-通讯录业务 pane
|
||||
replace_in_new_skill: true
|
||||
|
||||
- logical_name: contacts_pane
|
||||
purpose: 通讯录业务子层作用域
|
||||
scope:
|
||||
logical_parent: main_window
|
||||
primary_locator:
|
||||
name_contains: 通讯录
|
||||
expected_count: 1
|
||||
postcondition: 可枚举 wlist_view
|
||||
replace_in_new_skill: true
|
||||
|
||||
- logical_name: new_customer_row
|
||||
purpose: 侧栏进入「新的客户」
|
||||
scope:
|
||||
logical_parent: contacts_pane
|
||||
ancestor_name: wlist_view
|
||||
primary_locator:
|
||||
row_leaf_name: status_layout
|
||||
child_name: main_chattext
|
||||
child_value: 新的客户
|
||||
expected_count: 1
|
||||
postcondition: panel_title_label Value=新的客户 或出现 search_external
|
||||
failure_note: 禁止点全树第一个 status_layout(常为群聊)
|
||||
replace_in_new_skill: true
|
||||
|
||||
- logical_name: panel_title
|
||||
purpose: 校验已进入新的客户页
|
||||
scope:
|
||||
logical_parent: contacts_pane
|
||||
primary_locator:
|
||||
name: panel_title_label
|
||||
value: 新的客户
|
||||
expected_count: 1
|
||||
replace_in_new_skill: true
|
||||
|
||||
- logical_name: add_customer_button
|
||||
purpose: 打开添加客户窗口
|
||||
scope:
|
||||
logical_parent: contacts_pane
|
||||
ancestor_name: search_external
|
||||
primary_locator:
|
||||
name: wbtn_container
|
||||
child_name: ww_btn_text
|
||||
child_value: 添加
|
||||
expected_count: 1
|
||||
postcondition: 出现 SearchExternalsWnd
|
||||
failure_note: 禁止点搜索旁 plus_button 下的 wbtn_container
|
||||
replace_in_new_skill: true
|
||||
|
||||
- logical_name: add_customer_window
|
||||
purpose: 添加客户窗口
|
||||
scope:
|
||||
process_name: WXWork
|
||||
primary_locator:
|
||||
window_class: SearchExternalsWnd
|
||||
title: 添加客户
|
||||
expected_count: 1
|
||||
postcondition: search_input 可定位
|
||||
replace_in_new_skill: true
|
||||
|
||||
- logical_name: search_input
|
||||
purpose: 输入待搜索账号
|
||||
scope:
|
||||
logical_parent: add_customer_window
|
||||
ancestor_name: search_layout
|
||||
primary_locator:
|
||||
name: searchedit
|
||||
expected_count: 1
|
||||
ready_condition:
|
||||
visible: true
|
||||
enabled: true
|
||||
postcondition: 结果列表或用户不存在弹窗出现
|
||||
replace_in_new_skill: true
|
||||
|
||||
- logical_name: verify_message_input
|
||||
purpose: 填写添加验证语
|
||||
scope:
|
||||
window_class: InputReasonWnd
|
||||
title_any:
|
||||
- 输入认证信息
|
||||
- 发送添加邀请
|
||||
primary_locator:
|
||||
name: name_edit
|
||||
expected_count: 1
|
||||
postcondition: 输入值可读取或发送按钮可用
|
||||
replace_in_new_skill: true
|
||||
26
examples/real_desktop_rpa/assets/selector-capture.sample.xml
Normal file
26
examples/real_desktop_rpa/assets/selector-capture.sample.xml
Normal file
@@ -0,0 +1,26 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<!--
|
||||
脱敏教学摘录:展示“拾取器长路径”而非可直接复制的生产 selector。
|
||||
生产代码应保留稳定窗口、业务父容器和叶子属性,删除无业务含义的深层 index。
|
||||
-->
|
||||
<selector name="search_input" source="inspector-export">
|
||||
<window process="WXWork" class="SearchExternalsWnd" title="添加客户" />
|
||||
<uia name="content_container" index="1" />
|
||||
<uia name="search_layout" index="0" />
|
||||
<uia name="inner_search_layout" index="1" />
|
||||
<uia name="searchedit" index="1" />
|
||||
</selector>
|
||||
|
||||
<!--
|
||||
新的客户:长路径示意 → 生产应改为
|
||||
wlist_view 内 child main_chattext.Value=新的客户 的行,再点该行 status_layout。
|
||||
禁止全树第一个 status_layout。
|
||||
-->
|
||||
<selector name="new_customer_row" source="inspector-export">
|
||||
<window process="WXWork" class="WeWorkWindow" title="企业微信" />
|
||||
<pane name="企业微信-通讯录" />
|
||||
<uia name="wlist_view" />
|
||||
<uia name="item" index="2" />
|
||||
<uia name="status_layout" />
|
||||
<uia name="main_chattext" value="新的客户" />
|
||||
</selector>
|
||||
129
examples/real_desktop_rpa/scripts/service/desktop_session.py
Normal file
129
examples/real_desktop_rpa/scripts/service/desktop_session.py
Normal file
@@ -0,0 +1,129 @@
|
||||
"""Windows 顶层窗口作用域:枚举、唯一性校验和激活。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ctypes
|
||||
import ntpath
|
||||
import time
|
||||
from ctypes import wintypes
|
||||
from dataclasses import dataclass
|
||||
from typing import Iterable
|
||||
|
||||
from util.selectors import WindowSpec
|
||||
|
||||
|
||||
class DesktopRpaError(RuntimeError):
|
||||
def __init__(self, code: str, message: str) -> None:
|
||||
super().__init__(f"ERROR:{code} {message}")
|
||||
self.code = code
|
||||
self.message = message
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class WindowCandidate:
|
||||
hwnd: int
|
||||
pid: int
|
||||
process_name: str
|
||||
class_name: str
|
||||
title: str
|
||||
visible: bool
|
||||
|
||||
|
||||
user32 = ctypes.windll.user32
|
||||
kernel32 = ctypes.windll.kernel32
|
||||
SW_RESTORE = 9
|
||||
SW_MAXIMIZE = 3
|
||||
PROCESS_QUERY_LIMITED_INFORMATION = 0x1000
|
||||
|
||||
|
||||
def _process_name(pid: int) -> str:
|
||||
handle = kernel32.OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, False, pid)
|
||||
if not handle:
|
||||
return ""
|
||||
try:
|
||||
size = wintypes.DWORD(32768)
|
||||
buffer = ctypes.create_unicode_buffer(size.value)
|
||||
if not kernel32.QueryFullProcessImageNameW(
|
||||
handle,
|
||||
0,
|
||||
buffer,
|
||||
ctypes.byref(size),
|
||||
):
|
||||
return ""
|
||||
return ntpath.splitext(ntpath.basename(buffer.value))[0]
|
||||
finally:
|
||||
kernel32.CloseHandle(handle)
|
||||
|
||||
|
||||
def enumerate_top_windows() -> list[WindowCandidate]:
|
||||
found: list[WindowCandidate] = []
|
||||
|
||||
@ctypes.WINFUNCTYPE(ctypes.c_bool, wintypes.HWND, wintypes.LPARAM)
|
||||
def callback(hwnd, _lparam):
|
||||
pid = wintypes.DWORD()
|
||||
user32.GetWindowThreadProcessId(hwnd, ctypes.byref(pid))
|
||||
cls = ctypes.create_unicode_buffer(256)
|
||||
user32.GetClassNameW(hwnd, cls, 256)
|
||||
length = user32.GetWindowTextLengthW(hwnd)
|
||||
title = ctypes.create_unicode_buffer(length + 1)
|
||||
user32.GetWindowTextW(hwnd, title, length + 1)
|
||||
found.append(
|
||||
WindowCandidate(
|
||||
hwnd=int(hwnd),
|
||||
pid=int(pid.value),
|
||||
process_name=_process_name(int(pid.value)),
|
||||
class_name=cls.value,
|
||||
title=title.value,
|
||||
visible=bool(user32.IsWindowVisible(hwnd)),
|
||||
)
|
||||
)
|
||||
return True
|
||||
|
||||
user32.EnumWindows(callback, 0)
|
||||
return found
|
||||
|
||||
|
||||
def resolve_unique_window(
|
||||
spec: WindowSpec,
|
||||
*,
|
||||
candidates: Iterable[WindowCandidate] | None = None,
|
||||
) -> WindowCandidate:
|
||||
pool = list(candidates) if candidates is not None else enumerate_top_windows()
|
||||
matched = [
|
||||
item
|
||||
for item in pool
|
||||
if item.visible
|
||||
and item.process_name.casefold() == spec.process_name.casefold()
|
||||
and item.class_name == spec.class_name
|
||||
and (spec.title is None or item.title == spec.title)
|
||||
]
|
||||
if not matched:
|
||||
raise DesktopRpaError("WINDOW_NOT_FOUND", f"{spec.logical_name} 未找到")
|
||||
if len(matched) != 1:
|
||||
raise DesktopRpaError(
|
||||
"ELEMENT_AMBIGUOUS",
|
||||
f"{spec.logical_name} 匹配 {len(matched)} 个窗口,请按 PID/账号补充作用域",
|
||||
)
|
||||
return matched[0]
|
||||
|
||||
|
||||
def activate_window(window: WindowCandidate, *, maximize: bool = False) -> None:
|
||||
"""激活窗口。
|
||||
|
||||
maximize=True:主窗进入依赖完整侧栏的流程时使用。
|
||||
maximize=False:只前置;**禁止**对已最大化窗口无条件 SW_RESTORE。
|
||||
"""
|
||||
hwnd = window.hwnd
|
||||
iconic = bool(user32.IsIconic(hwnd))
|
||||
zoomed = bool(user32.IsZoomed(hwnd))
|
||||
if maximize:
|
||||
if iconic:
|
||||
user32.ShowWindow(hwnd, SW_RESTORE)
|
||||
if not zoomed:
|
||||
user32.ShowWindow(hwnd, SW_MAXIMIZE)
|
||||
else:
|
||||
if iconic:
|
||||
user32.ShowWindow(hwnd, SW_RESTORE)
|
||||
if not user32.SetForegroundWindow(hwnd):
|
||||
raise DesktopRpaError("WINDOW_ACTIVATE_FAILED", f"无法激活 hwnd={hwnd}")
|
||||
time.sleep(0.35 if maximize else 0.2)
|
||||
127
examples/real_desktop_rpa/scripts/service/uia_locator.py
Normal file
127
examples/real_desktop_rpa/scripts/service/uia_locator.py
Normal file
@@ -0,0 +1,127 @@
|
||||
"""作用域内 UIA 唯一定位、真实点击和逐字输入。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import random
|
||||
from typing import Any, Iterator
|
||||
|
||||
from service.desktop_session import DesktopRpaError, WindowCandidate, activate_window
|
||||
from util.selectors import ElementSpec
|
||||
|
||||
|
||||
def require_uiautomation():
|
||||
try:
|
||||
import uiautomation as auto
|
||||
except ImportError as exc:
|
||||
raise DesktopRpaError("UIA_UNAVAILABLE", "缺少 uiautomation 依赖") from exc
|
||||
return auto
|
||||
|
||||
|
||||
def root_from_window(window: WindowCandidate):
|
||||
auto = require_uiautomation()
|
||||
return auto.ControlFromHandle(window.hwnd)
|
||||
|
||||
|
||||
def control_value(control: Any) -> str:
|
||||
try:
|
||||
auto = require_uiautomation()
|
||||
raw = control.GetPropertyValue(auto.PropertyId.ValueValueProperty)
|
||||
return str(raw or "").strip()
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
|
||||
def _walk(root: Any, *, depth: int = 0, max_depth: int = 25) -> Iterator[Any]:
|
||||
if depth > max_depth:
|
||||
return
|
||||
yield root
|
||||
if depth == max_depth:
|
||||
return
|
||||
try:
|
||||
children = root.GetChildren()
|
||||
except Exception:
|
||||
children = []
|
||||
for child in children:
|
||||
yield from _walk(child, depth=depth + 1, max_depth=max_depth)
|
||||
|
||||
|
||||
def _child_has_value(control: Any, child_name: str, expected_value: str, *, max_depth: int = 10) -> bool:
|
||||
for node in _walk(control, max_depth=max_depth):
|
||||
if (getattr(node, "Name", None) or "") != child_name:
|
||||
continue
|
||||
if control_value(node) == expected_value:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _matches(control: Any, spec: ElementSpec) -> bool:
|
||||
if spec.name is not None and (getattr(control, "Name", None) or "") != spec.name:
|
||||
return False
|
||||
if spec.value is not None and not spec.child_value_name:
|
||||
if control_value(control) != spec.value:
|
||||
return False
|
||||
if spec.automation_id is not None and (
|
||||
getattr(control, "AutomationId", None) or ""
|
||||
) != spec.automation_id:
|
||||
return False
|
||||
if spec.control_type is not None and (
|
||||
(getattr(control, "ControlTypeName", None) or "").removesuffix("Control")
|
||||
!= spec.control_type.removesuffix("Control")
|
||||
):
|
||||
return False
|
||||
if spec.class_name is not None and (
|
||||
getattr(control, "ClassName", None) or ""
|
||||
) != spec.class_name:
|
||||
return False
|
||||
if spec.child_value_name and spec.value:
|
||||
if not _child_has_value(control, spec.child_value_name, spec.value):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def locate_unique(root: Any, spec: ElementSpec, *, max_depth: int = 25):
|
||||
search_root = root
|
||||
if spec.ancestor_name:
|
||||
ancestor_spec = ElementSpec(
|
||||
logical_name=f"{spec.logical_name}.ancestor",
|
||||
name=spec.ancestor_name,
|
||||
)
|
||||
search_root = locate_unique(root, ancestor_spec, max_depth=max_depth)
|
||||
|
||||
matched = [
|
||||
control for control in _walk(search_root, max_depth=max_depth) if _matches(control, spec)
|
||||
]
|
||||
if not matched:
|
||||
raise DesktopRpaError("ELEMENT_NOT_FOUND", f"{spec.logical_name} 未找到")
|
||||
if len(matched) != spec.expected_count:
|
||||
raise DesktopRpaError(
|
||||
"ELEMENT_AMBIGUOUS",
|
||||
f"{spec.logical_name} 期望 {spec.expected_count} 个,实际 {len(matched)} 个",
|
||||
)
|
||||
return matched[0]
|
||||
|
||||
|
||||
def click_control(
|
||||
control: Any,
|
||||
*,
|
||||
window: WindowCandidate,
|
||||
maximize: bool = False,
|
||||
) -> None:
|
||||
activate_window(window, maximize=maximize)
|
||||
rect = control.BoundingRectangle
|
||||
if rect.width() <= 0 or rect.height() <= 0 or rect.left < -10000:
|
||||
raise DesktopRpaError("ELEMENT_NOT_READY", "目标元素不可见或边界矩形无效")
|
||||
control.Click(
|
||||
x=random.randint(-2, 2),
|
||||
y=random.randint(-2, 2),
|
||||
simulateMove=True,
|
||||
)
|
||||
|
||||
|
||||
def type_text(control: Any, text: str, *, window: WindowCandidate) -> None:
|
||||
auto = require_uiautomation()
|
||||
click_control(control, window=window, maximize=False)
|
||||
auto.SendKeys("{Ctrl}a", waitTime=0.05)
|
||||
auto.SendKeys("{Delete}", waitTime=0.05)
|
||||
for char in text:
|
||||
auto.SendKeys(char, waitTime=random.uniform(0.04, 0.12))
|
||||
@@ -0,0 +1,58 @@
|
||||
"""企业微信成功案例的安全提炼:probe 和 dry-run,不默认发送邀请。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import asdict
|
||||
from typing import Any
|
||||
|
||||
from service.desktop_session import activate_window, resolve_unique_window
|
||||
from service.uia_locator import locate_unique, root_from_window, type_text
|
||||
from util.selectors import (
|
||||
ADD_CUSTOMER_WINDOW,
|
||||
CONTACTS_ENTRY,
|
||||
MAIN_WINDOW,
|
||||
NEW_CUSTOMER_ROW,
|
||||
SEARCH_INPUT,
|
||||
)
|
||||
|
||||
|
||||
def probe() -> dict[str, Any]:
|
||||
"""最小 probe:主窗 + 通讯录入口。
|
||||
|
||||
生产技能应对「新的客户 / 添加」等关键节点做更深探测;本函数不等于生产完整度。
|
||||
调用前会 maximize 主窗,避免侧栏折叠导致 org_op_new 抖动。
|
||||
"""
|
||||
main = resolve_unique_window(MAIN_WINDOW)
|
||||
activate_window(main, maximize=True)
|
||||
root = root_from_window(main)
|
||||
contacts = locate_unique(root, CONTACTS_ENTRY)
|
||||
payload: dict[str, Any] = {
|
||||
"success": True,
|
||||
"minimized_probe": True,
|
||||
"main_window": asdict(main),
|
||||
"contacts_entry": {
|
||||
"name": getattr(contacts, "Name", ""),
|
||||
"control_type": getattr(contacts, "ControlTypeName", ""),
|
||||
},
|
||||
}
|
||||
# 可选:若已在通讯录页,尝试按 Value 定位「新的客户」行(失败不视为 probe 失败)
|
||||
try:
|
||||
row = locate_unique(root, NEW_CUSTOMER_ROW)
|
||||
payload["new_customer_row"] = {"name": getattr(row, "Name", "")}
|
||||
except Exception:
|
||||
payload["new_customer_row"] = None
|
||||
return payload
|
||||
|
||||
|
||||
def fill_search_dry_run(target: str) -> dict[str, Any]:
|
||||
"""要求操作者已打开“添加客户”窗口;只填写搜索框,不提交。"""
|
||||
search_window = resolve_unique_window(ADD_CUSTOMER_WINDOW)
|
||||
activate_window(search_window, maximize=False)
|
||||
root = root_from_window(search_window)
|
||||
search_input = locate_unique(root, SEARCH_INPUT)
|
||||
type_text(search_input, target, window=search_window)
|
||||
return {
|
||||
"success": True,
|
||||
"mode": "dry-run",
|
||||
"message": "已填写搜索条件,未发送邀请",
|
||||
}
|
||||
63
examples/real_desktop_rpa/scripts/util/selectors.py
Normal file
63
examples/real_desktop_rpa/scripts/util/selectors.py
Normal file
@@ -0,0 +1,63 @@
|
||||
"""脱敏 UIA element specs;复制到新技能后必须用真实拾取结果替换。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class WindowSpec:
|
||||
logical_name: str
|
||||
process_name: str
|
||||
class_name: str
|
||||
title: Optional[str] = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ElementSpec:
|
||||
logical_name: str
|
||||
name: Optional[str] = None
|
||||
"""UIA Name(内部 id,未必是界面中文)。"""
|
||||
value: Optional[str] = None
|
||||
"""UIA Value(可见中文文案常在此)。"""
|
||||
automation_id: Optional[str] = None
|
||||
control_type: Optional[str] = None
|
||||
class_name: Optional[str] = None
|
||||
ancestor_name: Optional[str] = None
|
||||
"""先定位到该 Name 的祖先,再在其下搜索。"""
|
||||
child_value_name: Optional[str] = None
|
||||
"""在候选控件子树中查找该 Name 的子控件,并用 value 匹配其 Value。"""
|
||||
expected_count: int = 1
|
||||
|
||||
|
||||
MAIN_WINDOW = WindowSpec("main_window", "WXWork", "WeWorkWindow", "企业微信")
|
||||
ADD_CUSTOMER_WINDOW = WindowSpec(
|
||||
"add_customer_window",
|
||||
"WXWork",
|
||||
"SearchExternalsWnd",
|
||||
"添加客户",
|
||||
)
|
||||
CONTACTS_ENTRY = ElementSpec(
|
||||
"contacts_entry",
|
||||
name="org_op_new",
|
||||
control_type="TreeItem",
|
||||
)
|
||||
# 示意:用子控件 main_chattext 的 Value 定位「新的客户」行(禁止全树第一个 status_layout)
|
||||
NEW_CUSTOMER_ROW = ElementSpec(
|
||||
"new_customer_row",
|
||||
name="status_layout",
|
||||
ancestor_name="wlist_view",
|
||||
child_value_name="main_chattext",
|
||||
value="新的客户",
|
||||
)
|
||||
ADD_BUTTON = ElementSpec(
|
||||
"add_customer_button",
|
||||
name="wbtn_container",
|
||||
ancestor_name="search_external",
|
||||
)
|
||||
SEARCH_INPUT = ElementSpec(
|
||||
"search_input",
|
||||
name="searchedit",
|
||||
ancestor_name="search_layout",
|
||||
)
|
||||
99
examples/real_desktop_rpa/tests/test_selector_contract.py
Normal file
99
examples/real_desktop_rpa/tests/test_selector_contract.py
Normal file
@@ -0,0 +1,99 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "scripts"))
|
||||
|
||||
from service.desktop_session import ( # noqa: E402
|
||||
DesktopRpaError,
|
||||
WindowCandidate,
|
||||
activate_window,
|
||||
resolve_unique_window,
|
||||
)
|
||||
from service import uia_locator # noqa: E402
|
||||
from util.selectors import ElementSpec, MAIN_WINDOW # noqa: E402
|
||||
|
||||
|
||||
class SelectorContractTest(unittest.TestCase):
|
||||
def test_unique_window_is_required(self) -> None:
|
||||
candidate = WindowCandidate(1, 10, "WXWork", "WeWorkWindow", "企业微信", True)
|
||||
self.assertEqual(
|
||||
resolve_unique_window(MAIN_WINDOW, candidates=[candidate]),
|
||||
candidate,
|
||||
)
|
||||
|
||||
def test_duplicate_windows_fail_instead_of_taking_first(self) -> None:
|
||||
candidates = [
|
||||
WindowCandidate(1, 10, "WXWork", "WeWorkWindow", "企业微信", True),
|
||||
WindowCandidate(2, 11, "WXWork", "WeWorkWindow", "企业微信", True),
|
||||
]
|
||||
with self.assertRaises(DesktopRpaError) as ctx:
|
||||
resolve_unique_window(MAIN_WINDOW, candidates=candidates)
|
||||
self.assertEqual(ctx.exception.code, "ELEMENT_AMBIGUOUS")
|
||||
|
||||
def test_hidden_window_is_not_a_candidate(self) -> None:
|
||||
hidden = WindowCandidate(1, 10, "WXWork", "WeWorkWindow", "企业微信", False)
|
||||
with self.assertRaises(DesktopRpaError) as ctx:
|
||||
resolve_unique_window(MAIN_WINDOW, candidates=[hidden])
|
||||
self.assertEqual(ctx.exception.code, "WINDOW_NOT_FOUND")
|
||||
|
||||
def test_activate_without_maximize_skips_restore_when_zoomed(self) -> None:
|
||||
window = WindowCandidate(42, 10, "WXWork", "WeWorkWindow", "企业微信", True)
|
||||
with patch("service.desktop_session.user32") as user32:
|
||||
user32.IsIconic.return_value = False
|
||||
user32.IsZoomed.return_value = True
|
||||
user32.SetForegroundWindow.return_value = True
|
||||
activate_window(window, maximize=False)
|
||||
user32.ShowWindow.assert_not_called()
|
||||
user32.SetForegroundWindow.assert_called_once_with(42)
|
||||
|
||||
def test_activate_maximize_when_not_zoomed(self) -> None:
|
||||
window = WindowCandidate(42, 10, "WXWork", "WeWorkWindow", "企业微信", True)
|
||||
with patch("service.desktop_session.user32") as user32:
|
||||
user32.IsIconic.return_value = False
|
||||
user32.IsZoomed.return_value = False
|
||||
user32.SetForegroundWindow.return_value = True
|
||||
activate_window(window, maximize=True)
|
||||
user32.ShowWindow.assert_called()
|
||||
user32.SetForegroundWindow.assert_called_once_with(42)
|
||||
|
||||
def test_locate_unique_matches_child_value(self) -> None:
|
||||
row = MagicMock()
|
||||
row.Name = "status_layout"
|
||||
row.AutomationId = ""
|
||||
row.ControlTypeName = "TreeControl"
|
||||
row.ClassName = ""
|
||||
label = MagicMock()
|
||||
label.Name = "main_chattext"
|
||||
label.GetChildren.return_value = []
|
||||
row.GetChildren.return_value = [label]
|
||||
|
||||
root = MagicMock()
|
||||
root.Name = "wlist_view"
|
||||
root.AutomationId = ""
|
||||
root.ControlTypeName = "TreeControl"
|
||||
root.ClassName = ""
|
||||
root.GetChildren.return_value = [row]
|
||||
|
||||
with patch.object(uia_locator, "control_value", side_effect=lambda c: "新的客户" if c is label else ""):
|
||||
spec = ElementSpec(
|
||||
"new_customer_row",
|
||||
name="status_layout",
|
||||
ancestor_name="wlist_view",
|
||||
child_value_name="main_chattext",
|
||||
value="新的客户",
|
||||
)
|
||||
# ancestor is root itself when searching from root with name wlist_view
|
||||
found = uia_locator.locate_unique(root, ElementSpec("wlist", name="wlist_view"))
|
||||
self.assertIs(found, root)
|
||||
found_row = uia_locator.locate_unique(root, spec)
|
||||
self.assertIs(found_row, row)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user