feat: add three-surface RPA locator standards and examples

examples/README.md

@@ -8,14 +8,16 @@
 
 根 `README.md`、`SKILL.md`、`references/`、`development/` 仍按模板主目录规则编写，**不要**从 `examples/` 复制。
 
-## 四类外部交互方式
+## 外部交互示例矩阵
 
-一级目录固定为四种模式（不可删除、不可移动）：
+adapter 仍是 `真实/仿真 × API/RPA` 四档；RPA 再按浏览器、Windows 桌面和手机三种界面拆分。原四类目录不可删除、不可移动，桌面和手机目录用于补足界面维度。
 
 | 目录 | 类型 | 状态 | 适用场景 |
 |---|---|---|---|
 | `real_browser_rpa` | 真实浏览器 RPA | **已有案例** | 真实第三方网站、登录态、人工验证、滚动采集 |
 | `simulator_browser_rpa` | 仿真浏览器 RPA | **已有案例** | 自有 sandbox、可控页面、表单 RPA、adapter 分层 |
+| `real_desktop_rpa` | 真实 Windows 桌面 RPA | **已有成功来源并已提炼** | UIA 拾取、窗口/控件、probe、dry-run/assisted |
+| `mobile_rpa` | Android/iOS RPA | **仅规格，待成功案例** | 设备、控件树、Native/WebView、example 完成契约 |
 | `real_api` | 真实 API | **占位** | 真实系统接口、token、权限、限流（尚未沉淀可复制实现） |
 | `simulator_api` | 仿真 API | **占位** | mock/sandbox API、无需浏览器（尚未沉淀可复制实现） |
 
@@ -44,6 +46,8 @@
 |---|---|
 | 真实网站 + 浏览器 + 登录/验证码/滚动 | **real_browser_rpa** |
 | 仿真页面 + 浏览器 + 表单/批量提交 | **simulator_browser_rpa** |
+| Windows 原生软件 + UIA | **real_desktop_rpa** |
+| Android/iOS App | **mobile_rpa**（当前仅规格，不可作为成功实现参考） |
 | 真实系统 API | **real_api**（当前占位，不可作为实现参考） |
 | mock / sandbox API（无浏览器） | **simulator_api**（当前占位，不可作为实现参考） |
 
@@ -60,7 +64,8 @@ examples/<mode>/
   tests/
 ```
 
-- 仅 `simulator_browser_rpa` 可额外包含 `sandbox/`
+- `simulator_browser_rpa` 可额外包含 `sandbox/`
+- `real_desktop_rpa` 可额外包含脱敏的 `assets/element-map` 与 selector capture
 - 仅未来确实需要样例数据时可加 `fixtures/`
 - **不要**在 `examples/<mode>/` 下要求或添加：`SKILL.md`、根市场 `README.md`、`references/`、`development/`、`assets/`、`.github/`、release workflow 等完整 skill 仓库层文件
 
@@ -76,6 +81,8 @@ examples/<mode>/
 
 - 真实浏览器 RPA → 参考 `examples/real_browser_rpa/`
 - 仿真浏览器 RPA → 参考 `examples/simulator_browser_rpa/`（adapter 分层权威参考：`scripts/service/adapter/`）
+- Windows 桌面 RPA → 参考 `examples/real_desktop_rpa/`（真实成功来源：`add-friend-wecom`）
+- 手机 RPA → 先按 `examples/mobile_rpa/` 定义完成标准；真实成功案例补齐前不得照骨架宣称跑通
 - `real_api/`、`simulator_api/` 当前为规划占位，**不得**作为实现参考
 
 > 示例提供的是**参考架构与边界**，不是业务代码原样复制。复制前先读对应 README 的 copy map 与「禁止照抄」章节。

examples/mobile_rpa/README.md

@@ -0,0 +1,94 @@
+# Mobile RPA Example Specification
+
+本目录定义 Android/iOS example 必须具备的结构和验收契约。**当前没有团队真实成功案例，因此本目录不是可复制实现，也不得宣称已跑通。**
+
+先读：
+
+1. [`../../development/RPA_ELEMENT_LOCATING.md`](../../development/RPA_ELEMENT_LOCATING.md)
+2. [`../../development/RPA_MOBILE.md`](../../development/RPA_MOBILE.md)
+
+## 为什么现在不放伪代码实现
+
+移动端 selector、driver、Activity/context、权限弹窗和设备兼容性必须在真实授权设备上验证。没有成功案例时放一套看似完整的代码，会诱导开发者和 AI 猜 `resource-id`、默认取第一个设备或用坐标硬点，违背模板目标。
+
+## 未来成功 example 的固定结构
+
+```text
+mobile_rpa/
+  README.md
+  android/
+    README.md
+    assets/element-map.yaml
+    scripts/service/device_session.py
+    scripts/service/mobile_locator.py
+    scripts/service/example_flow.py
+    tests/test_locator_contract.py
+    tests/integration/test_real_device.py.sample
+  ios/
+    README.md
+    assets/element-map.yaml
+    scripts/service/device_session.py
+    scripts/service/mobile_locator.py
+    scripts/service/example_flow.py
+    tests/test_locator_contract.py
+    tests/integration/test_real_device.py.sample
+```
+
+只有对应端出现真实成功案例后才创建代码文件。
+
+## Android example 完成标准
+
+- 唯一设备 serial 和设备互斥
+- doctor：ADB授权、包名、Activity、屏幕状态、driver
+- Appium Inspector/UIAutoDev 拾取步骤和原始 XML
+- element map：resource-id/content-desc/text/class/父容器
+- Native/WebView context 选择
+- 点击、输入、滑动和真实后置状态
+- 权限弹窗、系统弹窗和人工验证
+- screenshot + page source 失败留证
+- 虚拟列表滚动后重新定位
+- 至少一台授权真机完整跑通
+
+## iOS example 完成标准
+
+- Mac、WebDriverAgent签名和设备 UDID 准备
+- doctor：WDA、bundle id、系统版本、屏幕状态
+- Appium Inspector 拾取和原始 page source
+- accessibility id/name、predicate/class chain 定位
+- Native/WebView context 选择
+- 系统 alert 和人工验证
+- screenshot + page source 失败留证
+- 至少一台授权真机完整跑通
+
+## element map 最小字段
+
+```yaml
+logical_name: login_button
+surface: android
+screen: 登录页
+scope:
+  package: com.example.app
+  activity: .LoginActivity
+primary_locator:
+  strategy: resource_id
+  value: com.example.app:id/login
+expected_count: 1
+ready_condition:
+  visible: true
+  enabled: true
+postcondition:
+  description: 首页业务标记出现
+verified_device: 待填写
+verified_app_version: 待填写
+```
+
+## 状态升级规则
+
+满足以下全部条件后，才可把本目录状态改为“已有成功案例”：
+
+1. 提供用户确认的成功项目来源。
+2. 在真实授权设备上跑通完整业务闭环。
+3. 保存脱敏 element map 和失败证据样例。
+4. 有默认离线单测和显式真实集成测试。
+5. 经过设备重连、App重启和页面变化验证。
+6. 用户人工验收通过。

examples/real_browser_rpa/README.md

@@ -33,6 +33,8 @@ real_browser_rpa/
 │   └── util/
 │       ├── constants.py           # 平台常量、selector 占位
 │       └── logging.py             # 结构化日志与脱敏
+├── assets/
+│   └── element-map.sample.yaml    # 元素清单模板；真实拾取后填写
 └── tests/
     ├── test_human_verification.py # 验证码误判防护
     └── test_stop_reason.py        # 停止原因文案
@@ -55,12 +57,13 @@ real_browser_rpa/
 ## 开发新 skill 时如何复制
 
 1. **先阅读**本 README 及下方「Copy map」「复制边界」章节
-2. 按 copy map **选择性复制**到新技能 `scripts/`（**不要**整包照搬 `examples/real_browser_rpa/`）
-3. 修改 `util/constants.py` 中的 `TARGET_PLATFORM`、`DEFAULT_START_URL`、selector 常量
-4. 替换 `task_rpa.py` 中的登录态 marker、搜索框/结果解析逻辑（**必须 F12 实测**）
-5. 替换 `human_verification.py` 中的验证码 selector（若目标站点 DOM 不同）
-6. 在 `task_service.py` 中接入真实 DB、任务日志、RpaVideoSession
-7. 在新技能根目录补齐 `scripts/main.py` CLI；`SKILL.md` 与根 `README.md` 按模板主目录规则编写，**不要**复制本 README
+2. **先完成元素工程**：阅读 `development/RPA_ELEMENT_LOCATING.md` 与 `development/RPA_BROWSER.md`，在真实站用 Pick Locator/F12 拾取并填写 element map
+3. 按 copy map **选择性复制**到新技能 `scripts/`（**不要**整包照搬 `examples/real_browser_rpa/`）
+4. 修改 `util/constants.py` 中的 `TARGET_PLATFORM`、`DEFAULT_START_URL`、selector 常量
+5. 替换 `task_rpa.py` 中的登录态 marker、搜索框/结果解析逻辑（**必须 F12 实测**）
+6. 替换 `human_verification.py` 中的验证码 selector（若目标站点 DOM 不同）
+7. 在 `task_service.py` 中接入真实 DB、任务日志、RpaVideoSession
+8. 在新技能根目录补齐 `scripts/main.py` CLI；`SKILL.md` 与根 `README.md` 按模板主目录规则编写，**不要**复制本 README
 
 ## Copy map（复制到新技能 `scripts/`）
 
@@ -73,6 +76,7 @@ real_browser_rpa/
 | `examples/real_browser_rpa/scripts/service/task_service.py` | `scripts/service/task_service.py` | 参考并合并编排层 |
 | `examples/real_browser_rpa/scripts/util/constants.py` | `scripts/util/constants.py` | 平台常量与 selector（须替换） |
 | `examples/real_browser_rpa/scripts/util/logging.py` | `scripts/util/logging.py` | 可选，按需合并 |
+| `examples/real_browser_rpa/assets/element-map.sample.yaml` | `assets/<platform>-element-map.yaml` | 真实 Pick Locator/F12 拾取后填写 |
 | `examples/real_browser_rpa/tests/` | `tests/` | 可按需复制测试思路 |
 | `examples/real_browser_rpa/README.md` | — | **只读参考**，不复制为新技能根 README |
 
@@ -121,6 +125,8 @@ real_browser_rpa/
 - 不要照抄平台名、slug、logger 名
 - 不要依赖 `D:\OpenClaw\client-commons\account-manager\scripts\main.py` 等开发机绝对路径作为生产依赖
 - 不要把示例里的 result selector 当成真实站点 selector
+- 不要复制“扫描所有 input 再按宽度/位置打分”的诊断式猜测作为生产主定位器
+- 不要用 `.first` / `.nth` 绕过多匹配；关键动作必须先解决唯一性
 - 不要把示例代码原样发布为业务 skill
 - 不要把真实账号、手机号、cookie、token 写入代码或 README
 

examples/real_browser_rpa/assets/element-map.sample.yaml

@@ -0,0 +1,45 @@
+surface: browser
+status: placeholder_requires_real_site_inspection
+elements:
+  - logical_name: search_input
+    purpose: 输入搜索关键词
+    scope:
+      kind: main
+      business_container: 待真实拾取
+    primary_locator:
+      strategy: role_or_label
+      value: 待真实拾取
+    fallback_locators: []
+    expected_count: 1
+    ready_condition:
+      visible: true
+      enabled: true
+    postcondition:
+      description: 搜索结果容器出现
+    verified_target_version: 待填写
+
+  - logical_name: search_button
+    purpose: 提交搜索
+    scope:
+      logical_parent: search_form
+    primary_locator:
+      strategy: role
+      value: button,name=搜索
+    fallback_locators: []
+    expected_count: 1
+    postcondition:
+      description: 结果页或业务错误状态出现
+    verified_target_version: 待填写
+
+  - logical_name: result_container
+    purpose: 承载搜索结果
+    scope:
+      kind: main
+    primary_locator:
+      strategy: stable_business_container
+      value: 待真实拾取
+    fallback_locators: []
+    expected_count: 1
+    postcondition:
+      description: 列表项、空结果或结束标记可区分
+    verified_target_version: 待填写

examples/real_browser_rpa/scripts/service/task_rpa.py

@@ -188,42 +188,22 @@ async def _ensure_logged_in(page, *, wait_sec: int) -> None:
 async def _find_search_input(page) -> Tuple[Optional[Any], str]:
     for sel in SEARCH_INPUT_SELECTORS:
         try:
-            loc = page.locator(sel).first
-            if await _visible(loc):
+            loc = page.locator(sel)
+            count = await loc.count()
+            if count == 1 and await _visible(loc):
                 return loc, sel
+            if count > 1:
+                logger.warning("search_input_ambiguous selector=%s count=%s", sel, count)
         except Exception:
             continue
 
-    candidates: List[Tuple[int, Any, str]] = []
-    try:
-        inputs = await page.locator("input").all()
-    except Exception:
-        inputs = []
-    for inp in inputs:
-        try:
-            if not await inp.is_visible():
-                continue
-            box = await inp.bounding_box()
-            if not box or box.get("width", 0) < 120:
-                continue
-            placeholder = await inp.get_attribute("placeholder") or ""
-            score = box.get("width", 0)
-            if "搜索" in placeholder:
-                score += 500
-            if box.get("y", 9999) < 200:
-                score += 200
-            candidates.append((score, inp, f"input placeholder={placeholder!r}"))
-        except Exception:
-            continue
-    if candidates:
-        candidates.sort(key=lambda x: x[0], reverse=True)
-        _, inp, sel = candidates[0]
-        return inp, sel
-
     try:
         ph = page.get_by_placeholder(re.compile("搜索"))
-        if await ph.count() > 0 and await ph.first.is_visible():
-            return ph.first, "placeholder=搜索"
+        count = await ph.count()
+        if count == 1 and await ph.is_visible():
+            return ph, "placeholder=搜索"
+        if count > 1:
+            logger.warning("search_input_ambiguous selector=placeholder_search count=%s", count)
     except Exception:
         pass
     return None, ""
@@ -232,14 +212,24 @@ async def _find_search_input(page) -> Tuple[Optional[Any], str]:
 async def _find_search_button(page):
     try:
         btn = page.get_by_role("button", name="搜索")
-        if await btn.count() > 0 and await btn.first.is_visible():
-            return btn.first
+        count = await btn.count()
+        if count == 1 and await btn.is_visible():
+            return btn
+        if count > 1:
+            logger.warning("search_button_ambiguous selector=role_button_search count=%s", count)
     except Exception:
         pass
     try:
-        btn = page.locator(SEARCH_BUTTON_SELECTOR).first
-        if await btn.count() > 0 and await btn.is_visible():
+        btn = page.locator(SEARCH_BUTTON_SELECTOR)
+        count = await btn.count()
+        if count == 1 and await btn.is_visible():
             return btn
+        if count > 1:
+            logger.warning(
+                "search_button_ambiguous selector=%s count=%s",
+                SEARCH_BUTTON_SELECTOR,
+                count,
+            )
     except Exception:
         pass
     return None

examples/real_desktop_rpa/README.md

@@ -0,0 +1,135 @@
+# Real Windows Desktop RPA Example
+
+这是从已跑通业务技能 `D:\OpenClaw\client-marketing\add-friend-wecom` 提炼的 Windows UIA 参考案例。它用于教技术人员和 AI 编程工具如何从真实拾取结果得到稳定定位器，并完成窗口、控件、动作、等待和诊断闭环。
+
+## 状态
+
+- **技术路线已真实验证**：Windows UIA + Python `uiautomation`
+- **来源业务已真实跑通**：企业微信添加客户
+- **本目录默认安全模式**：只提供 probe、dry-run、assisted 范式，不默认发送真实邀请
+- **不是完整 skill 仓库**：按 Copy map 选择性复制
+
+## 先读
+
+1. [`../../development/RPA_ELEMENT_LOCATING.md`](../../development/RPA_ELEMENT_LOCATING.md)
+2. [`../../development/RPA_DESKTOP.md`](../../development/RPA_DESKTOP.md)
+3. 本 README 的“原始拾取到生产定位器”
+
+## 成功案例验证了什么
+
+```text
+枚举目标顶层窗口
+→ 按进程/窗口类/标题确认作用域
+→ 从 HWND 获取 UIA root
+→ 在父容器中定位元素
+→ 激活窗口并真实点击/输入
+→ 等待子窗口或业务结果
+→ 处理独立/嵌套弹窗
+→ probe-ui 诊断
+```
+
+来源案例还提供 `dry-run / assisted / auto` 三档、频率控制和业务记忆。本 example 只提炼 UIA 元素工程；账号、风控和数据库仍由具体业务技能实现。
+
+## 目录结构
+
+```text
+real_desktop_rpa/
+  README.md
+  assets/
+    element-map.sample.yaml
+    selector-capture.sample.xml
+  scripts/
+    service/
+      desktop_session.py
+      uia_locator.py
+      wecom_reference_flow.py
+    util/
+      selectors.py
+  tests/
+    test_selector_contract.py
+```
+
+## 原始拾取到生产定位器
+
+来源案例的影刀 `selectorsV2` 导出通常包含：
+
+```text
+Window(class=SearchExternalsWnd,title=添加客户)
+  → 多层 Layout/index
+  → search_layout
+  → searchedit
+```
+
+正确提炼方式：
+
+1. 保留稳定顶层窗口：进程、Window Class、标题。
+2. 保留最小业务父容器：如 `search_layout`。
+3. 使用叶子元素稳定属性：如 `Name=searchedit`。
+4. 删除只反映当前布局的深层 `index`。
+5. 验证父容器内候选数量为 `1`。
+6. 重启应用后再次验证。
+
+不要把完整 XML 路径或 `item[0]` 原样变成生产定位器。
+
+## Element map
+
+本目录的 [`assets/element-map.sample.yaml`](assets/element-map.sample.yaml) 只展示来源案例的元素登记方法。复制后必须重新拾取目标软件并全部替换。
+
+关键规则：
+
+- 窗口不唯一时失败，不取 `hwnds[0]`。
+- 控件不唯一时失败，不取第一个匹配。
+- `item`、`status_layout`、`wbtn_container` 等通用 Name 必须放进稳定父容器。
+- 行顺序不是身份；用业务文案或稳定属性选择结果行。
+- 只有主定位成功、窗口前台且矩形有效时，才允许从 UIA BoundingRectangle 降级坐标点击。
+
+## 三种实施模式
+
+| 模式 | 行为 | 用途 |
+|---|---|---|
+| `probe` | 只检查依赖、窗口和关键控件 | 安装、自检、版本变更排障 |
+| `dry-run` | 导航并填写，提交前停止 | 开发和验收默认 |
+| `assisted` | 填写后等待用户人工确认 | 不可逆动作或现场联调 |
+
+业务技能若增加 `auto`，必须实现明确确认策略、频率限制、业务后置状态和 `RESULT_UNCONFIRMED`。
+
+## Copy map
+
+| 来源 | 目标 | 说明 |
+|---|---|---|
+| `scripts/service/desktop_session.py` | `scripts/service/desktop_session.py` | 顶层窗口枚举、唯一性、激活 |
+| `scripts/service/uia_locator.py` | `scripts/service/uia_locator.py` | 作用域内唯一定位、点击、输入 |
+| `scripts/util/selectors.py` | `scripts/util/<platform>_selectors.py` | element spec；必须全部替换 |
+| `scripts/service/wecom_reference_flow.py` | `scripts/service/<business>_flow.py` | 只复制流程分层，不复制企微 selector |
+| `assets/element-map.sample.yaml` | `assets/<platform>-element-map.yaml` | 真实拾取后填写 |
+| `tests/test_selector_contract.py` | `tests/test_selector_contract.py` | 定位器契约测试思路 |
+
+## 必须补齐的业务能力
+
+- 目标应用安装/版本 doctor
+- 真实业务元素清单
+- 失败截图、UIA树和顶层窗口快照
+- Activity/RpaVideoSession 步骤
+- 不可逆动作确认
+- 业务成功/失败/结果不明确状态
+- 频率限制、幂等和断点续跑
+- 真实集成测试（默认不进 CI）
+
+## 禁止照抄
+
+- `WXWork`、`WeWorkWindow`、`SearchExternalsWnd`
+- `org_op_new`、`searchedit`、`wbtn_container`
+- 企业微信列表第 0/1 行含义
+- 来源案例的本机路径、账号和业务数据
+- 任何未经目标软件真实拾取验证的 fallback
+
+## 验收
+
+- `probe` 能明确报告依赖、窗口和关键元素
+- 每个关键元素在目标作用域内唯一
+- 关闭重开软件后仍能定位
+- 多窗口时不会误选
+- UI变化时停止并保存截图/UIA树
+- dry-run 不执行不可逆提交
+- assisted 清楚提示用户确认
+- 提交后仅在业务后置状态成立时报告成功

examples/real_desktop_rpa/assets/element-map.sample.yaml

@@ -0,0 +1,64 @@
+source_case: add-friend-wecom
+status: verified_source_redacted_example
+surface: windows
+driver: uiautomation
+elements:
+  - logical_name: main_window
+    purpose: 企业微信主窗口
+    scope:
+      process_name: WXWork
+    primary_locator:
+      window_class: WeWorkWindow
+      title: 企业微信
+    expected_count: 1
+    postcondition: UIA root 可读取
+    replace_in_new_skill: true
+
+  - logical_name: contacts_entry
+    purpose: 打开通讯录
+    scope:
+      logical_parent: main_window
+    primary_locator:
+      name: org_op_new
+      control_type: TreeItem
+    expected_count: 1
+    postcondition: 通讯录业务面板出现
+    replace_in_new_skill: true
+
+  - logical_name: add_customer_window
+    purpose: 添加客户窗口
+    scope:
+      process_name: WXWork
+    primary_locator:
+      window_class: SearchExternalsWnd
+      title: 添加客户
+    expected_count: 1
+    postcondition: search_input 可定位
+    replace_in_new_skill: true
+
+  - logical_name: search_input
+    purpose: 输入待搜索账号
+    scope:
+      logical_parent: add_customer_window
+      ancestor_name: search_layout
+    primary_locator:
+      name: searchedit
+    expected_count: 1
+    ready_condition:
+      visible: true
+      enabled: true
+    postcondition: 结果列表或用户不存在弹窗出现
+    replace_in_new_skill: true
+
+  - logical_name: verify_message_input
+    purpose: 填写添加验证语
+    scope:
+      window_class: InputReasonWnd
+      title_any:
+        - 输入认证信息
+        - 发送添加邀请
+    primary_locator:
+      name: name_edit
+    expected_count: 1
+    postcondition: 输入值可读取或发送按钮可用
+    replace_in_new_skill: true

examples/real_desktop_rpa/assets/selector-capture.sample.xml

@@ -0,0 +1,12 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!--
+  脱敏教学摘录：展示“拾取器长路径”而非可直接复制的生产 selector。
+  生产代码应保留稳定窗口、业务父容器和叶子属性，删除无业务含义的深层 index。
+-->
+<selector name="search_input" source="inspector-export">
+  <window process="WXWork" class="SearchExternalsWnd" title="添加客户" />
+  <uia name="content_container" index="1" />
+  <uia name="search_layout" index="0" />
+  <uia name="inner_search_layout" index="1" />
+  <uia name="searchedit" index="1" />
+</selector>

examples/real_desktop_rpa/scripts/service/desktop_session.py

@@ -0,0 +1,111 @@
+"""Windows 顶层窗口作用域：枚举、唯一性校验和激活。"""
+
+from __future__ import annotations
+
+import ctypes
+import ntpath
+from ctypes import wintypes
+from dataclasses import dataclass
+from typing import Iterable
+
+from util.selectors import WindowSpec
+
+
+class DesktopRpaError(RuntimeError):
+    def __init__(self, code: str, message: str) -> None:
+        super().__init__(f"ERROR:{code} {message}")
+        self.code = code
+        self.message = message
+
+
+@dataclass(frozen=True)
+class WindowCandidate:
+    hwnd: int
+    pid: int
+    process_name: str
+    class_name: str
+    title: str
+    visible: bool
+
+
+user32 = ctypes.windll.user32
+kernel32 = ctypes.windll.kernel32
+SW_RESTORE = 9
+PROCESS_QUERY_LIMITED_INFORMATION = 0x1000
+
+
+def _process_name(pid: int) -> str:
+    handle = kernel32.OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, False, pid)
+    if not handle:
+        return ""
+    try:
+        size = wintypes.DWORD(32768)
+        buffer = ctypes.create_unicode_buffer(size.value)
+        if not kernel32.QueryFullProcessImageNameW(
+            handle,
+            0,
+            buffer,
+            ctypes.byref(size),
+        ):
+            return ""
+        return ntpath.splitext(ntpath.basename(buffer.value))[0]
+    finally:
+        kernel32.CloseHandle(handle)
+
+
+def enumerate_top_windows() -> list[WindowCandidate]:
+    found: list[WindowCandidate] = []
+
+    @ctypes.WINFUNCTYPE(ctypes.c_bool, wintypes.HWND, wintypes.LPARAM)
+    def callback(hwnd, _lparam):
+        pid = wintypes.DWORD()
+        user32.GetWindowThreadProcessId(hwnd, ctypes.byref(pid))
+        cls = ctypes.create_unicode_buffer(256)
+        user32.GetClassNameW(hwnd, cls, 256)
+        length = user32.GetWindowTextLengthW(hwnd)
+        title = ctypes.create_unicode_buffer(length + 1)
+        user32.GetWindowTextW(hwnd, title, length + 1)
+        found.append(
+            WindowCandidate(
+                hwnd=int(hwnd),
+                pid=int(pid.value),
+                process_name=_process_name(int(pid.value)),
+                class_name=cls.value,
+                title=title.value,
+                visible=bool(user32.IsWindowVisible(hwnd)),
+            )
+        )
+        return True
+
+    user32.EnumWindows(callback, 0)
+    return found
+
+
+def resolve_unique_window(
+    spec: WindowSpec,
+    *,
+    candidates: Iterable[WindowCandidate] | None = None,
+) -> WindowCandidate:
+    pool = list(candidates) if candidates is not None else enumerate_top_windows()
+    matched = [
+        item
+        for item in pool
+        if item.visible
+        and item.process_name.casefold() == spec.process_name.casefold()
+        and item.class_name == spec.class_name
+        and (spec.title is None or item.title == spec.title)
+    ]
+    if not matched:
+        raise DesktopRpaError("WINDOW_NOT_FOUND", f"{spec.logical_name} 未找到")
+    if len(matched) != 1:
+        raise DesktopRpaError(
+            "ELEMENT_AMBIGUOUS",
+            f"{spec.logical_name} 匹配 {len(matched)} 个窗口，请按 PID/账号补充作用域",
+        )
+    return matched[0]
+
+
+def activate_window(window: WindowCandidate) -> None:
+    user32.ShowWindow(window.hwnd, SW_RESTORE)
+    if not user32.SetForegroundWindow(window.hwnd):
+        raise DesktopRpaError("WINDOW_ACTIVATE_FAILED", f"无法激活 hwnd={window.hwnd}")

examples/real_desktop_rpa/scripts/service/uia_locator.py

@@ -0,0 +1,96 @@
+"""作用域内 UIA 唯一定位、真实点击和逐字输入。"""
+
+from __future__ import annotations
+
+import random
+from typing import Any, Iterator
+
+from service.desktop_session import DesktopRpaError, WindowCandidate, activate_window
+from util.selectors import ElementSpec
+
+
+def require_uiautomation():
+    try:
+        import uiautomation as auto
+    except ImportError as exc:
+        raise DesktopRpaError("UIA_UNAVAILABLE", "缺少 uiautomation 依赖") from exc
+    return auto
+
+
+def root_from_window(window: WindowCandidate):
+    auto = require_uiautomation()
+    return auto.ControlFromHandle(window.hwnd)
+
+
+def _walk(root: Any, *, depth: int = 0, max_depth: int = 25) -> Iterator[Any]:
+    if depth > max_depth:
+        return
+    yield root
+    if depth == max_depth:
+        return
+    try:
+        children = root.GetChildren()
+    except Exception:
+        children = []
+    for child in children:
+        yield from _walk(child, depth=depth + 1, max_depth=max_depth)
+
+
+def _matches(control: Any, spec: ElementSpec) -> bool:
+    if spec.name is not None and (getattr(control, "Name", None) or "") != spec.name:
+        return False
+    if spec.automation_id is not None and (
+        getattr(control, "AutomationId", None) or ""
+    ) != spec.automation_id:
+        return False
+    if spec.control_type is not None and (
+        (getattr(control, "ControlTypeName", None) or "").removesuffix("Control")
+        != spec.control_type.removesuffix("Control")
+    ):
+        return False
+    if spec.class_name is not None and (
+        getattr(control, "ClassName", None) or ""
+    ) != spec.class_name:
+        return False
+    return True
+
+
+def locate_unique(root: Any, spec: ElementSpec, *, max_depth: int = 25):
+    search_root = root
+    if spec.ancestor_name:
+        ancestor_spec = ElementSpec(
+            logical_name=f"{spec.logical_name}.ancestor",
+            name=spec.ancestor_name,
+        )
+        search_root = locate_unique(root, ancestor_spec, max_depth=max_depth)
+
+    matched = [control for control in _walk(search_root, max_depth=max_depth) if _matches(control, spec)]
+    if not matched:
+        raise DesktopRpaError("ELEMENT_NOT_FOUND", f"{spec.logical_name} 未找到")
+    if len(matched) != spec.expected_count:
+        raise DesktopRpaError(
+            "ELEMENT_AMBIGUOUS",
+            f"{spec.logical_name} 期望 {spec.expected_count} 个，实际 {len(matched)} 个",
+        )
+    return matched[0]
+
+
+def click_control(control: Any, *, window: WindowCandidate) -> None:
+    activate_window(window)
+    rect = control.BoundingRectangle
+    if rect.width() <= 0 or rect.height() <= 0 or rect.left < -10000:
+        raise DesktopRpaError("ELEMENT_NOT_READY", "目标元素不可见或边界矩形无效")
+    control.Click(
+        x=random.randint(-2, 2),
+        y=random.randint(-2, 2),
+        simulateMove=True,
+    )
+
+
+def type_text(control: Any, text: str, *, window: WindowCandidate) -> None:
+    auto = require_uiautomation()
+    click_control(control, window=window)
+    auto.SendKeys("{Ctrl}a", waitTime=0.05)
+    auto.SendKeys("{Delete}", waitTime=0.05)
+    for char in text:
+        auto.SendKeys(char, waitTime=random.uniform(0.04, 0.12))

examples/real_desktop_rpa/scripts/service/wecom_reference_flow.py

@@ -0,0 +1,37 @@
+"""企业微信成功案例的安全提炼：probe 和 dry-run，不默认发送邀请。"""
+
+from __future__ import annotations
+
+from dataclasses import asdict
+from typing import Any
+
+from service.desktop_session import resolve_unique_window
+from service.uia_locator import locate_unique, root_from_window, type_text
+from util.selectors import ADD_CUSTOMER_WINDOW, CONTACTS_ENTRY, MAIN_WINDOW, SEARCH_INPUT
+
+
+def probe() -> dict[str, Any]:
+    main = resolve_unique_window(MAIN_WINDOW)
+    root = root_from_window(main)
+    contacts = locate_unique(root, CONTACTS_ENTRY)
+    return {
+        "success": True,
+        "main_window": asdict(main),
+        "contacts_entry": {
+            "name": getattr(contacts, "Name", ""),
+            "control_type": getattr(contacts, "ControlTypeName", ""),
+        },
+    }
+
+
+def fill_search_dry_run(target: str) -> dict[str, Any]:
+    """要求操作者已打开“添加客户”窗口；只填写搜索框，不提交。"""
+    search_window = resolve_unique_window(ADD_CUSTOMER_WINDOW)
+    root = root_from_window(search_window)
+    search_input = locate_unique(root, SEARCH_INPUT)
+    type_text(search_input, target, window=search_window)
+    return {
+        "success": True,
+        "mode": "dry-run",
+        "message": "已填写搜索条件，未发送邀请",
+    }

examples/real_desktop_rpa/scripts/util/selectors.py

@@ -0,0 +1,44 @@
+"""脱敏 UIA element specs；复制到新技能后必须用真实拾取结果替换。"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Optional
+
+
+@dataclass(frozen=True)
+class WindowSpec:
+    logical_name: str
+    process_name: str
+    class_name: str
+    title: Optional[str] = None
+
+
+@dataclass(frozen=True)
+class ElementSpec:
+    logical_name: str
+    name: Optional[str] = None
+    automation_id: Optional[str] = None
+    control_type: Optional[str] = None
+    class_name: Optional[str] = None
+    ancestor_name: Optional[str] = None
+    expected_count: int = 1
+
+
+MAIN_WINDOW = WindowSpec("main_window", "WXWork", "WeWorkWindow", "企业微信")
+ADD_CUSTOMER_WINDOW = WindowSpec(
+    "add_customer_window",
+    "WXWork",
+    "SearchExternalsWnd",
+    "添加客户",
+)
+CONTACTS_ENTRY = ElementSpec(
+    "contacts_entry",
+    name="org_op_new",
+    control_type="TreeItem",
+)
+SEARCH_INPUT = ElementSpec(
+    "search_input",
+    name="searchedit",
+    ancestor_name="search_layout",
+)

examples/real_desktop_rpa/tests/test_selector_contract.py

@@ -0,0 +1,44 @@
+from __future__ import annotations
+
+import sys
+import unittest
+from pathlib import Path
+
+
+ROOT = Path(__file__).resolve().parents[1]
+sys.path.insert(0, str(ROOT / "scripts"))
+
+from service.desktop_session import (  # noqa: E402
+    DesktopRpaError,
+    WindowCandidate,
+    resolve_unique_window,
+)
+from util.selectors import MAIN_WINDOW  # noqa: E402
+
+
+class SelectorContractTest(unittest.TestCase):
+    def test_unique_window_is_required(self) -> None:
+        candidate = WindowCandidate(1, 10, "WXWork", "WeWorkWindow", "企业微信", True)
+        self.assertEqual(
+            resolve_unique_window(MAIN_WINDOW, candidates=[candidate]),
+            candidate,
+        )
+
+    def test_duplicate_windows_fail_instead_of_taking_first(self) -> None:
+        candidates = [
+            WindowCandidate(1, 10, "WXWork", "WeWorkWindow", "企业微信", True),
+            WindowCandidate(2, 11, "WXWork", "WeWorkWindow", "企业微信", True),
+        ]
+        with self.assertRaises(DesktopRpaError) as ctx:
+            resolve_unique_window(MAIN_WINDOW, candidates=candidates)
+        self.assertEqual(ctx.exception.code, "ELEMENT_AMBIGUOUS")
+
+    def test_hidden_window_is_not_a_candidate(self) -> None:
+        hidden = WindowCandidate(1, 10, "WXWork", "WeWorkWindow", "企业微信", False)
+        with self.assertRaises(DesktopRpaError) as ctx:
+            resolve_unique_window(MAIN_WINDOW, candidates=[hidden])
+        self.assertEqual(ctx.exception.code, "WINDOW_NOT_FOUND")
+
+
+if __name__ == "__main__":
+    unittest.main()

examples/simulator_browser_rpa/README.md

@@ -18,6 +18,7 @@
 - **真实第三方网站 + 高风控 + 人工验证码** → 优先看 [`../real_browser_rpa/README.md`](../real_browser_rpa/README.md)
 - 目标系统有稳定 **官方 API** → 不要硬写浏览器 RPA
 - **不要把本示例的业务字段、selector、sandbox URL 原样复制**到你的 skill
+- 复制前先读 `development/RPA_ELEMENT_LOCATING.md` 与 `development/RPA_BROWSER.md`；即使 DOM 可控也要定义作用域、唯一性和后置状态
 
 ## 目录结构