Release v1.0.61: desktop RPA Name/Value and activate fixes

examples/real_desktop_rpa/README.md

@@ -5,24 +5,24 @@
 ## 状态
 
 - **技术路线已真实验证**：Windows UIA + Python `uiautomation`
-- **来源业务已真实跑通**：企业微信添加客户
+- **来源业务已真实跑通**：企业微信添加客户（通讯录 → 新的客户 → 添加 → 搜索 → 发送）
 - **本目录默认安全模式**：只提供 probe、dry-run、assisted 范式，不默认发送真实邀请
 - **不是完整 skill 仓库**：按 Copy map 选择性复制
 
 ## 先读
 
 1. [`../../development/RPA_ELEMENT_LOCATING.md`](../../development/RPA_ELEMENT_LOCATING.md)
-2. [`../../development/RPA_DESKTOP.md`](../../development/RPA_DESKTOP.md)
+2. [`../../development/RPA_DESKTOP.md`](../../development/RPA_DESKTOP.md)（含 Name/Value、最大化激活、同名叶子）
 3. 本 README 的“原始拾取到生产定位器”
 
 ## 成功案例验证了什么
 
 ```text
 枚举目标顶层窗口
-→ 按进程/窗口类/标题确认作用域
-→ 从 HWND 获取 UIA root
-→ 在父容器中定位元素
-→ 激活窗口并真实点击/输入
+→ 激活并最大化主窗（勿无条件 SW_RESTORE）
+→ 从 HWND 获取 UIA root / 业务子 pane
+→ 在父容器中定位（Name 与 Value 分开看）
+→ 真实点击并校验后置状态
 → 等待子窗口或业务结果
 → 处理独立/嵌套弹窗
 → probe-ui 诊断
@@ -30,6 +30,8 @@
 
 来源案例还提供 `dry-run / assisted / auto` 三档、频率控制和业务记忆。本 example 只提炼 UIA 元素工程；账号、风控和数据库仍由具体业务技能实现。
 
+本目录 `probe()` 是**最小探针**（主窗 + 通讯录入口，可选尝试新的客户行），**不等于**生产技能完整 probe。
+
 ## 目录结构
 
 ```text
@@ -51,25 +53,39 @@ real_desktop_rpa/
 
 ## 原始拾取到生产定位器
 
-来源案例的影刀 `selectorsV2` 导出通常包含：
+### 搜索框（添加客户窗）
 
 ```text
-Window(class=SearchExternalsWnd,title=添加客户)
-  → 多层 Layout/index
-  → search_layout
-  → searchedit
+原始：Window(SearchExternalsWnd) → 多层 Layout/index → search_layout → searchedit
+提炼：窗口 → search_layout → Name=searchedit
+```
+
+### 新的客户（易错）
+
+```text
+原始：通讯录 pane → wlist_view → item[2] → status_layout → main_chattext
+提炼：wlist_view 内找 子控件 main_chattext.Value=新的客户 的那一行，再点该行 status_layout
+禁止：全树第一个 Name=status_layout（常为「群聊」）
+禁止：把选择器标题「新的客户」当成 UIA Name 去搜
+```
+
+### 添加按钮
+
+```text
+提炼：search_external → wbtn_container（子文案 Value=添加）
+禁止：搜索旁 plus_button 下的同名 wbtn_container
 ```
 
 正确提炼方式：
 
-1. 保留稳定顶层窗口：进程、Window Class、标题。
-2. 保留最小业务父容器：如 `search_layout`。
-3. 使用叶子元素稳定属性：如 `Name=searchedit`。
-4. 删除只反映当前布局的深层 `index`。
+1. 保留稳定顶层窗口：进程、Window Class、标题；需要时加上业务子 pane。
+2. 保留最小业务父容器。
+3. 叶子用稳定 Name **和/或** Value / 子控件 Value。
+4. 删除只反映当前布局的深层 index；index 仅作已验证兜底并写清含义。
 5. 验证父容器内候选数量为 `1`。
 6. 重启应用后再次验证。
 
-不要把完整 XML 路径或 `item[0]` 原样变成生产定位器。
+不要把完整 XML 路径或未校验的 `item[0]` 原样变成生产定位器。
 
 ## Element map
 
@@ -79,8 +95,9 @@ Window(class=SearchExternalsWnd,title=添加客户)
 
 - 窗口不唯一时失败，不取 `hwnds[0]`。
 - 控件不唯一时失败，不取第一个匹配。
-- `item`、`status_layout`、`wbtn_container` 等通用 Name 必须放进稳定父容器。
-- 行顺序不是身份；用业务文案或稳定属性选择结果行。
+- `item`、`status_layout`、`wbtn_container` 等通用 Name 必须放进稳定父容器，并用文案校验。
+- 行顺序不是身份；用业务文案（Value）选择结果行。
+- 主窗依赖完整侧栏时先 maximize；`activate(maximize=False)` 不得冲掉已最大化状态。
 - 只有主定位成功、窗口前台且矩形有效时，才允许从 UIA BoundingRectangle 降级坐标点击。
 
 ## 三种实施模式
@@ -97,8 +114,8 @@ Window(class=SearchExternalsWnd,title=添加客户)
 
 | 来源 | 目标 | 说明 |
 |---|---|---|
-| `scripts/service/desktop_session.py` | `scripts/service/desktop_session.py` | 顶层窗口枚举、唯一性、激活 |
-| `scripts/service/uia_locator.py` | `scripts/service/uia_locator.py` | 作用域内唯一定位、点击、输入 |
+| `scripts/service/desktop_session.py` | `scripts/service/desktop_session.py` | 顶层窗口枚举、唯一性、激活/最大化 |
+| `scripts/service/uia_locator.py` | `scripts/service/uia_locator.py` | 作用域内唯一定位（含 Value）、点击、输入 |
 | `scripts/util/selectors.py` | `scripts/util/<platform>_selectors.py` | element spec；必须全部替换 |
 | `scripts/service/wecom_reference_flow.py` | `scripts/service/<business>_flow.py` | 只复制流程分层，不复制企微 selector |
 | `assets/element-map.sample.yaml` | `assets/<platform>-element-map.yaml` | 真实拾取后填写 |
@@ -118,14 +135,14 @@ Window(class=SearchExternalsWnd,title=添加客户)
 ## 禁止照抄
 
 - `WXWork`、`WeWorkWindow`、`SearchExternalsWnd`
-- `org_op_new`、`searchedit`、`wbtn_container`
-- 企业微信列表第 0/1 行含义
+- `org_op_new`、`searchedit`、`wbtn_container`、`main_chattext`
+- 企业微信列表第 0/1/2 行含义或固定 index
 - 来源案例的本机路径、账号和业务数据
 - 任何未经目标软件真实拾取验证的 fallback
 
 ## 验收
 
-- `probe` 能明确报告依赖、窗口和关键元素
+- `probe` 能明确报告依赖、窗口和关键元素（生产技能覆盖关键跳转节点）
 - 每个关键元素在目标作用域内唯一
 - 关闭重开软件后仍能定位
 - 多窗口时不会误选

examples/real_desktop_rpa/assets/element-map.sample.yaml

@@ -2,6 +2,10 @@ source_case: add-friend-wecom
 status: verified_source_redacted_example
 surface: windows
 driver: uiautomation
+notes:
+  - 选择器显示名 ≠ UIA Name；可见中文常在 Value / 子 Label
+  - 禁止全树第一个 status_layout / wbtn_container
+  - 主窗流程先 maximize；activate 勿无条件 SW_RESTORE
 elements:
   - logical_name: main_window
     purpose: 企业微信主窗口
@@ -11,7 +15,7 @@ elements:
       window_class: WeWorkWindow
       title: 企业微信
     expected_count: 1
-    postcondition: UIA root 可读取
+    postcondition: UIA root 可读取；业务需要时已最大化
     replace_in_new_skill: true
 
   - logical_name: contacts_entry
@@ -21,8 +25,57 @@ elements:
     primary_locator:
       name: org_op_new
       control_type: TreeItem
+      value_hint: 通讯录
     expected_count: 1
-    postcondition: 通讯录业务面板出现
+    postcondition: 出现企业微信-通讯录业务 pane
+    replace_in_new_skill: true
+
+  - logical_name: contacts_pane
+    purpose: 通讯录业务子层作用域
+    scope:
+      logical_parent: main_window
+    primary_locator:
+      name_contains: 通讯录
+    expected_count: 1
+    postcondition: 可枚举 wlist_view
+    replace_in_new_skill: true
+
+  - logical_name: new_customer_row
+    purpose: 侧栏进入「新的客户」
+    scope:
+      logical_parent: contacts_pane
+      ancestor_name: wlist_view
+    primary_locator:
+      row_leaf_name: status_layout
+      child_name: main_chattext
+      child_value: 新的客户
+    expected_count: 1
+    postcondition: panel_title_label Value=新的客户 或出现 search_external
+    failure_note: 禁止点全树第一个 status_layout（常为群聊）
+    replace_in_new_skill: true
+
+  - logical_name: panel_title
+    purpose: 校验已进入新的客户页
+    scope:
+      logical_parent: contacts_pane
+    primary_locator:
+      name: panel_title_label
+      value: 新的客户
+    expected_count: 1
+    replace_in_new_skill: true
+
+  - logical_name: add_customer_button
+    purpose: 打开添加客户窗口
+    scope:
+      logical_parent: contacts_pane
+      ancestor_name: search_external
+    primary_locator:
+      name: wbtn_container
+      child_name: ww_btn_text
+      child_value: 添加
+    expected_count: 1
+    postcondition: 出现 SearchExternalsWnd
+    failure_note: 禁止点搜索旁 plus_button 下的 wbtn_container
     replace_in_new_skill: true
 
   - logical_name: add_customer_window

examples/real_desktop_rpa/assets/selector-capture.sample.xml

@@ -10,3 +10,17 @@
   <uia name="inner_search_layout" index="1" />
   <uia name="searchedit" index="1" />
 </selector>
+
+<!--
+  新的客户：长路径示意 → 生产应改为
+  wlist_view 内 child main_chattext.Value=新的客户 的行，再点该行 status_layout。
+  禁止全树第一个 status_layout。
+-->
+<selector name="new_customer_row" source="inspector-export">
+  <window process="WXWork" class="WeWorkWindow" title="企业微信" />
+  <pane name="企业微信-通讯录" />
+  <uia name="wlist_view" />
+  <uia name="item" index="2" />
+  <uia name="status_layout" />
+  <uia name="main_chattext" value="新的客户" />
+</selector>

examples/real_desktop_rpa/scripts/service/desktop_session.py

@@ -4,6 +4,7 @@ from __future__ import annotations
 
 import ctypes
 import ntpath
+import time
 from ctypes import wintypes
 from dataclasses import dataclass
 from typing import Iterable
@@ -31,6 +32,7 @@ class WindowCandidate:
 user32 = ctypes.windll.user32
 kernel32 = ctypes.windll.kernel32
 SW_RESTORE = 9
+SW_MAXIMIZE = 3
 PROCESS_QUERY_LIMITED_INFORMATION = 0x1000
 
 
@@ -105,7 +107,23 @@ def resolve_unique_window(
     return matched[0]
 
 
-def activate_window(window: WindowCandidate) -> None:
-    user32.ShowWindow(window.hwnd, SW_RESTORE)
-    if not user32.SetForegroundWindow(window.hwnd):
-        raise DesktopRpaError("WINDOW_ACTIVATE_FAILED", f"无法激活 hwnd={window.hwnd}")
+def activate_window(window: WindowCandidate, *, maximize: bool = False) -> None:
+    """激活窗口。
+
+    maximize=True：主窗进入依赖完整侧栏的流程时使用。
+    maximize=False：只前置；**禁止**对已最大化窗口无条件 SW_RESTORE。
+    """
+    hwnd = window.hwnd
+    iconic = bool(user32.IsIconic(hwnd))
+    zoomed = bool(user32.IsZoomed(hwnd))
+    if maximize:
+        if iconic:
+            user32.ShowWindow(hwnd, SW_RESTORE)
+        if not zoomed:
+            user32.ShowWindow(hwnd, SW_MAXIMIZE)
+    else:
+        if iconic:
+            user32.ShowWindow(hwnd, SW_RESTORE)
+    if not user32.SetForegroundWindow(hwnd):
+        raise DesktopRpaError("WINDOW_ACTIVATE_FAILED", f"无法激活 hwnd={hwnd}")
+    time.sleep(0.35 if maximize else 0.2)

examples/real_desktop_rpa/scripts/service/uia_locator.py

@@ -22,6 +22,15 @@ def root_from_window(window: WindowCandidate):
     return auto.ControlFromHandle(window.hwnd)
 
 
+def control_value(control: Any) -> str:
+    try:
+        auto = require_uiautomation()
+        raw = control.GetPropertyValue(auto.PropertyId.ValueValueProperty)
+        return str(raw or "").strip()
+    except Exception:
+        return ""
+
+
 def _walk(root: Any, *, depth: int = 0, max_depth: int = 25) -> Iterator[Any]:
     if depth > max_depth:
         return
@@ -36,9 +45,21 @@ def _walk(root: Any, *, depth: int = 0, max_depth: int = 25) -> Iterator[Any]:
         yield from _walk(child, depth=depth + 1, max_depth=max_depth)
 
 
+def _child_has_value(control: Any, child_name: str, expected_value: str, *, max_depth: int = 10) -> bool:
+    for node in _walk(control, max_depth=max_depth):
+        if (getattr(node, "Name", None) or "") != child_name:
+            continue
+        if control_value(node) == expected_value:
+            return True
+    return False
+
+
 def _matches(control: Any, spec: ElementSpec) -> bool:
     if spec.name is not None and (getattr(control, "Name", None) or "") != spec.name:
         return False
+    if spec.value is not None and not spec.child_value_name:
+        if control_value(control) != spec.value:
+            return False
     if spec.automation_id is not None and (
         getattr(control, "AutomationId", None) or ""
     ) != spec.automation_id:
@@ -52,6 +73,9 @@ def _matches(control: Any, spec: ElementSpec) -> bool:
         getattr(control, "ClassName", None) or ""
     ) != spec.class_name:
         return False
+    if spec.child_value_name and spec.value:
+        if not _child_has_value(control, spec.child_value_name, spec.value):
+            return False
     return True
 
 
@@ -64,7 +88,9 @@ def locate_unique(root: Any, spec: ElementSpec, *, max_depth: int = 25):
         )
         search_root = locate_unique(root, ancestor_spec, max_depth=max_depth)
 
-    matched = [control for control in _walk(search_root, max_depth=max_depth) if _matches(control, spec)]
+    matched = [
+        control for control in _walk(search_root, max_depth=max_depth) if _matches(control, spec)
+    ]
     if not matched:
         raise DesktopRpaError("ELEMENT_NOT_FOUND", f"{spec.logical_name} 未找到")
     if len(matched) != spec.expected_count:
@@ -75,8 +101,13 @@ def locate_unique(root: Any, spec: ElementSpec, *, max_depth: int = 25):
     return matched[0]
 
 
-def click_control(control: Any, *, window: WindowCandidate) -> None:
-    activate_window(window)
+def click_control(
+    control: Any,
+    *,
+    window: WindowCandidate,
+    maximize: bool = False,
+) -> None:
+    activate_window(window, maximize=maximize)
     rect = control.BoundingRectangle
     if rect.width() <= 0 or rect.height() <= 0 or rect.left < -10000:
         raise DesktopRpaError("ELEMENT_NOT_READY", "目标元素不可见或边界矩形无效")
@@ -89,7 +120,7 @@ def click_control(control: Any, *, window: WindowCandidate) -> None:
 
 def type_text(control: Any, text: str, *, window: WindowCandidate) -> None:
     auto = require_uiautomation()
-    click_control(control, window=window)
+    click_control(control, window=window, maximize=False)
     auto.SendKeys("{Ctrl}a", waitTime=0.05)
     auto.SendKeys("{Delete}", waitTime=0.05)
     for char in text:

examples/real_desktop_rpa/scripts/service/wecom_reference_flow.py

@@ -5,28 +5,49 @@ from __future__ import annotations
 from dataclasses import asdict
 from typing import Any
 
-from service.desktop_session import resolve_unique_window
+from service.desktop_session import activate_window, resolve_unique_window
 from service.uia_locator import locate_unique, root_from_window, type_text
-from util.selectors import ADD_CUSTOMER_WINDOW, CONTACTS_ENTRY, MAIN_WINDOW, SEARCH_INPUT
+from util.selectors import (
+    ADD_CUSTOMER_WINDOW,
+    CONTACTS_ENTRY,
+    MAIN_WINDOW,
+    NEW_CUSTOMER_ROW,
+    SEARCH_INPUT,
+)
 
 
 def probe() -> dict[str, Any]:
+    """最小 probe：主窗 + 通讯录入口。
+
+    生产技能应对「新的客户 / 添加」等关键节点做更深探测；本函数不等于生产完整度。
+    调用前会 maximize 主窗，避免侧栏折叠导致 org_op_new 抖动。
+    """
     main = resolve_unique_window(MAIN_WINDOW)
+    activate_window(main, maximize=True)
     root = root_from_window(main)
     contacts = locate_unique(root, CONTACTS_ENTRY)
-    return {
+    payload: dict[str, Any] = {
         "success": True,
+        "minimized_probe": True,
         "main_window": asdict(main),
         "contacts_entry": {
             "name": getattr(contacts, "Name", ""),
             "control_type": getattr(contacts, "ControlTypeName", ""),
         },
     }
+    # 可选：若已在通讯录页，尝试按 Value 定位「新的客户」行（失败不视为 probe 失败）
+    try:
+        row = locate_unique(root, NEW_CUSTOMER_ROW)
+        payload["new_customer_row"] = {"name": getattr(row, "Name", "")}
+    except Exception:
+        payload["new_customer_row"] = None
+    return payload
 
 
 def fill_search_dry_run(target: str) -> dict[str, Any]:
     """要求操作者已打开“添加客户”窗口；只填写搜索框，不提交。"""
     search_window = resolve_unique_window(ADD_CUSTOMER_WINDOW)
+    activate_window(search_window, maximize=False)
     root = root_from_window(search_window)
     search_input = locate_unique(root, SEARCH_INPUT)
     type_text(search_input, target, window=search_window)

examples/real_desktop_rpa/scripts/util/selectors.py

@@ -18,10 +18,16 @@ class WindowSpec:
 class ElementSpec:
     logical_name: str
     name: Optional[str] = None
+    """UIA Name（内部 id，未必是界面中文）。"""
+    value: Optional[str] = None
+    """UIA Value（可见中文文案常在此）。"""
     automation_id: Optional[str] = None
     control_type: Optional[str] = None
     class_name: Optional[str] = None
     ancestor_name: Optional[str] = None
+    """先定位到该 Name 的祖先，再在其下搜索。"""
+    child_value_name: Optional[str] = None
+    """在候选控件子树中查找该 Name 的子控件，并用 value 匹配其 Value。"""
     expected_count: int = 1
 
 
@@ -37,6 +43,19 @@ CONTACTS_ENTRY = ElementSpec(
     name="org_op_new",
     control_type="TreeItem",
 )
+# 示意：用子控件 main_chattext 的 Value 定位「新的客户」行（禁止全树第一个 status_layout）
+NEW_CUSTOMER_ROW = ElementSpec(
+    "new_customer_row",
+    name="status_layout",
+    ancestor_name="wlist_view",
+    child_value_name="main_chattext",
+    value="新的客户",
+)
+ADD_BUTTON = ElementSpec(
+    "add_customer_button",
+    name="wbtn_container",
+    ancestor_name="search_external",
+)
 SEARCH_INPUT = ElementSpec(
     "search_input",
     name="searchedit",

examples/real_desktop_rpa/tests/test_selector_contract.py

@@ -3,6 +3,7 @@ from __future__ import annotations
 import sys
 import unittest
 from pathlib import Path
+from unittest.mock import MagicMock, patch
 
 
 ROOT = Path(__file__).resolve().parents[1]
@@ -11,9 +12,11 @@ sys.path.insert(0, str(ROOT / "scripts"))
 from service.desktop_session import (  # noqa: E402
     DesktopRpaError,
     WindowCandidate,
+    activate_window,
     resolve_unique_window,
 )
-from util.selectors import MAIN_WINDOW  # noqa: E402
+from service import uia_locator  # noqa: E402
+from util.selectors import ElementSpec, MAIN_WINDOW  # noqa: E402
 
 
 class SelectorContractTest(unittest.TestCase):
@@ -39,6 +42,58 @@ class SelectorContractTest(unittest.TestCase):
             resolve_unique_window(MAIN_WINDOW, candidates=[hidden])
         self.assertEqual(ctx.exception.code, "WINDOW_NOT_FOUND")
 
+    def test_activate_without_maximize_skips_restore_when_zoomed(self) -> None:
+        window = WindowCandidate(42, 10, "WXWork", "WeWorkWindow", "企业微信", True)
+        with patch("service.desktop_session.user32") as user32:
+            user32.IsIconic.return_value = False
+            user32.IsZoomed.return_value = True
+            user32.SetForegroundWindow.return_value = True
+            activate_window(window, maximize=False)
+            user32.ShowWindow.assert_not_called()
+            user32.SetForegroundWindow.assert_called_once_with(42)
+
+    def test_activate_maximize_when_not_zoomed(self) -> None:
+        window = WindowCandidate(42, 10, "WXWork", "WeWorkWindow", "企业微信", True)
+        with patch("service.desktop_session.user32") as user32:
+            user32.IsIconic.return_value = False
+            user32.IsZoomed.return_value = False
+            user32.SetForegroundWindow.return_value = True
+            activate_window(window, maximize=True)
+            user32.ShowWindow.assert_called()
+            user32.SetForegroundWindow.assert_called_once_with(42)
+
+    def test_locate_unique_matches_child_value(self) -> None:
+        row = MagicMock()
+        row.Name = "status_layout"
+        row.AutomationId = ""
+        row.ControlTypeName = "TreeControl"
+        row.ClassName = ""
+        label = MagicMock()
+        label.Name = "main_chattext"
+        label.GetChildren.return_value = []
+        row.GetChildren.return_value = [label]
+
+        root = MagicMock()
+        root.Name = "wlist_view"
+        root.AutomationId = ""
+        root.ControlTypeName = "TreeControl"
+        root.ClassName = ""
+        root.GetChildren.return_value = [row]
+
+        with patch.object(uia_locator, "control_value", side_effect=lambda c: "新的客户" if c is label else ""):
+            spec = ElementSpec(
+                "new_customer_row",
+                name="status_layout",
+                ancestor_name="wlist_view",
+                child_value_name="main_chattext",
+                value="新的客户",
+            )
+            # ancestor is root itself when searching from root with name wlist_view
+            found = uia_locator.locate_unique(root, ElementSpec("wlist", name="wlist_view"))
+            self.assertIs(found, root)
+            found_row = uia_locator.locate_unique(root, spec)
+            self.assertIs(found_row, row)
+
 
 if __name__ == "__main__":
     unittest.main()