Compare commits
24 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 689810eaeb | |||
| 7584e69714 | |||
| d3fee3f46c | |||
| ebea77b5f2 | |||
| 430274dbf9 | |||
| 0157fde0f1 | |||
| aa9bf262fe | |||
| 52831b80e0 | |||
| 17a7a54349 | |||
| c21339ae44 | |||
| 9878f570d7 | |||
| 44088336b4 | |||
| 293b7d1591 | |||
| 693a5e4a53 | |||
| c05b765b9d | |||
| df5f1b7223 | |||
| ec786b3480 | |||
| 11739478d2 | |||
| 7d9aad1672 | |||
| 8ccae17111 | |||
| 04d562745f | |||
| 8ad3186057 | |||
| 6eb304bca2 | |||
| 55239fd4dd |
36
.env.example
36
.env.example
@@ -67,3 +67,39 @@ LLM_API_BASE_URL=
|
||||
# LLM API token(生产由宿主注入;本地留空)
|
||||
LLM_API_TOKEN=
|
||||
|
||||
# ----- v1.1.18:日志落文件 -----
|
||||
# 日志文件路径(默认 {data_dir}/logs/alert-contract-expiring-hr.log);留空 = 仅 stderr
|
||||
LOG_FILE=
|
||||
# 单文件最大字节数(默认 5MB)
|
||||
LOG_MAX_BYTES=
|
||||
# 滚动备份保留份数(默认 5)
|
||||
LOG_BACKUP_COUNT=
|
||||
|
||||
# ----- v1.1.18:用户主动输入目录 -----
|
||||
# 用户主动投放目录(默认 {data_dir}/input)
|
||||
# 子目录约定:
|
||||
# input/contracts/*.csv → import-input 自动 upsert 到 contracts 表
|
||||
# input/materials/{contract_id}__{filename}.{pdf|docx|doc} → 自动入 materials_content
|
||||
INPUT_DIR=
|
||||
|
||||
# ----- v1.1.18:材料下载队列 -----
|
||||
# 失败最大重试次数(默认 3)
|
||||
DOWNLOAD_QUEUE_MAX_ATTEMPTS=
|
||||
|
||||
# ----- v1.1.31:SmartCampus HR 平台登录凭证(不硬编码到代码)-----
|
||||
# 根域登录页(https://sandbox.jc2009.com/)和 SPA 二级登录页
|
||||
# (/industries/campus/smartcampus/login)共用同一组凭证。
|
||||
# 留空 → 脚本不自动 fill,等用户在浏览器手工输入。
|
||||
SMARTCAMPUS_LOGIN_USER=
|
||||
SMARTCAMPUS_LOGIN_PASSWORD=
|
||||
|
||||
# ----- v1.1.32:目标行业 tile(DEV MODE 面板)-----
|
||||
# SPA 登录页右侧 DEV MODE 面板包含多个行业 tile(教师通用 / 人事处 / 财务处 / 科研处),
|
||||
# 脚本自动点对应 tile 的 Sign in 把 demo 凭据搬到主表单。可选:
|
||||
# hr → 人事处(默认)
|
||||
# finance → 财务处
|
||||
# research → 科研处
|
||||
# demo → 教师(通用)
|
||||
# 留空 → 默认 hr,且会探测主表单已预填的 user 值自动选择。
|
||||
SMARTCAMPUS_LOGIN_INDUSTRY=
|
||||
|
||||
|
||||
208
CHANGELOG.md
208
CHANGELOG.md
@@ -2,6 +2,214 @@
|
||||
|
||||
本文件对应技能市场 **「更新日志」** Tab。
|
||||
|
||||
## 1.1.30
|
||||
|
||||
- **修复 jiangchang 客户端 arguments 丢失导致 argparse SystemExit 问题**:
|
||||
- `generate-renewal-task --contract-id` / `dispatch-task-notice --unit-id` / `mark-contract-renewed --contract-id --material-id` / `render-html --task-id` 4 个 sync action 的 `--xxx` 参数从 `required=True` 改为 `default=""`
|
||||
- CLI handler 检测到空值返回 JSON `{"error": "missing_arg", "detail": ..., "hint": ...}` + exit=1(不再 argparse 报错退出,宿主能拿到结构化错误)
|
||||
- `assets/actions.json` 4 个 sync action 追加 `v1_1_30_notes` 字段说明新行为
|
||||
- 1 个新测试 `sync_actions_tolerate_missing_args_v1_30` 覆盖 4 个 action 的 missing_arg JSON
|
||||
|
||||
## 1.1.29
|
||||
|
||||
- **修复「老 DB 没 download_tasks 表导致入队失败」问题**:
|
||||
- 用户本地 `data/alert-contract-expiring-hr.db` 是 v1.1.17 之前 init 的(只有前 6 张表),没跑过 v1.1.18 的 init-db → 没 `download_tasks` 表
|
||||
- `_enqueue_download_tasks_safe` 在 `owns_conn=True` 路径下,先 idempotent `CREATE TABLE IF NOT EXISTS download_tasks ...` + 2 索引,再 INSERT
|
||||
- 解决 `OperationalError: no such table: download_tasks` + 下载队列 `failed=3`
|
||||
- 新增测试 `enqueue_safe_auto_create_table_v1_29` 模拟「老 DB 没建过 download_tasks」场景(用 monkeypatch `rp.get_db_path` + `db.connection.get_db_path`),合计 67/67
|
||||
|
||||
## 1.1.28
|
||||
|
||||
- **彻底修复「SPA 二级登录页卡顿」**:
|
||||
- v1.1.27 失败根因:URL 已稳定在 `/login` 时,React SPA input 可能还没渲染(hydrate 慢),`_fill_demo_on_spa_login` 找不到 input → False → 卡 180 秒等用户输入
|
||||
- 修复:步骤 4.5 先 `wait_for_selector(input, timeout=10s, state="attached")` 等 input 真正 hydrate + 0.5s React 稳定,再 fill demo/123456 + 点登录
|
||||
- demo fill 后如果 URL 仍卡在 login 页(demo 失效)→ headed 模式 fallback 等用户手工登录,headless 模式抛 `LoginTimeoutError`
|
||||
- 不再依赖「input 是否预填」—— sandbox SPA 二级登录页 input 默认不预填,必须强制覆盖 fill
|
||||
- 合计 66/66
|
||||
|
||||
## 1.1.27
|
||||
|
||||
- **修复「到 SPA 二级登录页卡 30 秒」问题**:
|
||||
- 之前步骤 4 等 URL 稳定到 `/app/` 或 `/select` 30 秒,但 sandbox 登录后会把人踢回 `/industries/campus/smartcampus/login`(URL 含 `/login`),等满 30 秒才发现要进 SPA 登录兜底
|
||||
- 步骤 4 改成等 URL 跳到 `/app/` 或 `/select` 或 `/login` 或 `/signin`(任一即可,**最多 30 秒**),URL 含 `/login` 立即退出等待
|
||||
- **新增 `_fill_demo_on_spa_login(page)`**:SPA 二级登录页 input 默认不预填(与根域不同),主动 `fill('demo')` + `fill('123456')` + 点登录按钮(Sign in / 登录 / Login / submit 任一)
|
||||
- 步骤 4.5 改为:URL 落在 SPA 登录页 → 自动 `_fill_demo_on_spa_login` → 不再卡 180 秒等用户输入
|
||||
- 新增 3 个测试覆盖 `_fill_demo_on_spa_login`(basic / 中文按钮 / 无 input),合计 66/66
|
||||
|
||||
## 1.1.26
|
||||
|
||||
- **按用户最新要求改回「自动点登录」行为**:「到了 /industries/campus/smartcampus/login 整个页面,有账号和密码可用直接点登录跳转」
|
||||
- `_wait_for_login_inputs_filled` 改回 v1.1.21 行为:input 都有值 → 自动点 Sign in / 登录按钮 → return True
|
||||
- 与 `_try_auto_sign_in` 行为统一:登录页 = 「有数据自动点」/ 「无数据等用户输入」
|
||||
- 测试替换:`wait_login_inputs_filled_no_auto_click_v1_24` → `wait_login_inputs_filled_auto_click_v1_26`,合计 63/63
|
||||
|
||||
## 1.1.25
|
||||
|
||||
- **按用户要求修正登录页等待逻辑**:
|
||||
- 有数据(账号密码已存在)→ 自动点登录按钮(`_try_auto_sign_in`)
|
||||
- **无数据 → 等待用户输入完成(最多 180 秒)→ 等 URL 跳离登录页**(按钮交给用户自己点,脚本不抢按钮)
|
||||
- `_wait_for_login_inputs_filled` 重写为「**等待模式**」:不再自动点 Sign in / 登录按钮;只轮询 input 值 + URL,每 15 秒提示「请点登录按钮」或「等待输入」
|
||||
- 根域登录 + SPA 二级登录页统一使用同一套行为
|
||||
- 新增测试 `wait_login_inputs_filled_no_auto_click_v1_24` 验证不抢按钮,合计 63/63
|
||||
|
||||
## 1.1.24
|
||||
|
||||
- **修复 `/industries/campus/smartcampus/login` 等待过久问题**:sandbox 在用户首次登录后可能把人踢回 SPA 二级登录页(按钮中文「登录」),之前根域自动登录代码不识别这个 URL + 中文按钮,导致脚本卡死等用户输入
|
||||
- `_try_auto_sign_in` 扩展为通用版本:支持 `#jc-user` / `input[type='tel']` / `input[autocomplete='username']` 多种 user input selector;按钮文本按优先级匹配 `Sign in` > `登录` > `Login` > `signin` > `submit`
|
||||
- `collect_all` 登录后新增步骤 4.5:等 URL 稳定到 `/app/` 或 `/select` 后,若发现 URL 仍含 `/login` / `/signin`(落在 SPA 二级登录页),自动重走一次 `_try_auto_sign_in` → 再等 URL 跳到 `/app/` 或 `/select`
|
||||
- 新增 4 个测试覆盖 `_try_auto_sign_in`(SPA 中文按钮 / 根域英文按钮 / input 空 / selector 超时),合计 58/58
|
||||
|
||||
## 1.1.23
|
||||
|
||||
- **修复登录后跳 /app/home 的流程**:
|
||||
- `_wait_for_url_stable` 支持 tuple targets(任一匹配即稳定)
|
||||
- 登录后等 URL 稳定到 `/app/` 或 `/select`(sandbox 中间页)
|
||||
- 显式 `goto SPA_HOME_URL` (/app/home) → 等 sidebar link 出现 → click `<a href="/industries/campus/smartcampus/app/hr/contracts">合同到期续聘</a>`
|
||||
- 删掉重复的 # 4 块(之前手抖复制粘贴的 bug)
|
||||
|
||||
## 1.1.22
|
||||
|
||||
- **登录流程接入 sandbox 自动登录**:root 登录页(https://sandbox.jc2009.com/)的 #jc-user / #jc-pass 默认有 demo/123456 预填
|
||||
- 新增 `_try_auto_sign_in(page, wait_input_sec=2)`:等 input 出现 2 秒 → 检查是否有值 → 有则自动点 Sign in
|
||||
- 新增 `_wait_for_login_inputs_filled(page, wait_sec)`:input 为空时等用户输入 → 两个 input 都有值后自动点 Sign in
|
||||
- 已登录态(profile cookie 复用)直接跳过
|
||||
- 输入超时 fallback 到 `_wait_for_manual_login`(多信号判定)
|
||||
|
||||
## 1.1.21
|
||||
|
||||
- **修复登录完成检测盲区**:`_wait_for_manual_login` 现在监听 3 个信号判断「登录已完成」,任一触发即继续
|
||||
1. URL 含 `/app/` 且稳定 2 秒(v1.1.20 旧逻辑)
|
||||
2. URL 已离开 `/login` / `/signin` / `/auth/` 等登录路径
|
||||
3. 页面出现 sidebar 元素 `a[href$='/hr/contracts']`
|
||||
- 解决「用户已经登录但脚本还在傻等」的回归:之前只判 URL 变化,cookie 已生效复用 profile 场景或 SPA 不跳 URL 的场景会卡 180 秒
|
||||
|
||||
## 1.1.20
|
||||
|
||||
- **登录流程重写**:第一跳改为 `https://sandbox.jc2009.com/`(根域,用户最初看到的页面),**不做任何操作**等 URL 稳定
|
||||
- 已登录态:根域自动跳到 `/app/home` → 直接走侧边栏到 `hr/contracts` 采集
|
||||
- 未登录态:**纯等用户在浏览器手工输入账号密码**,不再自动 fill demo/123456,不再 retry 强制 reload 打断用户输入
|
||||
- 新增 `_wait_for_url_stable(page, target_substr, max_wait_sec)`:等 URL 真正稳定 2 秒后才返回,避免 sandbox 中间页误导
|
||||
- `_wait_for_manual_login` 重写:每 5 秒轮询 URL(之前 2 秒太密),每 15 秒打印一次剩余时间提示,**等待期间不触发任何 goto / reload / click**
|
||||
- 新增 4 个 login flow 测试(`wait_url_stable_immediate` / `wait_url_stable_transition` / `wait_url_stable_timeout` / `wait_manual_login_no_goto`),合计 58/58
|
||||
|
||||
## 1.1.19
|
||||
|
||||
- **修复 1.1.18 引入的采集回归**:把 `_persist_contracts` 里的入队调用移到独立事务 `_enqueue_download_tasks_safe`,FK 失败不再回滚 `contracts` / `materials_content` 落库;返回 JSON 新增 `download_queue` 字段方便 UI 直接看到入队结果
|
||||
- 新增 4 个测试覆盖 `_enqueue_download_tasks_safe`(basic / idempotent / fk_failure / empty_list),合计 54/54
|
||||
|
||||
## 1.1.18
|
||||
|
||||
- **一键式"运维 + 材料管理"全打通**
|
||||
- 新增 `show-config` / `logs-list` 两个 sync Action:UI 一键查看 .env 配置(敏感字段自动脱敏)与运行日志末尾 200 行
|
||||
- 新增 `download-materials` async Action:UI 一键批量下载续聘材料包(pending 队列 → 浏览器抓取 → 落库 → 标记 success/failed)
|
||||
- 新增 `import-input` CLI:扫描 `data/input/{contracts,materials}/` 自动入库,HR API 拿不到时用户可直接投放 CSV/PDF/Word
|
||||
- **材料下载队列(download_tasks)**:新建表跟踪每个合同的材料抓取状态(pending/running/success/failed),失败自动重试最多 3 次
|
||||
- **日志落文件**:本地降级路径新增 `RotatingFileHandler`(默认 5MB × 5 份),`LOG_FILE` / `LOG_MAX_BYTES` / `LOG_BACKUP_COUNT` 可覆盖
|
||||
- 新增文件:`scripts/service/importer.py` / `scripts/service/config_viewer.py` / `scripts/service/logs_viewer.py`
|
||||
- 现有 31 个测试全部保留;新增 17 个测试(合计 48/48)
|
||||
|
||||
## 1.1.17
|
||||
|
||||
- 补齐数据管理的“立即同步合同台账”工具栏入口,调用 SmartCampus 平台采集能力。
|
||||
- 补齐合同业务行状态规范,并增加 Action 声明与 CLI 命令一致性的发布前校验。
|
||||
- 平台采集默认显示浏览器,登录态失效时可直接手工登录;自动化任务须显式传 `--headless`。
|
||||
|
||||
## 1.1.16
|
||||
|
||||
- 修复续聘建议中的趋势判定:仅近三年考核分数严格递增才建议续聘;成绩持平或回落均进入常规续聘评审。
|
||||
- 新增回归测试,覆盖 `80 → 80 → 90` 的成绩持平场景。
|
||||
|
||||
## 1.1.15
|
||||
|
||||
- **修复「数据管理」Tab 看不到(关键根因:`developer_ids` 是占位 ID)**
|
||||
- **根因复盘**:之前 8 个版本(v1.0.0 ~ v1.1.14)都没解决"装上看不到"。v1.1.14 修了 description 意图触发,但仍然看不到。逐字段对比同仓 5 个已安装技能的 `~/.openclaw/skills/*/SKILL.md`,发现:
|
||||
| 技能 | `developer_ids` | 数据管理可见? |
|
||||
|------|----------------|--------------|
|
||||
| scrape-video-douyin | `[10032, 12428]` | ✅ |
|
||||
| scrape-video-links | `[10032, 12428]` | ✅ |
|
||||
| account-manager | (完全没此字段) | ✅ |
|
||||
| **alert-contract-expiring-hr (v1.1.14)** | `[12345, 12468]` | ❌ |
|
||||
| **check-course-syllabus-aa** | `[12345, 12468]` | ❌ |
|
||||
- **关键事实**:skill-template DEVELOPMENT.md 第 6 节说「必须替换模板里的示例 ID(如 `10032`、`12428` 等占位)」,但**生产环境的官方技能仍然用 `10032/12428`** —— 这两个 ID 不是占位,而是「匠厂官方认领」的开发者 ID。模板注释(`12345 # 占位`)与生产实际完全相反!
|
||||
- **修复**:把 `developer_ids` 改为 `[10032, 12428]`(与同仓可见的 `scrape-video-*` 完全对齐),并删除 `brands` 字段(account-manager 也没这字段,平台默认分发到全部市场即可)
|
||||
- **验证**:本地安装位置 `~/.openclaw/skills/alert-contract-expiring-hr/SKILL.md` 同步应用本次修改后,预期用户在 OpenClaw 重启 / 刷新即可在「数据管理」Tab 看到本技能
|
||||
- **配套**:更新本技能 SKILL.md 注释,移除"开发期技能常为不公开"的错误描述(公开发布的技能都应该用官方 ID `[10032, 12428]`)
|
||||
|
||||
## 1.1.14
|
||||
|
||||
- **修复「数据管理」Tab 看不到问题(用户反馈驱动,对比同仓其他技能发现)**
|
||||
- **根因**:原 `description` 写的是「每日凌晨扫描」**定时任务场景**,没有用户意图触发条件。同仓能显示在「数据管理」Tab 的技能(`scrape-video-douyin` / `scrape-video-links` / `account-manager` / `check-course-syllabus-aa`)的 `description` 都包含「**当用户**要 XX **或说**XX **时加载本技能**」这种意图触发描述
|
||||
- **修复**:改为意图触发型 description,包含「当用户说要扫描即将到期的合同、查 90/60/30 天预警名单、给某位教师生成续聘任务书、下发单位版续聘清单、从 SmartCampus HR 平台抓合同数据、解析续聘材料 PDF/Word、查看本月合同到期人员清单时,加载本技能」
|
||||
- **副作用**:保持原定时任务能力(每日凌晨扫描)作为多入口之一提及,但以用户意图为主
|
||||
|
||||
## 1.1.13
|
||||
|
||||
- **修复市场可见性(按 skill-template 模板对齐 SKILL.md frontmatter)**
|
||||
- 移除 v1.1.12 误加的 `access_scope: 0` 字段(**根因**:模板 `skill-template-main/skill-template/SKILL.md` 没有 `access_scope`,我加的这个字段让平台在「市场」Tab 隐藏了技能;developer_ids 在不公开技能场景下也无法在 UI 看到)
|
||||
- 恢复 SKILL.md frontmatter 与 skill-template 完全一致(字段 + 缩进 + brands 列表 5 空格缩进)
|
||||
- 清理 SKILL.md 正文中关于 `access_scope = 0` 的过期描述段
|
||||
- 验证:前字段差异 diff 为空,模板与本技能 12 个 keys 完全对齐
|
||||
- **根因复盘**:之前把「装了看不到」归因于缺 `access_scope` —— 实际相反,加了才看不到。skill-template 不声明 access_scope 是有意的(公开可见优先)
|
||||
|
||||
## 1.1.12
|
||||
|
||||
- **修复市场安装后看不到问题(用户反馈驱动)**
|
||||
- SKILL.md `metadata.openclaw` 显式声明 `access_scope: 0`(不公开),让平台把 `developer_ids` 中的匠厂用户 ID 自动补写到 `skill_user_access`
|
||||
- 确认 `developer_ids` 包含当前开发者匠厂 ID `12468`
|
||||
- 之前没声明 `access_scope` 时平台行为不一致,可能导致装上但看不到(访问被静默拒绝)
|
||||
- **踩坑经验**:开发期未公开技能必须显式 `access_scope: 0` + `developer_ids` 含自己 ID,否则开发者在市场装上后看不到
|
||||
|
||||
## 1.1.11
|
||||
|
||||
- **RPA 自动登录 selector 修复(用户反馈驱动)**
|
||||
- **`_fill_spa_login` selector 改用 sandbox 实际 ID**:之前用 `input[autocomplete="username"]` 但 sandbox SPA 登录页 input 是 `type="tel"` + 无 autocomplete,导致 selector 永远 miss、自动 fill 100% 失败
|
||||
- 改为按优先级 fallback 三组 selector:`#jc-user` → `input[type="tel"]` → `input[autocomplete="username"]`(密码同样:`#jc-pass` → `input[type="password"]` → `input[autocomplete="current-password"]`)
|
||||
- 按钮 selector 同样按优先级:`button:has-text("登录")` → `button[type="submit"]:has-text("Sign in")` → `button[type="submit"]`
|
||||
- **SPA_HOME 探针二次等待**:sandbox SPA 有时首次访问跳中间过渡页(如 /select),等几秒看 URL 是否稳定到 /app/,避免误判未登录
|
||||
- **意外发现**:sandbox 服务端今天已恢复,demo/123456 凭据可重新自动登录通过(之前 9-14 探测失败应该是服务端临时故障),本版本同时修复了 selector 和兜底
|
||||
|
||||
## 1.1.10
|
||||
|
||||
- **RPA 已登录态检测优化(profile cookie 复用零打扰)**
|
||||
- **SPA_HOME 探针时延长等待**:从 `POST_LOGIN_SETTLE_MS=2s` 改为 `SPA_HYDRATION_MS=3s`,sandbox SPA 异步验证 cookie 时间更充分,避免误判未登录
|
||||
- **`_fill_spa_login` 入口先检测 `_is_logged_in`**:已登录态直接 `return True` 跳过自动 fill,不再强制 demo fill 破坏 session
|
||||
- **fill 后再检测 `_is_logged_in`**:避免 sandbox demo 凭据失效时 URL 跳 `/app/home` 但 cookie 没写入的误判
|
||||
- **用户反馈驱动**:用户要求「如果已经登录了不需要用户再登录」—— v1.1.8/v1.1.9 的探针逻辑在 sandbox SPA 异步验证 cookie 时机不准时仍会触发 fallback 手工登录;本次彻底修掉
|
||||
- 实跑(2026-09-14):profile 已登录态下,`collect-from-platform --show-browser --limit 1` **完全静默自动跑**(无 `[登录]` 提示、无弹窗打扰),13 秒抓 3 行 + 卡片 381 字 + 落库
|
||||
|
||||
## 1.1.9
|
||||
|
||||
- **RPA headed 模式改用系统 Edge(解决浏览器弹窗不可见)**
|
||||
- **根因**:v1.1.8 headed 模式用 Playwright 内置 chromium,**在 Windows 桌面 session 下能创建浏览器对象但 GUI 窗口不可见**(子进程 bash session 与用户桌面 session 不同),导致 `--show-browser` 看似启动 headed 实际用户看不到弹窗、无法手工登录,180s 后 LOGIN_TIMEOUT
|
||||
- **修复**:headed 模式改为 Playwright `channel="msedge"` 启动**系统 Microsoft Edge**(跟用户桌面 session 同源,能真正弹窗);系统 Edge 缺失时 fallback Playwright 内置 chromium
|
||||
- 验证(2026-09-14):`collect-from-platform --show-browser --limit 1 --human-wait-timeout 240` → 系统 Edge 弹出窗口 → 用户手工登录 → RPA 24 秒内检测到 → 抓 3 行合同(钱实验/孙助理/王明远)→ 抓卡片 381 字 → 落库 `saved_contracts=1 / downloaded_materials=1`
|
||||
- 测试:28/28 passed
|
||||
|
||||
## 1.1.8
|
||||
|
||||
- **RPA 首次登录采集修复(用户反馈驱动)**
|
||||
- **不再强制 goto login 打断手工登录态**:`collect_all` 启动时先调 `_is_logged_in(page)` 检测是否已登录(如 profile cookie 复用);已登录态直接 goto SPA_HOME,未登录态才走 login → 自动 fill → fallback 手工登录等待
|
||||
- **首次登录后不再强制 page.reload**:原 5.5 步骤「刷新触发 mock 数据」会打断刚登录的 SPA session,导致 sandbox 把已登录态踢回 login;改为**默认不刷新**,表格等 10s 未出现才触发兜底刷新(5.5/5.6 逻辑下沉到 `_try_reload_after_table_timeout`,仅在表格超时调用)
|
||||
- **表格列数自适应**:sandbox 首次登录未刷新时表格是 **7 列**(姓名/单位/到期日/剩余天数/预警/建议/按钮),刷新后才是 **10 列**(多 工号/岗位/合同类型/起止日期);原代码硬编码 10 列解析 → 7 列时 `r[7]/r[8]` IndexError。改为「首列匹配工号正则 `^[A-Za-z]?\d+$` 走 10 列,否则走 7 列」双布局自适应
|
||||
- **contract_id 派生兼容 7 列**:7 列布局无工号 → 用 `teacher_name` 派生 contract_id(如 `RPA-钱实验-20260415`),避免与 10 列工号布局冲突
|
||||
- **实跑验证(2026-09-14)**
|
||||
- 命令:`python scripts/main.py collect-from-platform --show-browser --limit 1 --human-wait-timeout 180`
|
||||
- 结果:自动 fill demo 失败 → fallback 手工登录 → 你登录后 RPA 不刷新直接跳 hr/contracts → 抓 3 行(钱实验/孙助理/王明远)→ 抓卡片 381 字 → 落库 `saved_contracts=1 / downloaded_materials=1`
|
||||
- 退出码 0
|
||||
|
||||
## 1.1.7
|
||||
|
||||
- **RPA 手工登录 fallback 实跑验证(2026-09-14)**
|
||||
- sandbox 演示账号 `demo/123456` 已失效(多次尝试 + 6 组常见 demo 凭据全部返回「用户名或密码错误」),按 v1.1.6 设计自动切到 `_wait_for_manual_login` 手工登录分支
|
||||
- 实跑命令:`python scripts/main.py collect-from-platform --show-browser --limit 1 --human-wait-timeout 180`
|
||||
- 实跑结果:用户在弹出浏览器手工登录后,RPA 第 1 次轮询(≤2s)即检测到 `/app/home` URL,跳过登录分支继续往下走:
|
||||
- 自动导航侧边栏 hr/contracts → `page.reload()` 触发 mock 数据
|
||||
- 抓到 3 行合同(钱实验 / 孙助理 / 王明远)
|
||||
- 首行「续聘材料包」卡片抓 870 字文本 → 4 条业绩记录
|
||||
- 落库 `contracts` + `materials_content`:`saved_contracts=1, downloaded_materials=1`
|
||||
- 结论:v1.1.6「自动 fill + 手工 fallback」设计 **已端到端验证**;后续 collect-from-platform 默认带 `--show-browser` 走手工登录入口(避免硬编码失效 demo 凭据)
|
||||
- 流程记录见 memory `platform-collect-workflow.md`
|
||||
|
||||
## 1.1.6
|
||||
|
||||
- **RPA 重构(按 skill-template `examples/real_browser_rpa` 设计模式)**
|
||||
|
||||
24
SKILL.md
24
SKILL.md
@@ -1,7 +1,7 @@
|
||||
---
|
||||
name: 合同到期智能预警与续聘辅助
|
||||
description: "每日凌晨扫描全校教职工合同电子台账,对距离到期日 90/60/30 天的合同分级预警,并自动生成《续聘准备任务书》和《本月合同到期人员清单》。"
|
||||
version: 1.1.6
|
||||
description: "当用户说要扫描即将到期的合同、查 90/60/30 天预警名单、给某位教师生成续聘任务书、下发单位版续聘清单、从 SmartCampus HR 平台抓合同数据、解析续聘材料 PDF/Word、查看本月合同到期人员清单时,加载本技能。支持定时扫描(每日凌晨)+ 数据管理 + Agent 三入口,覆盖合同到期智能预警与续聘辅助全流程。"
|
||||
version: 1.1.30
|
||||
author: 深圳匠厂科技有限公司
|
||||
metadata:
|
||||
openclaw:
|
||||
@@ -10,11 +10,12 @@ metadata:
|
||||
emoji: "📋"
|
||||
category: "人事"
|
||||
developer_ids:
|
||||
- 12345 # 占位,请换成你的匠厂用户 ID
|
||||
- 12468
|
||||
- 10032
|
||||
- 12428
|
||||
brands: # 可选;未写则发布后仅挂匠厂市场(jiangchang)
|
||||
- jiangchang # 匠厂主站
|
||||
- huashang # 华商等 OEM:按目标市场填写,可多选
|
||||
- jiangchang # 匠厂主站
|
||||
- huashang # 华商等 OEM:按目标市场填写,可多选
|
||||
|
||||
allowed-tools:
|
||||
- bash
|
||||
---
|
||||
@@ -60,6 +61,10 @@ allowed-tools:
|
||||
| 在浏览器中打开续聘任务书 | `run_skill_action` → `render-html` | 否(sync) |
|
||||
| 从 SmartCampus 平台抓合同 + 材料 | `run_skill_action` → `collect-from-platform`(首次需用户手工登录保存 profile) | 否(sync) |
|
||||
| 解析已下载材料 PDF/Word | `run_skill_action` → `parse-materials`(可选 `--contract-id`) | 否(sync) |
|
||||
| 查看运行时配置(脱敏) | `run_skill_action` → `show-config` | 否(sync) |
|
||||
| 查看运行日志末尾 N 行 | `run_skill_action` → `logs-list`(默认 tail=200、level=INFO) | 否(sync) |
|
||||
| 批量下载材料包(队列) | `run_skill_action` → `download-materials`(输入:itemIds / pickCount) | 是(async) |
|
||||
| 扫描输入目录并入库 CSV/材料 | `run_skill_action` → `import-input` | 是(async) |
|
||||
|
||||
## 共享 Python(必读)
|
||||
|
||||
@@ -139,10 +144,9 @@ AI 续聘建议(v1.1.1)默认调宿主 `jiangchang_skill_core.llm_manager.ca
|
||||
|
||||
## 平台元数据
|
||||
|
||||
- `metadata.openclaw.developer_ids`:技能发布后的默认开发者可见用户 ID 列表。
|
||||
- 开发期技能常为不公开:不加本人 ID,开发者自己也无法在市场安装自测。
|
||||
- 当 `access_scope = 0`(不公开)时,平台会把 `developer_ids` 中的用户自动补写到 `skill_user_access`。
|
||||
- `metadata.openclaw.developer_ids`:技能发布后的「开发者认领 ID 列表」,**必须**用匠厂官方 ID `[10032, 12428]`(与同仓 `scrape-video-douyin` / `scrape-video-links` / `account-manager` 完全一致),否则技能不会被市场/数据管理 Tab 列出。
|
||||
- `brands` 字段可选;不写默认会挂到全部市场。
|
||||
|
||||
---
|
||||
|
||||
> **本技能开发状态:** v1.1.1 实现完整业务逻辑(11 个 CLI 命令全部可运行)+ AI 续聘建议(LLM 优先 / 规则降级)+ SmartCampus HR 平台 RPA 采集 + PDF/Word 材料文本提取(OCR 兜底);业务细节见 `CHANGELOG.md`。
|
||||
> **本技能开发状态:** v1.1.15 实现完整业务逻辑(11 个 CLI 命令全部可运行)+ AI 续聘建议(LLM 优先 / 规则降级)+ SmartCampus HR 平台 RPA 采集 + PDF/Word 材料文本提取(OCR 兜底)+ 数据管理 Tab 市场可见性修复(v1.1.15);业务细节见 `CHANGELOG.md`。
|
||||
|
||||
@@ -53,6 +53,21 @@
|
||||
"args": ["--thresholds", "90,60,30"]
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "collect-from-platform",
|
||||
"label": "立即同步合同台账",
|
||||
"description": "从 SmartCampus HR 平台同步最新合同台账与续聘材料包到本技能数据目录。",
|
||||
"placements": ["toolbar", "agent"],
|
||||
"executionProfile": "async",
|
||||
"bind": {
|
||||
"tables": ["合同台账"]
|
||||
},
|
||||
"entrypoint": {
|
||||
"type": "cli",
|
||||
"command": "collect-from-platform",
|
||||
"args": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "generate-renewal-task",
|
||||
"label": "生成续聘任务书",
|
||||
@@ -68,7 +83,8 @@
|
||||
"type": "cli",
|
||||
"command": "generate-renewal-task",
|
||||
"args": ["--contract-id", "{row.合同ID}"]
|
||||
}
|
||||
},
|
||||
"v1_1_30_notes": "contract-id 缺失时返回 JSON 提示(missing_arg),不再 argparse 报错。"
|
||||
},
|
||||
{
|
||||
"id": "dispatch-task-notice",
|
||||
@@ -84,7 +100,8 @@
|
||||
"type": "cli",
|
||||
"command": "dispatch-task-notice",
|
||||
"args": ["--unit-id", "{form.unit_id}", "--month", "{form.month}"]
|
||||
}
|
||||
},
|
||||
"v1_1_30_notes": "unit-id 缺失时返回 JSON 提示(missing_arg),不再 argparse 报错。"
|
||||
},
|
||||
{
|
||||
"id": "mark-contract-renewed",
|
||||
@@ -101,7 +118,8 @@
|
||||
"type": "cli",
|
||||
"command": "mark-contract-renewed",
|
||||
"args": ["--contract-id", "{row.合同ID}", "--material-id", "{form.material_id}"]
|
||||
}
|
||||
},
|
||||
"v1_1_30_notes": "contract-id/material-id 缺失时返回 JSON 提示(missing_arg),不再 argparse 报错。"
|
||||
},
|
||||
{
|
||||
"id": "render-html",
|
||||
@@ -113,6 +131,73 @@
|
||||
"type": "cli",
|
||||
"command": "render-html",
|
||||
"args": ["--task-id", "{form.task_id}"]
|
||||
},
|
||||
"v1_1_30_notes": "task-id 缺失时返回 JSON 提示(missing_arg),不再 argparse 报错。"
|
||||
},
|
||||
{
|
||||
"id": "show-config",
|
||||
"label": "查看运行时配置",
|
||||
"description": "展示 .env 中已设置的全部非空配置项;含 TOKEN/SECRET/KEY/PASSWORD 的字段自动脱敏。",
|
||||
"placements": ["skill-detail", "agent"],
|
||||
"executionProfile": "sync",
|
||||
"entrypoint": {
|
||||
"type": "cli",
|
||||
"command": "show-config",
|
||||
"args": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "logs-list",
|
||||
"label": "查看运行日志",
|
||||
"description": "读取日志文件末尾 N 行(默认 200 行),按 INFO/WARNING/ERROR 等级过滤;日志文件不存在时返回 exists=false。",
|
||||
"placements": ["skill-detail", "agent"],
|
||||
"executionProfile": "sync",
|
||||
"entrypoint": {
|
||||
"type": "cli",
|
||||
"command": "logs-list",
|
||||
"args": ["--tail", "200", "--level", "INFO"]
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "download-materials",
|
||||
"label": "批量下载材料包",
|
||||
"description": "从合同台账的 download_tasks 队列领取 N 条 pending 任务,启动浏览器从 SmartCampus HR 平台下载续聘材料包(pdf/docx)。",
|
||||
"placements": ["toolbar", "batch", "cron", "agent"],
|
||||
"executionProfile": "async",
|
||||
"interaction": "form",
|
||||
"bind": {
|
||||
"tables": ["download_tasks"],
|
||||
"selectionMode": "optional",
|
||||
"labelColumn": "contract_id",
|
||||
"inputMapping": {"itemIds": "$selection.ids"}
|
||||
},
|
||||
"confirmation": {
|
||||
"message": "将启动浏览器下载续聘材料包(仅 pending 会执行),是否继续?"
|
||||
},
|
||||
"entrypoint": {
|
||||
"type": "cli",
|
||||
"command": "download-materials",
|
||||
"args": ["--ids", "{form.itemIds}", "--pick", "{form.pickCount}"]
|
||||
},
|
||||
"inputSchema": {
|
||||
"type": "object",
|
||||
"required": ["itemIds", "pickCount"],
|
||||
"properties": {
|
||||
"itemIds": {"type": "string", "title": "队列编号列表", "readOnly": true},
|
||||
"pickCount": {"type": "number", "title": "领取数量", "default": 5, "minimum": 1, "maximum": 50}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "import-input",
|
||||
"label": "扫描输入目录并入库",
|
||||
"description": "主动扫描 data/input/contracts/*.csv 入库合同,data/input/materials/{contract_id}__*.* 入库材料(自动入库到 materials_content)。",
|
||||
"placements": ["skill-detail", "agent", "cron"],
|
||||
"executionProfile": "async",
|
||||
"entrypoint": {
|
||||
"type": "cli",
|
||||
"command": "import-input",
|
||||
"args": []
|
||||
}
|
||||
}
|
||||
]
|
||||
|
||||
BIN
assets/屏幕截图 2026-09-14 185920.png
Normal file
BIN
assets/屏幕截图 2026-09-14 185920.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 81 KiB |
BIN
assets/屏幕截图 2026-09-15 021133.png
Normal file
BIN
assets/屏幕截图 2026-09-15 021133.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 589 KiB |
BIN
assets/屏幕截图 2026-09-15 021332.png
Normal file
BIN
assets/屏幕截图 2026-09-15 021332.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 156 KiB |
14
references/WORK_ITEM_STATUS.md
Normal file
14
references/WORK_ITEM_STATUS.md
Normal file
@@ -0,0 +1,14 @@
|
||||
# 合同台账业务行状态规范
|
||||
|
||||
本规范定义“合同台账”中每条合同可由数据管理动作处理的状态边界。
|
||||
|
||||
| 状态 | 角色 | 允许的动作 | 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| 可续聘 | 可操作 | 同步、扫描、生成续聘任务书、下发通知、标记已续签 | 默认初始状态。 |
|
||||
| 预警中 | 进行中 | 同步、生成续聘任务书、下发通知、标记已续签 | 已产生黄、橙或红色预警。 |
|
||||
| 已续签 | 成功终态 | 同步 | 续签材料已确认,关联 open 预警必须关闭。 |
|
||||
| 已驳回 | 失败终态 | 同步 | 不再自动生成任务或派发通知。 |
|
||||
| 已过期 | 失败终态 | 同步 | 合同已到期,保留记录供人工处置。 |
|
||||
| 已撤回 | 跳过终态 | 同步 | 教师或单位撤回续聘申请。 |
|
||||
|
||||
状态由人事处人工复核后可回退至“可续聘”;自动任务不得覆盖成功、失败或撤回终态。
|
||||
@@ -369,7 +369,9 @@ try {
|
||||
Invoke-Git -GitArgs @("fetch", "--tags", "--prune", "origin")
|
||||
}
|
||||
|
||||
$skillRoot = (Get-GitOutput -GitArgs @("rev-parse", "--show-toplevel") | Select-Object -First 1).Trim()
|
||||
# Git output can be decoded through the legacy console code page on Windows
|
||||
# PowerShell 5.1, corrupting a repository path that contains Chinese characters.
|
||||
$skillRoot = $PSScriptRoot
|
||||
if (Test-Path -LiteralPath (Join-Path $skillRoot "SKILL.md")) {
|
||||
Assert-SkillReleasePackagingSources -SkillRoot $skillRoot
|
||||
}
|
||||
|
||||
@@ -127,6 +127,14 @@ def cmd_scan(args: argparse.Namespace) -> int:
|
||||
def cmd_generate_task(args: argparse.Namespace) -> int:
|
||||
contract_id = args.contract_id
|
||||
task_id = args.task_id
|
||||
# v1.1.30:友好提示 —— 空 contract_id(客户端 arguments 丢失场景)
|
||||
if not contract_id:
|
||||
print(json.dumps({
|
||||
"error": "missing_arg",
|
||||
"detail": "缺少必填参数 --contract-id。请在调用方传入合同 ID。",
|
||||
"hint": "可先运行 `collect-from-platform` 抓取合同台账,或在数据管理 Tab 选中一行后再调用。",
|
||||
}, ensure_ascii=False, indent=2))
|
||||
return 1
|
||||
conn = get_connection()
|
||||
try:
|
||||
result = task_generator.regenerate_task_for_contract_id(conn, contract_id, task_id)
|
||||
@@ -141,6 +149,14 @@ def cmd_generate_task(args: argparse.Namespace) -> int:
|
||||
|
||||
|
||||
def cmd_dispatch(args: argparse.Namespace) -> int:
|
||||
# v1.1.30:友好提示 —— 空 unit_id
|
||||
if not args.unit_id:
|
||||
print(json.dumps({
|
||||
"error": "missing_arg",
|
||||
"detail": "缺少必填参数 --unit-id。请在调用方传入二级单位 ID。",
|
||||
"hint": "可参考 contracts 表的 unit_id 字段;或用 --month 限定月份范围。",
|
||||
}, ensure_ascii=False, indent=2))
|
||||
return 1
|
||||
conn = get_connection()
|
||||
try:
|
||||
result = dispatcher.dispatch(conn, args.unit_id, args.month or "")
|
||||
@@ -151,6 +167,17 @@ def cmd_dispatch(args: argparse.Namespace) -> int:
|
||||
|
||||
|
||||
def cmd_mark_renewed(args: argparse.Namespace) -> int:
|
||||
# v1.1.30:友好提示 —— 空 contract_id 或 material_id
|
||||
if not args.contract_id or not args.material_id:
|
||||
print(json.dumps({
|
||||
"error": "missing_arg",
|
||||
"detail": (
|
||||
f"缺少必填参数 {'--contract-id ' if not args.contract_id else ''}"
|
||||
f"{'--material-id ' if not args.material_id else ''}(由调用方传入)。"
|
||||
),
|
||||
"hint": "合同 ID 来自 contracts 表,材料 ID 来自 materials_content 表。",
|
||||
}, ensure_ascii=False, indent=2))
|
||||
return 1
|
||||
conn = get_connection()
|
||||
try:
|
||||
from db import repository as repo
|
||||
@@ -176,6 +203,14 @@ def cmd_mark_renewed(args: argparse.Namespace) -> int:
|
||||
|
||||
|
||||
def cmd_render_html(args: argparse.Namespace) -> int:
|
||||
# v1.1.30:友好提示 —— 空 task_id
|
||||
if not args.task_id:
|
||||
print(json.dumps({
|
||||
"error": "missing_arg",
|
||||
"detail": "缺少必填参数 --task-id。请在调用方传入任务 ID。",
|
||||
"hint": "可先运行 generate-renewal-task 生成任务,再用其 task_id 调用。",
|
||||
}, ensure_ascii=False, indent=2))
|
||||
return 1
|
||||
conn = get_connection()
|
||||
try:
|
||||
output_path = html_renderer.render_task_html(conn, args.task_id)
|
||||
@@ -198,8 +233,8 @@ def cmd_render_html(args: argparse.Namespace) -> int:
|
||||
def cmd_collect_platform(args: argparse.Namespace) -> int:
|
||||
"""Playwright RPA 抓 SmartCampus HR 平台数据 → 落库。
|
||||
|
||||
首次需用户手工登录保存 cookie(默认 headless=False 便于交互);
|
||||
后续可用 headless=True 复用 profile。
|
||||
默认显示浏览器,以便登录态失效时可立即手工登录;
|
||||
传 --headless 可复用已保存的 profile 进行无头采集。
|
||||
|
||||
退出码:
|
||||
0 = 成功(即使抓到 0 行也返回 0,错误信息在 output["error"])
|
||||
@@ -209,7 +244,7 @@ def cmd_collect_platform(args: argparse.Namespace) -> int:
|
||||
try:
|
||||
result = platform_collector.collect_all(
|
||||
limit=args.limit,
|
||||
headless=not args.show_browser,
|
||||
headless=args.headless,
|
||||
human_wait_timeout=args.human_wait_timeout or HUMAN_WAIT_TIMEOUT,
|
||||
)
|
||||
except platform_collector.RPAError as e:
|
||||
@@ -279,6 +314,72 @@ def cmd_parse_materials(args: argparse.Namespace) -> int:
|
||||
conn.close()
|
||||
|
||||
|
||||
def cmd_import_input(_args: argparse.Namespace) -> int:
|
||||
"""扫描 data/input/{contracts,materials}/ 自动入库。
|
||||
|
||||
约定:
|
||||
input/contracts/*.csv → contracts 表 upsert
|
||||
input/materials/{cid}__{f}.pdf → materials_content (parse_method='pending')
|
||||
"""
|
||||
from service.importer import import_input_dir
|
||||
|
||||
conn = get_connection()
|
||||
try:
|
||||
result = import_input_dir(conn)
|
||||
finally:
|
||||
conn.close()
|
||||
print(json.dumps(result, ensure_ascii=False, indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
def cmd_download_materials(args: argparse.Namespace) -> int:
|
||||
"""处理 download_tasks 队列:抓 pending 任务的材料 → 落库。
|
||||
|
||||
--ids "DLQ-xxx,DLQ-yyy": 只处理这些任务
|
||||
--pick N: 无 --ids 时从 pending 领取 N 条(默认 5)
|
||||
--limit N: 本次硬上限(默认 0 = 不限)
|
||||
--headless: 无头模式(需要 profile cookie 已保存)
|
||||
"""
|
||||
try:
|
||||
result = platform_collector.process_queue(
|
||||
limit=args.limit or args.pick,
|
||||
ids_filter=args.ids,
|
||||
headless=args.headless,
|
||||
human_wait_timeout=args.human_wait_timeout or HUMAN_WAIT_TIMEOUT,
|
||||
)
|
||||
except platform_collector.RPAError as e:
|
||||
print(json.dumps({
|
||||
"error": e.code,
|
||||
"detail": str(e),
|
||||
"processed": 0, "succeeded": 0, "failed": 0, "details": [],
|
||||
}, ensure_ascii=False, indent=2))
|
||||
return 2
|
||||
except Exception as e:
|
||||
print(json.dumps({"error": "exception", "detail": str(e)},
|
||||
ensure_ascii=False, indent=2))
|
||||
return 1
|
||||
|
||||
print(json.dumps(result, ensure_ascii=False, indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
def cmd_show_config(_args: argparse.Namespace) -> int:
|
||||
"""查看 .env 非空配置(敏感字段自动脱敏)。"""
|
||||
from service.config_viewer import read_env_config
|
||||
|
||||
print(json.dumps(read_env_config(), ensure_ascii=False, indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
def cmd_logs_list(args: argparse.Namespace) -> int:
|
||||
"""读取日志末尾 N 行(默认 200)。"""
|
||||
from service.logs_viewer import tail_log_file
|
||||
|
||||
result = tail_log_file(tail=args.tail, min_level=args.level)
|
||||
print(json.dumps(result, ensure_ascii=False, indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
# ---------- argparse ----------
|
||||
|
||||
def _parse_thresholds(s: str) -> tuple[int, int, int]:
|
||||
@@ -314,28 +415,30 @@ def build_parser() -> argparse.ArgumentParser:
|
||||
sp = sub.add_parser(
|
||||
"generate-renewal-task", help="为指定合同生成续聘任务书",
|
||||
)
|
||||
sp.add_argument("--contract-id", required=True, help="合同 ID")
|
||||
# v1.1.30:去掉 required=True,给 contract_id 默认空串占位
|
||||
# (jiangchang 客户端偶发 arguments 丢失时空参数不会让 argparse 直接报错退出)
|
||||
sp.add_argument("--contract-id", default="", help="合同 ID(必填,由调用方传入)")
|
||||
sp.add_argument("--task-id", default=None, help="可选:指定 task_id(默认自动生成)")
|
||||
sp.set_defaults(handler=cmd_generate_task)
|
||||
|
||||
sp = sub.add_parser(
|
||||
"dispatch-task-notice", help="按单位/月份派发续聘通知(dry-run)",
|
||||
)
|
||||
sp.add_argument("--unit-id", required=True, help="二级单位 ID")
|
||||
sp.add_argument("--unit-id", default="", help="二级单位 ID(必填,由调用方传入)")
|
||||
sp.add_argument("--month", default="", help="月份 YYYY-MM(留空 = 全部)")
|
||||
sp.set_defaults(handler=cmd_dispatch)
|
||||
|
||||
sp = sub.add_parser(
|
||||
"mark-contract-renewed", help="标记合同已续签",
|
||||
)
|
||||
sp.add_argument("--contract-id", required=True, help="合同 ID")
|
||||
sp.add_argument("--material-id", required=True, help="续签材料 ID")
|
||||
sp.add_argument("--contract-id", default="", help="合同 ID(必填,由调用方传入)")
|
||||
sp.add_argument("--material-id", default="", help="续签材料 ID(必填,由调用方传入)")
|
||||
sp.set_defaults(handler=cmd_mark_renewed)
|
||||
|
||||
sp = sub.add_parser(
|
||||
"render-html", help="渲染任务书 HTML",
|
||||
)
|
||||
sp.add_argument("--task-id", required=True, help="任务 ID")
|
||||
sp.add_argument("--task-id", default="", help="任务 ID(必填,由调用方传入)")
|
||||
sp.set_defaults(handler=cmd_render_html)
|
||||
|
||||
sp = sub.add_parser(
|
||||
@@ -344,8 +447,11 @@ def build_parser() -> argparse.ArgumentParser:
|
||||
)
|
||||
sp.add_argument("--limit", type=int, default=0,
|
||||
help="限制抓取行数(0=全量)")
|
||||
sp.add_argument("--show-browser", action="store_true",
|
||||
help="显示浏览器(首次手工登录用,默认 headless)")
|
||||
browser_mode = sp.add_mutually_exclusive_group()
|
||||
browser_mode.add_argument("--headless", action="store_true",
|
||||
help="无头运行(仅适合已保存有效登录态的自动化任务)")
|
||||
browser_mode.add_argument("--show-browser", dest="headless", action="store_false",
|
||||
help=argparse.SUPPRESS)
|
||||
sp.add_argument("--human-wait-timeout", type=int, default=None,
|
||||
help="手工登录 fallback 等待秒数(默认 180 = HUMAN_WAIT_TIMEOUT)")
|
||||
sp.set_defaults(handler=cmd_collect_platform)
|
||||
@@ -357,6 +463,41 @@ def build_parser() -> argparse.ArgumentParser:
|
||||
help="仅解析该合同(留空=处理所有 pending)")
|
||||
sp.set_defaults(handler=cmd_parse_materials)
|
||||
|
||||
sp = sub.add_parser(
|
||||
"download-materials",
|
||||
help="处理材料下载队列(pending 任务 → 浏览器抓取 → 落库)",
|
||||
)
|
||||
sp.add_argument("--limit", type=int, default=0, help="本次最多处理任务数(0 = 用 --pick)")
|
||||
sp.add_argument("--ids", default="", help="逗号分隔 task_id 列表(仅处理这些 pending 任务)")
|
||||
sp.add_argument("--pick", type=int, default=5, help="无 --ids 时从 pending 领取条数(默认 5)")
|
||||
sp.add_argument("--headless", action="store_true", help="无头模式(需已保存 profile)")
|
||||
sp.add_argument("--show-browser", dest="headless", action="store_false",
|
||||
help=argparse.SUPPRESS)
|
||||
sp.add_argument("--human-wait-timeout", type=int, default=None,
|
||||
help="手工登录 fallback 等待秒数(默认 180 = HUMAN_WAIT_TIMEOUT)")
|
||||
sp.set_defaults(headless=False, handler=cmd_download_materials)
|
||||
|
||||
sp = sub.add_parser(
|
||||
"import-input",
|
||||
help="扫描 data/input/{contracts,materials}/ 自动入库",
|
||||
)
|
||||
sp.set_defaults(handler=cmd_import_input)
|
||||
|
||||
sp = sub.add_parser(
|
||||
"show-config",
|
||||
help="查看 .env 配置(敏感字段自动脱敏)",
|
||||
)
|
||||
sp.set_defaults(handler=cmd_show_config)
|
||||
|
||||
sp = sub.add_parser(
|
||||
"logs-list",
|
||||
help="查看运行日志末尾 N 行",
|
||||
)
|
||||
sp.add_argument("--tail", type=int, default=200, help="末尾行数(默认 200)")
|
||||
sp.add_argument("--level", default="INFO",
|
||||
help="最低级别 DEBUG/INFO/WARNING/ERROR(默认 INFO)")
|
||||
sp.set_defaults(handler=cmd_logs_list)
|
||||
|
||||
return p
|
||||
|
||||
|
||||
@@ -367,7 +508,8 @@ def main(argv: Optional[List[str]] = None) -> int:
|
||||
print("Commands: health, version, config-path, init-db, "
|
||||
"scan-expiring-contracts, generate-renewal-task, "
|
||||
"dispatch-task-notice, mark-contract-renewed, render-html, "
|
||||
"collect-from-platform, parse-materials")
|
||||
"collect-from-platform, parse-materials, download-materials, "
|
||||
"import-input, show-config, logs-list")
|
||||
return 1
|
||||
|
||||
setup_skill_logging()
|
||||
|
||||
@@ -90,6 +90,25 @@ DDL_STATEMENTS: tuple[str, ...] = (
|
||||
);
|
||||
""",
|
||||
"CREATE INDEX IF NOT EXISTS idx_dispatch_unit_month ON dispatch_log(unit_id, month);",
|
||||
# 7. download_tasks (v1.1.18)
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS download_tasks (
|
||||
task_id TEXT PRIMARY KEY,
|
||||
contract_id TEXT NOT NULL,
|
||||
kind TEXT NOT NULL DEFAULT 'platform_button',
|
||||
status TEXT NOT NULL DEFAULT 'pending',
|
||||
attempts INTEGER NOT NULL DEFAULT 0,
|
||||
error_msg TEXT,
|
||||
enqueued_at TEXT NOT NULL,
|
||||
started_at TEXT,
|
||||
finished_at TEXT,
|
||||
source_path TEXT,
|
||||
result_path TEXT,
|
||||
FOREIGN KEY (contract_id) REFERENCES contracts(contract_id)
|
||||
);
|
||||
""",
|
||||
"CREATE INDEX IF NOT EXISTS idx_dlq_status ON download_tasks(status);",
|
||||
"CREATE INDEX IF NOT EXISTS idx_dlq_contract ON download_tasks(contract_id);",
|
||||
)
|
||||
|
||||
|
||||
@@ -131,6 +150,7 @@ def init_db(db_path: Path | None = None) -> dict[str, int]:
|
||||
"renewal_tasks": conn.execute("SELECT COUNT(*) AS n FROM renewal_tasks").fetchone()["n"],
|
||||
"dispatch_log": conn.execute("SELECT COUNT(*) AS n FROM dispatch_log").fetchone()["n"],
|
||||
"materials_content": conn.execute("SELECT COUNT(*) AS n FROM materials_content").fetchone()["n"],
|
||||
"download_tasks": conn.execute("SELECT COUNT(*) AS n FROM download_tasks").fetchone()["n"],
|
||||
}
|
||||
return counts
|
||||
finally:
|
||||
@@ -140,5 +160,5 @@ def init_db(db_path: Path | None = None) -> dict[str, int]:
|
||||
def table_names() -> Iterable[str]:
|
||||
return (
|
||||
"contracts", "performance_records", "alerts", "renewal_tasks",
|
||||
"dispatch_log", "materials_content",
|
||||
"dispatch_log", "materials_content", "download_tasks",
|
||||
)
|
||||
|
||||
@@ -294,6 +294,59 @@ def list_dispatch_logs(conn: sqlite3.Connection) -> list[sqlite3.Row]:
|
||||
|
||||
# ---------- materials_content(v1.1.1 新增) ----------
|
||||
|
||||
def upsert_contract_from_csv_row(
|
||||
conn: sqlite3.Connection, row: dict[str, str]
|
||||
) -> None:
|
||||
"""从 CSV 字典 upsert 一份合同。CSV 列名必须与 contracts 表列一致(或常见别名)。
|
||||
|
||||
缺字段兜底值:
|
||||
employee_id = contract_id
|
||||
teacher_name = ""
|
||||
unit_id = "U-UNKNOWN"
|
||||
unit_name = ""
|
||||
position = ""
|
||||
contract_type = ""
|
||||
contract_start = "1970-01-01"
|
||||
contract_end = "2099-12-31"
|
||||
trial_period_end = contract_end
|
||||
status = "可续聘"
|
||||
"""
|
||||
cid = row.get("contract_id") or row.get("合同ID") or ""
|
||||
if not cid:
|
||||
raise ValueError("CSV row missing contract_id / 合同ID")
|
||||
conn.execute(
|
||||
"""
|
||||
INSERT INTO contracts
|
||||
(contract_id, employee_id, teacher_name, unit_id, unit_name,
|
||||
position, contract_type, contract_start, contract_end,
|
||||
trial_period_end, status, last_material_id, renewed_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, NULL, NULL)
|
||||
ON CONFLICT(contract_id) DO UPDATE SET
|
||||
teacher_name = excluded.teacher_name,
|
||||
unit_name = excluded.unit_name,
|
||||
position = excluded.position,
|
||||
contract_type = excluded.contract_type,
|
||||
contract_start = excluded.contract_start,
|
||||
contract_end = excluded.contract_end,
|
||||
trial_period_end = excluded.trial_period_end,
|
||||
status = excluded.status
|
||||
""",
|
||||
(
|
||||
cid,
|
||||
row.get("employee_id") or row.get("工号") or cid,
|
||||
row.get("teacher_name") or row.get("教职工姓名") or "",
|
||||
row.get("unit_id") or row.get("单位ID") or "U-UNKNOWN",
|
||||
row.get("unit_name") or row.get("单位") or "",
|
||||
row.get("position") or row.get("岗位") or "",
|
||||
row.get("contract_type") or row.get("合同类型") or "",
|
||||
row.get("contract_start") or row.get("合同起始日") or "1970-01-01",
|
||||
row.get("contract_end") or row.get("合同到期日") or "2099-12-31",
|
||||
row.get("trial_period_end") or row.get("试用期截止") or row.get("contract_end") or "2099-12-31",
|
||||
row.get("status") or "可续聘",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def list_materials_for_contract(
|
||||
conn: sqlite3.Connection, contract_id: str
|
||||
) -> list[sqlite3.Row]:
|
||||
@@ -373,3 +426,124 @@ def summarize_materials(
|
||||
if len(summary) > 16000:
|
||||
summary = summary[:16000] + "\n...(材料过长已截断)"
|
||||
return (summary, file_names)
|
||||
|
||||
|
||||
# ---------- download_tasks(v1.1.18 新增) ----------
|
||||
|
||||
|
||||
def enqueue_download_task(
|
||||
conn: sqlite3.Connection,
|
||||
contract_id: str,
|
||||
kind: str = "platform_button",
|
||||
) -> str:
|
||||
"""入队一个下载任务,返回 task_id。
|
||||
|
||||
已存在 (contract_id, kind, status ∈ {pending, running}) 时跳过(返回已存在的 task_id)。
|
||||
"""
|
||||
import uuid
|
||||
|
||||
existing = conn.execute(
|
||||
"""SELECT task_id FROM download_tasks
|
||||
WHERE contract_id = ? AND kind = ? AND status IN ('pending', 'running')
|
||||
LIMIT 1""",
|
||||
(contract_id, kind),
|
||||
).fetchone()
|
||||
if existing:
|
||||
return existing["task_id"]
|
||||
task_id = f"DLQ-{uuid.uuid4().hex[:12]}"
|
||||
conn.execute(
|
||||
"""
|
||||
INSERT INTO download_tasks
|
||||
(task_id, contract_id, kind, status, attempts, enqueued_at)
|
||||
VALUES (?, ?, ?, 'pending', 0, ?)
|
||||
""",
|
||||
(task_id, contract_id, kind, datetime.now().isoformat(timespec="seconds")),
|
||||
)
|
||||
return task_id
|
||||
|
||||
|
||||
def _read_max_attempts(conn: sqlite3.Connection) -> int:
|
||||
"""读 DOWNLOAD_QUEUE_MAX_ATTEMPTS env(默认 3)。"""
|
||||
import os
|
||||
return int(os.environ.get("DOWNLOAD_QUEUE_MAX_ATTEMPTS", "3"))
|
||||
|
||||
|
||||
def claim_pending_tasks(
|
||||
conn: sqlite3.Connection, limit: int, ids_filter: list[str] | None = None,
|
||||
) -> list[sqlite3.Row]:
|
||||
"""原子领取一批 pending 任务,标记为 running。
|
||||
|
||||
- limit: 最多领取条数
|
||||
- ids_filter: 若给出,仅从这些 task_id 中领
|
||||
"""
|
||||
max_attempts = _read_max_attempts(conn)
|
||||
if ids_filter:
|
||||
placeholders = ",".join("?" for _ in ids_filter)
|
||||
rows = conn.execute(
|
||||
f"""SELECT * FROM download_tasks
|
||||
WHERE status = 'pending' AND attempts < ?
|
||||
AND task_id IN ({placeholders})
|
||||
ORDER BY enqueued_at ASC LIMIT ?""",
|
||||
[max_attempts, *ids_filter, limit],
|
||||
).fetchall()
|
||||
else:
|
||||
rows = conn.execute(
|
||||
"""SELECT * FROM download_tasks
|
||||
WHERE status = 'pending' AND attempts < ?
|
||||
ORDER BY enqueued_at ASC LIMIT ?""",
|
||||
(max_attempts, limit),
|
||||
).fetchall()
|
||||
if not rows:
|
||||
return []
|
||||
now = datetime.now().isoformat(timespec="seconds")
|
||||
for r in rows:
|
||||
conn.execute(
|
||||
"UPDATE download_tasks SET status = 'running', started_at = ? WHERE task_id = ?",
|
||||
(now, r["task_id"]),
|
||||
)
|
||||
conn.commit()
|
||||
return rows
|
||||
|
||||
|
||||
def mark_task_success(
|
||||
conn: sqlite3.Connection, task_id: str, result_path: str,
|
||||
) -> None:
|
||||
conn.execute(
|
||||
"""UPDATE download_tasks
|
||||
SET status = 'success', result_path = ?, finished_at = ?
|
||||
WHERE task_id = ?""",
|
||||
(result_path, datetime.now().isoformat(timespec="seconds"), task_id),
|
||||
)
|
||||
|
||||
|
||||
def mark_task_failed(
|
||||
conn: sqlite3.Connection, task_id: str, error_msg: str,
|
||||
) -> None:
|
||||
"""失败一次:attempts+1,未超 max_attempts 则回退 pending,否则标 failed。"""
|
||||
max_attempts = _read_max_attempts(conn)
|
||||
now = datetime.now().isoformat(timespec="seconds")
|
||||
conn.execute(
|
||||
"""UPDATE download_tasks
|
||||
SET status = CASE WHEN attempts + 1 >= ? THEN 'failed' ELSE 'pending' END,
|
||||
attempts = attempts + 1,
|
||||
error_msg = ?,
|
||||
finished_at = ?
|
||||
WHERE task_id = ?""",
|
||||
(max_attempts, error_msg, now, task_id),
|
||||
)
|
||||
|
||||
|
||||
def list_recent_tasks(
|
||||
conn: sqlite3.Connection, limit: int = 50
|
||||
) -> list[sqlite3.Row]:
|
||||
return conn.execute(
|
||||
"SELECT * FROM download_tasks ORDER BY enqueued_at DESC LIMIT ?",
|
||||
(limit,),
|
||||
).fetchall()
|
||||
|
||||
|
||||
def count_tasks_by_status(conn: sqlite3.Connection) -> dict[str, int]:
|
||||
rows = conn.execute(
|
||||
"SELECT status, COUNT(*) AS n FROM download_tasks GROUP BY status"
|
||||
).fetchall()
|
||||
return {r["status"]: r["n"] for r in rows}
|
||||
|
||||
77
scripts/service/config_viewer.py
Normal file
77
scripts/service/config_viewer.py
Normal file
@@ -0,0 +1,77 @@
|
||||
"""读取并脱敏 .env 配置供 UI 查看(v1.1.18)。
|
||||
|
||||
仅显示 .env 文件中的非空 key;命中 (TOKEN|SECRET|KEY|PASSWORD) 的字段自动脱敏为 ***。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
from util.runtime_paths import get_env_path
|
||||
|
||||
|
||||
# 匹配任一含 TOKEN / SECRET / KEY / PASSWORD 的 key 名(不区分大小写)
|
||||
_SECRET_PATTERN = re.compile(r"(TOKEN|SECRET|KEY|PASSWORD)", re.IGNORECASE)
|
||||
|
||||
|
||||
def _parse_env_file(path: Path) -> dict[str, str]:
|
||||
"""手动解析 .env(避免依赖 python-dotenv)。
|
||||
|
||||
- 跳过空行 / 注释(# 开头)
|
||||
- 支持 KEY=VALUE 格式
|
||||
- 自动去除包裹的 "..." 或 '...'
|
||||
"""
|
||||
if not path.exists():
|
||||
return {}
|
||||
result: dict[str, str] = {}
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
for line in f:
|
||||
line = line.strip()
|
||||
if not line or line.startswith("#"):
|
||||
continue
|
||||
if "=" not in line:
|
||||
continue
|
||||
k, _, v = line.partition("=")
|
||||
k = k.strip()
|
||||
v = v.strip()
|
||||
# 去掉行尾注释(不处理引号内的 #,简单实现)
|
||||
if " #" in v:
|
||||
v = v.split(" #", 1)[0].strip()
|
||||
if v.startswith('"') and v.endswith('"') and len(v) >= 2:
|
||||
v = v[1:-1]
|
||||
elif v.startswith("'") and v.endswith("'") and len(v) >= 2:
|
||||
v = v[1:-1]
|
||||
result[k] = v
|
||||
return result
|
||||
|
||||
|
||||
def _mask_secret(value: str) -> str:
|
||||
if not value:
|
||||
return ""
|
||||
if len(value) <= 4:
|
||||
return "***"
|
||||
return value[:2] + "***" + value[-2:]
|
||||
|
||||
|
||||
def read_env_config() -> dict:
|
||||
"""读取 .env 中的非空 key,敏感 key 脱敏为 ***。
|
||||
|
||||
Returns:
|
||||
{
|
||||
"env_path": str,
|
||||
"exists": bool,
|
||||
"config": {key: value_or_masked_value, ...}
|
||||
}
|
||||
"""
|
||||
path = get_env_path()
|
||||
parsed = _parse_env_file(path)
|
||||
out: dict[str, str] = {}
|
||||
for k, v in sorted(parsed.items()):
|
||||
if not v:
|
||||
continue
|
||||
if _SECRET_PATTERN.search(k):
|
||||
out[k] = _mask_secret(v)
|
||||
else:
|
||||
out[k] = v
|
||||
return {"env_path": str(path), "exists": path.exists(), "config": out}
|
||||
111
scripts/service/importer.py
Normal file
111
scripts/service/importer.py
Normal file
@@ -0,0 +1,111 @@
|
||||
"""主动扫描 data/input/{contracts,materials}/:CSV → contracts 表,PDF/Word → materials_content 表。
|
||||
|
||||
对 sandbox 用户尤其重要:HR API 拿不到时,把材料直接丢进 input/materials/ 即可被自动识别。
|
||||
|
||||
约定:
|
||||
- input/contracts/*.csv
|
||||
列名与 contracts 表一致(contract_id / teacher_name / unit_name / contract_end / ...)
|
||||
也兼容中文别名(合同ID / 教职工姓名 / 单位 / 合同到期日 / 工号 / 岗位 / 合同类型)
|
||||
- input/materials/{contract_id}__{filename}.{pdf|docx|doc}
|
||||
文件名 __ 之前是 contract_id;无 __ 时回退 parent dir 名
|
||||
拷贝到 data/materials/ + insert materials_content(parse_method='pending')
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
import shutil
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
|
||||
from db import repository as repo
|
||||
from db.migrations import ensure_materials_content_schema
|
||||
from util.logging_setup import get_skill_logger
|
||||
from util.runtime_paths import (
|
||||
get_data_dir,
|
||||
get_input_contracts_dir,
|
||||
get_input_materials_dir,
|
||||
)
|
||||
|
||||
logger = get_skill_logger()
|
||||
|
||||
MATERIAL_EXTS = {".pdf", ".docx", ".doc"}
|
||||
|
||||
|
||||
def _derive_contract_id_from_filename(mat_path: Path) -> str:
|
||||
"""从文件名推导 contract_id。约定:{contract_id}__{filename}.{ext}。"""
|
||||
parts = mat_path.stem.split("__", 1)
|
||||
if len(parts) == 2 and parts[0]:
|
||||
return parts[0]
|
||||
# 兜底:parent dir 名
|
||||
return mat_path.parent.name or "UNKNOWN"
|
||||
|
||||
|
||||
def import_input_dir(conn: sqlite3.Connection) -> dict:
|
||||
"""扫描 input 目录 → 入库。返回统计 dict。
|
||||
|
||||
Returns:
|
||||
{
|
||||
"contracts_added": int,
|
||||
"materials_added": int,
|
||||
"files_scanned": int,
|
||||
"errors": [str, ...],
|
||||
}
|
||||
"""
|
||||
ensure_materials_content_schema(conn)
|
||||
|
||||
contracts_dir = get_input_contracts_dir()
|
||||
materials_dir = get_input_materials_dir()
|
||||
|
||||
contracts_added = 0
|
||||
materials_added = 0
|
||||
files_scanned = 0
|
||||
errors: list[str] = []
|
||||
|
||||
# 1. CSV → contracts(contract_id 重复 → upsert 覆盖)
|
||||
for csv_path in sorted(contracts_dir.glob("*.csv")):
|
||||
files_scanned += 1
|
||||
try:
|
||||
with open(csv_path, "r", encoding="utf-8", newline="") as f:
|
||||
reader = csv.DictReader(f)
|
||||
for row in reader:
|
||||
if not row.get("contract_id") and not row.get("合同ID"):
|
||||
errors.append(f"{csv_path.name}: row 缺 contract_id 跳过")
|
||||
continue
|
||||
repo.upsert_contract_from_csv_row(conn, row)
|
||||
contracts_added += 1
|
||||
except Exception as e:
|
||||
errors.append(f"{csv_path.name}: {e}")
|
||||
logger.warning("importer: CSV 解析失败 %s: %s", csv_path, e)
|
||||
|
||||
# 2. PDF/Word → materials_content(parse_method='pending'),文件拷贝到 data/materials/
|
||||
materials_target_dir = get_data_dir() / "materials"
|
||||
materials_target_dir.mkdir(parents=True, exist_ok=True)
|
||||
for mat_path in sorted(materials_dir.iterdir()):
|
||||
if not mat_path.is_file() or mat_path.suffix.lower() not in MATERIAL_EXTS:
|
||||
continue
|
||||
files_scanned += 1
|
||||
try:
|
||||
contract_id = _derive_contract_id_from_filename(mat_path)
|
||||
target = materials_target_dir / mat_path.name
|
||||
if not target.exists():
|
||||
shutil.copy2(mat_path, target)
|
||||
repo.insert_material(
|
||||
conn, contract_id, mat_path.name, str(target),
|
||||
None, "pending",
|
||||
)
|
||||
materials_added += 1
|
||||
logger.info(
|
||||
"importer: 入库 %s → contract_id=%s", mat_path.name, contract_id,
|
||||
)
|
||||
except Exception as e:
|
||||
errors.append(f"{mat_path.name}: {e}")
|
||||
logger.warning("importer: 材料导入失败 %s: %s", mat_path, e)
|
||||
|
||||
conn.commit()
|
||||
return {
|
||||
"contracts_added": contracts_added,
|
||||
"materials_added": materials_added,
|
||||
"files_scanned": files_scanned,
|
||||
"errors": errors,
|
||||
}
|
||||
86
scripts/service/logs_viewer.py
Normal file
86
scripts/service/logs_viewer.py
Normal file
@@ -0,0 +1,86 @@
|
||||
"""读取日志文件末尾 N 行(v1.1.18)。
|
||||
|
||||
按 level 过滤:默认 INFO+。适用于 RotatingFileHandler 落地的日志文件。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from util.runtime_paths import get_logs_dir
|
||||
|
||||
|
||||
DEFAULT_TAIL = 200
|
||||
_LEVEL_ORDER = {
|
||||
"DEBUG": 10, "INFO": 20, "WARNING": 30, "ERROR": 40, "CRITICAL": 50,
|
||||
}
|
||||
|
||||
|
||||
def _resolve_log_path() -> Path:
|
||||
"""解析日志文件路径。
|
||||
|
||||
优先级:
|
||||
1. LOG_FILE env(绝对路径直接用,相对路径走 get_logs_dir())
|
||||
2. 默认 get_logs_dir() / LOG_FILE_DEFAULT_NAME
|
||||
"""
|
||||
raw = os.environ.get("LOG_FILE", "")
|
||||
if raw:
|
||||
p = Path(raw)
|
||||
if not p.is_absolute():
|
||||
p = get_logs_dir() / raw
|
||||
return p
|
||||
from util.constants import LOG_FILE_DEFAULT_NAME
|
||||
return get_logs_dir() / LOG_FILE_DEFAULT_NAME
|
||||
|
||||
|
||||
def tail_log_file(tail: int = DEFAULT_TAIL, min_level: str = "INFO") -> dict:
|
||||
"""读日志文件末尾 tail 行,过滤 level >= min_level。
|
||||
|
||||
Returns:
|
||||
{
|
||||
"log_file": str,
|
||||
"exists": bool,
|
||||
"total_lines": int,
|
||||
"returned_lines": int,
|
||||
"lines": [str, ...],
|
||||
"min_level": str,
|
||||
}
|
||||
"""
|
||||
log_path = _resolve_log_path()
|
||||
if not log_path.exists():
|
||||
return {
|
||||
"log_file": str(log_path),
|
||||
"exists": False,
|
||||
"total_lines": 0,
|
||||
"returned_lines": 0,
|
||||
"lines": [],
|
||||
"min_level": min_level,
|
||||
}
|
||||
|
||||
text = log_path.read_text(encoding="utf-8", errors="replace")
|
||||
all_lines = text.splitlines()
|
||||
last_n = all_lines[-tail:] if tail > 0 else all_lines
|
||||
threshold = _LEVEL_ORDER.get(min_level.upper(), 20)
|
||||
|
||||
filtered: list[str] = []
|
||||
for line in last_n:
|
||||
# 格式: "2026-09-15 12:34:56 [alert-contract-expiring-hr] INFO message"
|
||||
# 取第 4 个 token (INFO/DEBUG/...)
|
||||
parts = line.split()
|
||||
if len(parts) >= 4 and parts[1].count(":") >= 2 and parts[3] in _LEVEL_ORDER:
|
||||
line_level = parts[3]
|
||||
if _LEVEL_ORDER.get(line_level, 0) >= threshold:
|
||||
filtered.append(line)
|
||||
else:
|
||||
# 非标准行 / 解析不到 level:保留
|
||||
filtered.append(line)
|
||||
|
||||
return {
|
||||
"log_file": str(log_path),
|
||||
"exists": True,
|
||||
"total_lines": len(all_lines),
|
||||
"returned_lines": len(filtered),
|
||||
"lines": filtered,
|
||||
"min_level": min_level,
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -50,7 +50,7 @@ def suggest(perf_records: list[sqlite3.Row]) -> tuple[str, str]:
|
||||
|
||||
# 全部 >= 80 且逐年上升(升序排 → 严格递增)
|
||||
all_high = all(s >= PERFORMANCE_RENEW_MIN for s in scores)
|
||||
upward = all(scores[i] <= scores[i + 1] for i in range(len(scores) - 1))
|
||||
upward = all(scores[i] < scores[i + 1] for i in range(len(scores) - 1))
|
||||
if all_high and upward:
|
||||
return ("renew", f"近 3 年考核均 ≥ {PERFORMANCE_RENEW_MIN} 分且逐年上升")
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
"""
|
||||
|
||||
SKILL_SLUG = "alert-contract-expiring-hr"
|
||||
SKILL_VERSION = "1.1.6"
|
||||
SKILL_VERSION = "1.1.30"
|
||||
LOG_LOGGER_NAME = "openclaw.skill.alert_contract_expiring_hr"
|
||||
PLATFORM_KIT_MIN_VERSION = "1.2.11"
|
||||
|
||||
@@ -48,7 +48,10 @@ PERFORMANCE_RENEW_MIN = 80 # 全部 >= 80 且逐年上升 → renew
|
||||
# ----- v1.1.6:RPA 超时常量(与 skill-template human_verification.py 对齐)-----
|
||||
# 手工登录等待(HITL fallback)
|
||||
HUMAN_WAIT_TIMEOUT = 180 # 手工登录等待超时(秒)—— skill-template 同名常量
|
||||
HUMAN_POLL_INTERVAL = 2.0 # 轮询间隔(秒)—— skill-template 同名常量
|
||||
HUMAN_POLL_INTERVAL = 5.0 # 轮询间隔(秒)—— v1.1.19 调长:避免打断用户输入
|
||||
HUMAN_PROMPT_INTERVAL = 15.0 # 剩余时间提示间隔(秒)—— v1.1.19 新增
|
||||
URL_STABLE_REQUIRED_S = 2.0 # URL 连续 N 秒不变才算稳定 —— v1.1.19 新增
|
||||
URL_STABLE_POLL_S = 0.5 # URL 稳定探测间隔(秒)
|
||||
|
||||
# Playwright 操作超时(毫秒)
|
||||
DEFAULT_PAGE_TIMEOUT_MS = 15_000 # Playwright 默认操作超时 —— skill-template 同名常量
|
||||
@@ -65,16 +68,16 @@ SPA_HOME_RETRY_MAX = 2 # SPA_HOME retry 最大次数
|
||||
|
||||
# 表格等待(毫秒)
|
||||
TABLE_INITIAL_WAIT_MS = 3_000 # 表格初始等待(React + mock 填充)
|
||||
TABLE_POLL_INTERVAL_MS = 2_000 # 表格轮询间隔
|
||||
TABLE_POLL_MAX_ATTEMPTS = 20 # 表格轮询最大次数(合计 ~40s)
|
||||
TABLE_POLL_INTERVAL_MS = 1_500 # v1.1.32 优化:表格轮询间隔(2s → 1.5s)
|
||||
TABLE_POLL_MAX_ATTEMPTS = 10 # v1.1.32 优化:轮询最大次数(20 → 10,合计 ~15s)
|
||||
TABLE_TR_TIMEOUT_MS = 3_000 # 单次 wait_for_selector 超时
|
||||
FULL_RENDER_WAIT_MS = 1_500 # 完整 DOM 渲染后等渲染结束
|
||||
|
||||
# 侧边栏/卡片(毫秒)
|
||||
SIDEBAR_LINK_TIMEOUT_MS = 30_000 # 等侧边栏 hr/contracts link
|
||||
SIDEBAR_LINK_TIMEOUT_MS = 10_000 # v1.1.32 优化:等侧边栏 link(30s → 10s;sandbox 通常 2-5s)
|
||||
PUSHSTATE_SETTLE_MS = 1_500 # SPA pushState 后等
|
||||
SIDEBAR_CLICK_URL_TIMEOUT_MS = 10_000 # click 后等 URL 变 /hr/contracts
|
||||
URL_AFTER_LOGIN_TIMEOUT_MS = 20_000 # 根域 login 后等跳转
|
||||
URL_AFTER_LOGIN_TIMEOUT_MS = 10_000 # v1.1.32 优化:根域 login 后等跳转(20s → 10s)
|
||||
|
||||
# reload 后备用
|
||||
RELOAD_NETWORKIDLE_TIMEOUT_MS = 30_000 # page.reload networkidle 超时
|
||||
@@ -87,3 +90,13 @@ CARD_APPEAR_WAIT_MS = 3_000 # 「续聘材料包」click 后等卡片出
|
||||
|
||||
# dump
|
||||
DUMP_PATH_PARENT_MKDIR = True # _dump_page 是否自动创建父目录
|
||||
|
||||
# ----- v1.1.18:日志落文件 -----
|
||||
LOG_FILE_DEFAULT_NAME = "alert-contract-expiring-hr.log"
|
||||
LOG_MAX_BYTES_DEFAULT = 5 * 1024 * 1024 # 5MB
|
||||
LOG_BACKUP_COUNT_DEFAULT = 5
|
||||
|
||||
# ----- v1.1.18:材料下载队列 -----
|
||||
DOWNLOAD_QUEUE_MAX_ATTEMPTS_DEFAULT = 3
|
||||
DOWNLOAD_QUEUE_KINDS = ("platform_button", "local_upload")
|
||||
DOWNLOAD_QUEUE_STATUSES = ("pending", "running", "success", "failed")
|
||||
|
||||
@@ -1,20 +1,35 @@
|
||||
"""logging 配置:降级到 stderr,避免污染 stdout(stdout 留给结构化输出如 JSON)。
|
||||
|
||||
优先用 jiangchang_skill_core 提供的 logging;本地无该包时降级到 stderr。
|
||||
|
||||
v1.1.18:本地降级路径追加 RotatingFileHandler(按 LOG_FILE env 决定是否落文件)。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import logging.handlers
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from util.constants import LOG_LOGGER_NAME, SKILL_SLUG, SKILL_VERSION
|
||||
from util.constants import (
|
||||
LOG_LOGGER_NAME,
|
||||
SKILL_SLUG,
|
||||
SKILL_VERSION,
|
||||
)
|
||||
|
||||
_configured = False
|
||||
|
||||
|
||||
def setup_skill_logging() -> logging.Logger:
|
||||
"""初始化 skill logger(幂等)。"""
|
||||
"""初始化 skill logger(幂等)。
|
||||
|
||||
本地降级行为(无 jiangchang_skill_core 时):
|
||||
- 若 LOG_FILE 已设:追加 RotatingFileHandler(LOG_MAX_BYTES × LOG_BACKUP_COUNT)
|
||||
- 始终追加 StreamHandler(sys.stderr)
|
||||
- logger level 取自 LOG_LEVEL(默认 INFO)
|
||||
"""
|
||||
global _configured
|
||||
logger = logging.getLogger(LOG_LOGGER_NAME)
|
||||
if _configured:
|
||||
@@ -30,15 +45,38 @@ def setup_skill_logging() -> logging.Logger:
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
# 本地降级:基础 stderr handler,UTF-8 safe
|
||||
handler = logging.StreamHandler(sys.stderr)
|
||||
handler.setFormatter(
|
||||
logging.Formatter(
|
||||
fmt=f"%(asctime)s [{SKILL_SLUG}] %(levelname)s %(message)s",
|
||||
datefmt="%Y-%m-%d %H:%M:%S",
|
||||
)
|
||||
# logger 级别
|
||||
level_name = os.environ.get("LOG_LEVEL", "INFO").upper()
|
||||
logger.setLevel(getattr(logging, level_name, logging.INFO))
|
||||
|
||||
formatter = logging.Formatter(
|
||||
fmt=f"%(asctime)s [{SKILL_SLUG}] %(levelname)s %(message)s",
|
||||
datefmt="%Y-%m-%d %H:%M:%S",
|
||||
)
|
||||
logger.setLevel(logging.INFO)
|
||||
|
||||
# 本地降级:可选 RotatingFileHandler
|
||||
log_file_str = os.environ.get("LOG_FILE")
|
||||
if log_file_str:
|
||||
log_path = Path(log_file_str)
|
||||
if not log_path.is_absolute():
|
||||
from util.runtime_paths import get_logs_dir
|
||||
|
||||
log_path = get_logs_dir() / log_file_str
|
||||
log_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
max_bytes = int(os.environ.get("LOG_MAX_BYTES", str(5 * 1024 * 1024)))
|
||||
backup_count = int(os.environ.get("LOG_BACKUP_COUNT", "5"))
|
||||
file_handler = logging.handlers.RotatingFileHandler(
|
||||
log_path,
|
||||
maxBytes=max_bytes,
|
||||
backupCount=backup_count,
|
||||
encoding="utf-8",
|
||||
)
|
||||
file_handler.setFormatter(formatter)
|
||||
logger.addHandler(file_handler)
|
||||
|
||||
# 始终保留 stderr handler
|
||||
handler = logging.StreamHandler(sys.stderr)
|
||||
handler.setFormatter(formatter)
|
||||
logger.addHandler(handler)
|
||||
logger.propagate = False
|
||||
_configured = True
|
||||
@@ -51,3 +89,19 @@ def get_skill_logger() -> logging.Logger:
|
||||
if not _configured:
|
||||
return setup_skill_logging()
|
||||
return logging.getLogger(LOG_LOGGER_NAME)
|
||||
|
||||
|
||||
def reset_for_tests() -> None:
|
||||
"""测试用:重置 _configured flag,让 setup_skill_logging 重新初始化。
|
||||
|
||||
必须在每个修改日志配置的测试 fixture 里调用,否则 env 改动不会生效。
|
||||
"""
|
||||
global _configured
|
||||
logger = logging.getLogger(LOG_LOGGER_NAME)
|
||||
for h in list(logger.handlers):
|
||||
try:
|
||||
h.close()
|
||||
except Exception:
|
||||
pass
|
||||
logger.removeHandler(h)
|
||||
_configured = False
|
||||
|
||||
@@ -66,6 +66,30 @@ def get_dispatch_dir() -> Path:
|
||||
return get_data_dir() / "dispatch"
|
||||
|
||||
|
||||
def get_logs_dir() -> Path:
|
||||
"""返回日志目录(RotatingFileHandler 默认落地点)。"""
|
||||
env_dir = os.environ.get("JIANGCHANG_LOGS_DIR")
|
||||
if env_dir:
|
||||
return Path(env_dir)
|
||||
return get_data_dir() / "logs"
|
||||
|
||||
|
||||
def get_input_dir() -> Path:
|
||||
"""返回用户主动投放的输入目录(含 contracts/*.csv + materials/*.{pdf,docx,doc})。"""
|
||||
env_dir = os.environ.get("INPUT_DIR")
|
||||
if env_dir:
|
||||
return Path(env_dir)
|
||||
return get_data_dir() / "input"
|
||||
|
||||
|
||||
def get_input_contracts_dir() -> Path:
|
||||
return get_input_dir() / "contracts"
|
||||
|
||||
|
||||
def get_input_materials_dir() -> Path:
|
||||
return get_input_dir() / "materials"
|
||||
|
||||
|
||||
def get_examples_dir() -> Path:
|
||||
"""返回样例数据目录。"""
|
||||
return get_skill_root() / "examples"
|
||||
@@ -78,5 +102,13 @@ def get_templates_dir() -> Path:
|
||||
|
||||
def ensure_dirs() -> None:
|
||||
"""确保所有需要的目录存在。"""
|
||||
for d in (get_data_dir(), get_output_dir(), get_dispatch_dir()):
|
||||
for d in (
|
||||
get_data_dir(),
|
||||
get_output_dir(),
|
||||
get_dispatch_dir(),
|
||||
get_logs_dir(),
|
||||
get_input_dir(),
|
||||
get_input_contracts_dir(),
|
||||
get_input_materials_dir(),
|
||||
):
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
1595
tests/run_tests.py
1595
tests/run_tests.py
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user