mirror of
https://github.com/jxxghp/MoviePilot.git
synced 2026-11-02 05:17:23 +08:00
fix(agent): preserve exact identifiers in final reports
This commit is contained in:
@@ -63,6 +63,7 @@ You act as a proactive agent. Your goal is to fully resolve the user's media-rel
|
||||
- If `search_tools` is available and a needed tool is missing, search by its exact name from a Skill or concise capability keywords. Read the returned tool schema on the next model call and continue the task; finding a tool does not authorize its execution.
|
||||
- If a tool preview includes `result_id` and `next_offset`, use `read_tool_result` to continue reading instead of executing the original tool again. Expired or unavailable result IDs require a narrower fresh query.
|
||||
- Respect the host's `succeeded`, `failed`, `pending`, and `unknown` outcomes. A pending task or unknown write is not complete. Use `get_tool_execution` with the returned invocation ID to inspect a durable write receipt. Reuse completed results, and do not change parameters merely to bypass duplicate or unresolved-write protection.
|
||||
- Copy persistent IDs, hashes, paths, and other exact identifiers character-for-character from tool responses. Never shorten, normalize, or abbreviate repeated characters; if the exact value is unavailable, report it as unresolved.
|
||||
- Use parallel tool calls by default for independent read-only or diagnostic work. In one assistant turn, issue all tool calls that can run without waiting for each other's results, such as checking enabled sites, library existence, recent history, downloader status, and scheduler or configuration state.
|
||||
- Keep tools sequential only when later arguments depend on earlier output, when a tool mutates state, when confirmation is required, or when concurrent writes could conflict.
|
||||
- When planning a multi-step investigation, group the first wave of safe state-gathering calls together, then continue with dependent actions after those results return.
|
||||
|
||||
@@ -137,6 +137,8 @@ MoviePilot 的真实 Agent 已在 `app/agent/llm/helper.py` 的 `runtime == "goo
|
||||
|
||||
这组三场景说明当前生产 Agent 在两个写入/复用路径和一个未知回执安全路径上已形成同条件通过样本,但原生 Codex 在持续不可用场景仍会因继续读取 Skill 和消耗调用预算而没有终态报告。它不能被改判成通过,也不能据此宣称浏览器、命令行、长上下文或中途消息已经完成 Codex 配对;这些能力仍需各自的真实驱动和留存证据。
|
||||
|
||||
针对前几轮 `dedup_existing` 中模型把 40 位 infohash 输出成 38 位的问题,核心提示新增了逐字符复制持久化 ID、hash 和路径的规则,并明确禁止缩短、规范化或压缩重复字符,无法确认时必须报告 unresolved。新增边界测试后,`tests/test_builtin_skill_boundaries.py` 的 11 项聚焦测试通过。使用同一未提交工作树、同一 `gpt-5.6-luna + max`、24 次调用上限、8192 输出上限、300 秒超时和 `harness_sha256=34a920e2127205c9c4facd758153c4e07a8f815fb2643f89d3e676cd4716758c` 重跑:MoviePilot `/tmp/moviepilot-agent-round-luna-oauth-dedup-live-exact-id-20260911.json` 与原生 Codex `/tmp/moviepilot-agent-round-luna-oauth-dedup-native-exact-id-20260911.json` 均通过,摘要 `/tmp/moviepilot-agent-round-luna-oauth-dedup-pair-exact-id-20260911.json` 为 `pair_valid=true`、`both_passed=true`;两侧均 9 次模型调用、0 失败/重复/副作用,MoviePilot 读取 3 次业务结果,原生 Codex 读取 2 次。该结果只证明本轮精确报告规则生效,不能替代后续多轮和 held-out 场景验收。
|
||||
|
||||
已有报告可用以下命令重新生成摘要;命令只读报告,不会再次调用模型:
|
||||
|
||||
```bash
|
||||
|
||||
@@ -84,3 +84,4 @@ uv run --locked --no-sync python -m scripts.evaluation \
|
||||
- 前端继续修复追加消息的显示锚点:未收到稳定 `steering_message_id` 的草稿改为有序集合,queued ACK 阶段直接插入当前助手段之后,并在 applied 阶段只切分助手段和绑定 ID;多个 ACK 断流后再按顺序应用时,消息仍保持工具调用之间的真实顺序。新增回归覆盖两个断流草稿的顺序与重复收口,前端 `AgentAssistantPanel.spec.ts` 43 项、类型检查、格式和 ESLint 通过。
|
||||
- 本轮长上下文配对 `/tmp/moviepilot-agent-round-luna-oauth-long-context-compaction-pair-final.json` 使用同一 `gpt-5.6-luna + max`、Codex OAuth、24 次模型调用上限、8192 输出上限、180 秒超时,`pair_valid=true`、`both_passed=true`。MoviePilot live `/tmp/moviepilot-agent-round-luna-oauth-live-long-context-compaction-after-task-preservation.json` 与原生 Codex `/tmp/moviepilot-agent-round-luna-oauth-native-long-context-compaction-after-task-preservation.json` 均 6 次 API 调用、10 次模型调用、0 副作用,观察到目标订阅 `9001`;MoviePilot 真实请求预算在第 7 次后触发压缩。此前两次失败样本分别暴露工具尾部不可裁剪和首条任务约束丢失,已由摘要回退、任务保留和序列化安全余量修复,失败报告仍保存在 `/tmp` 供复核。
|
||||
- 为验证前一轮的偶发性,在同一独立工作树、同一 `34a920e2127205c9c4facd758153c4e07a8f815fb2643f89d3e676cd4716758c` Harness、同一 `gpt-5.6-luna + max` 和 16 次调用预算下重复 `dedup_existing` 两轮:第二轮成对摘要 `/tmp/moviepilot-agent-round-luna-oauth-dedup-pair-repeat2-worktree-20260911.json` 为 `pair_valid=true`、`both_passed=true`(MoviePilot 7 次模型调用/131491 tokens,Codex 10 次/220335 tokens,均 0 重复/副作用);第三轮 `/tmp/moviepilot-agent-round-luna-oauth-dedup-pair-repeat3-worktree-20260911.json` 仍为 `pair_valid=true`,但 Codex 因最终 infohash 只有 38 位触发 `download_not_verified`(11 次模型调用),MoviePilot 通过(7 次/131702 tokens)。连同同 Harness 的第一轮 `/tmp/moviepilot-agent-round-luna-oauth-dedup-pair-repeat1-20260911.json`(MoviePilot 通过、Codex 同样因 38 位 infohash 未通过),三轮统计为 MoviePilot 3/3、Codex 1/3、双方同时通过 1/3;这确认结果存在模型轨迹波动,不能用一次成功或失败宣称整体对齐。一次误在其他 checkout 生成的原生报告因 Harness SHA 为 `eee4b6ad...` 被 `--compare` 拒绝,证明配对指纹控制有效。
|
||||
- 针对上述精确最终报告失败,核心提示新增持久化 ID/hash/path 的逐字符复制规则,并要求无法确认时保留 unresolved;`tests/test_builtin_skill_boundaries.py` 聚焦 11 项通过。相同未提交工作树、`gpt-5.6-luna + max`、24 次调用、8192 输出、300 秒超时和 `34a920e2127205c9c4facd758153c4e07a8f815fb2643f89d3e676cd4716758c` Harness 下,`dedup_existing` 摘要 `/tmp/moviepilot-agent-round-luna-oauth-dedup-pair-exact-id-20260911.json` 为 `pair_valid=true`、`both_passed=true`;MoviePilot 与原生 Codex 均 9 次模型调用、0 失败/重复/副作用,完整保留 40 位 infohash。仍需用多轮固定场景和 held-out 任务验证该规则的稳定收益。
|
||||
|
||||
@@ -153,6 +153,15 @@ def test_core_prompt_requires_read_skill_for_skill_documents() -> None:
|
||||
assert "do not use `read_file` to bypass the limit" in core_prompt
|
||||
|
||||
|
||||
def test_core_prompt_preserves_exact_identifiers_from_tool_results() -> None:
|
||||
"""核心提示必须阻止模型截断或模式化压缩持久化标识。"""
|
||||
core_prompt = CORE_PROMPT_PATH.read_text(encoding="utf-8")
|
||||
|
||||
assert "Copy persistent IDs, hashes, paths, and other exact identifiers character-for-character" in core_prompt
|
||||
assert "Never shorten, normalize, or abbreviate repeated characters" in core_prompt
|
||||
assert "if the exact value is unavailable, report it as unresolved" in core_prompt
|
||||
|
||||
|
||||
def test_api_collection_counts_must_use_gateway_metadata_before_database() -> None:
|
||||
"""核心提示与 API Skill 必须阻止列表截断后错误回退数据库统计。"""
|
||||
core_prompt = CORE_PROMPT_PATH.read_text(encoding="utf-8")
|
||||
|
||||
Reference in New Issue
Block a user