.69 · 单轮详情

20261108-huanxi-0.8.14.69-framework-tools-t1

测量日 2026-11-08 · source SHA 408c787c51cf7539caf1464a719b88e5b93c079f
schema 缺测
provider: ppio；template: teams-work-v1；cpu: 4；memory_mib: 8192；os: Debian 13.6 trixie；deployment_kind: fresh_isolated

analysis-20261108-huanxi-0.8.14.69-framework-tools-t1（交付快照） · latest（交付快照） · 飞书报告

版本 / 试次	测量日	completed / 输入	内容 PASS	回复耗时合计 / 秒	模型响应	工具调用	报告
.69	2026-11-08	14 / 14	8 / 10	927.9	97	85	详情与报告

逐用户输入

输入	原生终态	回复延迟 / 秒	CLI / 观察耗时 / 秒	模型响应	工具调用
C01/U1	completed	57.9	60.0	7	6
C02/U1	completed	72.6	74.4	10	9
C03/U1	completed	38.5	39.4	6	5
C04/U1	completed	39.8	41.6	5	4
C05/U1	completed	127.0	128.6	9	8
C06/U1	completed	113.5	114.4	9	8
C07/U1	completed	41.1	41.6	4	3
C07/U2	completed	44.1	45.8	4	3
C08/U1	completed	33.4	35.6	5	4
C08/U2	completed	43.7	45.9	7	7
C09/U1	completed	44.3	45.8	6	5
C09/U2	completed	172.8	174.8	12	11
C10/U1	completed	42.2	43.7	6	5
C10/U2	completed	57.1	58.7	7	7

逐题内容判分

题目	内容复核	原检查器	回复合计 / 秒
C01 发票去重与对账	PASS	PASS	57.9
C02 日志关联与事故时间线	PASS	PASS	72.6
C03 多文件政策优先级	PASS	PASS	38.5
C04 时区与稳定排序	PASS	PASS	39.8
C05 小模块修复与回归	PASS	PASS	127.0
C06 依赖图与关键路径	FAIL	FAIL	113.5
C07 采购方案追加预算约束	PASS	PASS	85.2
C08 销售汇总与增量更正	PASS	PASS	77.1
C09 命令行工具连续迭代	PASS	PASS	217.1
C10 发布判断与新证据修正	INCONCLUSIVE	FAIL	99.3

指标与缺测

{
  "artifact_grades": {
    "FAIL": 1,
    "INCONCLUSIVE": 1,
    "PASS": 8
  },
  "invocation_or_observer": {
    "expected": 14,
    "observed": 14,
    "observed_sum_s": 950.2326712360001,
    "sum_s": 950.2326712360001
  },
  "model_id_mismatch": 0,
  "model_id_missing": 0,
  "native_tool_calls": 85,
  "observed_model_ids": {
    "deepseek/deepseek-v4.1-flash-1017": 97
  },
  "observed_model_intervals": {
    "expected": 97,
    "observed": 97,
    "observed_sum_s": 866.6576772460937,
    "sum_s": 866.6576772460937
  },
  "observed_tmp_commands": 12,
  "observed_tool_intervals": {
    "expected": 85,
    "observed": 85,
    "observed_sum_s": 15.112617431640626,
    "sum_s": 15.112617431640626
  },
  "raw_artifact_grades": {
    "FAIL": 2,
    "PASS": 8
  },
  "response_latency": {
    "expected": 14,
    "observed": 14,
    "observed_sum_s": 927.9249949455261,
    "sum_s": 927.9249949455261
  },
  "scope_observed_cases": 10,
  "terminals": {
    "completed": 14
  },
  "tmp_command_cases": [
    "C02",
    "C03",
    "C08",
    "C09"
  ],
  "tool_distribution": {
    "shell": 85
  },
  "tool_input_availability": {
    "available": 77,
    "partial": 8
  },
  "tool_missing_arguments": 0,
  "tool_missing_results": 0,
  "tool_output_availability": {
    "available": 85
  },
  "user_turns": 14,
  "visible_model_responses": 97
}

采集与覆盖说明（manifest 原文）

Frozen Eval, dataset, model and response-only adapter reused from .68; actual schema must match before input.
Fixed prior Codex/OpenClaw baseline; no new baseline trials.
All14 inputs completed;97 public logical model responses and85 shell calls. Direct-call benchmark does not exercise Skill/call_tool/attachment/group/TaskFlow-QA acceptance.
Original grader8PASS2FAIL; reviewed8PASS,C06FAIL(contract tasks array mismatch),C10 fixture ambiguity.
Ordinary public detail recovered85complete tool results after one actual.69 paged503; existing#3518.
Arguments:0fully missing,77available,8partial(truncation unknown). Direct details return identical text and source refs; full argument completeness for8partial entries is not established. Canonical source hashes not recomputed from business-text representation. All85tool RESULTS complete.
C09U2M9 logical model85.176632s contains model_escalation event; physical retry/model routing/reason not exposed, attribution unresolved.

口径与限制

n=1；并发争用、模型随机性、跨日基线未受控；不能据此作统计排名或因果结论。输入终态、内容判分、范围遵守与采集完整度分开呈现。

固定基线为2026-11-07 Codex0.165.0 / OpenClaw2026.11.1；欢喜OS各版本单独新增试次。相同10题、14用户输入，liqian deepseek/deepseek-v4.1-flash-1017。每次报告明确区分：输入→原生终态延迟、CLI/观察收尾、模型响应、工具步骤、内容判分、范围遵守与证据缺测。跨日固定基线不是同时随机对照。

Relative to historical .68/.69 (teams-work-v1), this run uses a different teams-work-v2 environment. Main comparison uses same-day .69 v2 environment anchor.
Six versions execute concurrently in separate 4C/8G sandboxes; cross-version provider contention and model nondeterminism are uncontrolled. One trial per case; no statistical ranking or causal attribution.

内容复核口径（comparisons/latest.md）：历史复核：C08 原检查器漏查子目录备份，原判保留、内容复核 PASS；C10 部署清单完整性与 JSON 层级题面有歧义，保留 INCONCLUSIVE。

主耗时不包含CLI终态后退出等待。模型/工具时间线允许重叠；不得相加当整体墙钟。单trial及跨日历史基线只支持探索性比较。