.80 · 单轮详情
20261112-huanxi-0.8.14.80-framework-tools-t1
测量日 2026-11-12 · source SHA 82fffd679ff476b988368c9a5b2ded8f9b3d6162
schema 0.8.2-runtime-tables.132
provider: ppio；template: teams-work-v2；cpu: 4；memory_mib: 8192；os: Debian 13.6 trixie；deployment_kind: fresh_isolated
analysis-20261112-huanxi-0.8.14.77-81（交付快照） · latest（交付快照） · 飞书报告
逐用户输入
| 输入 | 原生终态 | 回复延迟 / 秒 | CLI / 观察耗时 / 秒 | 模型响应 | 工具调用 |
|---|---|---|---|---|---|
| C01/U1 | completed | 42.6 | 671.4 | 5 | 4 |
| C02/U1 | completed | 58.0 | 59.3 | 6 | 5 |
| C03/U1 | completed | 34.8 | 36.9 | 4 | 3 |
| C04/U1 | completed | 98.3 | 100.1 | 9 | 8 |
| C05/U1 | completed | 97.6 | 98.1 | 8 | 7 |
| C06/U1 | completed | 46.9 | 47.0 | 4 | 3 |
| C07/U1 | completed | 57.1 | 59.2 | 5 | 4 |
| C07/U2 | completed | 73.1 | 73.7 | 8 | 8 |
| C08/U1 | completed | 33.8 | 34.8 | 4 | 3 |
| C08/U2 | completed | 69.9 | 71.8 | 6 | 5 |
| C09/U1 | completed | 51.3 | 53.1 | 6 | 5 |
| C09/U2 | completed | 73.9 | 75.8 | 7 | 6 |
| C10/U1 | completed | 56.5 | 57.3 | 7 | 6 |
| C10/U2 | completed | 79.7 | 82.0 | 6 | 5 |
逐题内容判分
| 题目 | 内容复核 | 原检查器 | 回复合计 / 秒 |
|---|---|---|---|
| C01 发票去重与对账 | PASS | PASS | 42.6 |
| C02 日志关联与事故时间线 | PASS | PASS | 58.0 |
| C03 多文件政策优先级 | PASS | PASS | 34.8 |
| C04 时区与稳定排序 | PASS | PASS | 98.3 |
| C05 小模块修复与回归 | PASS | PASS | 97.6 |
| C06 依赖图与关键路径 | PASS | PASS | 46.9 |
| C07 采购方案追加预算约束 | PASS | PASS | 130.2 |
| C08 销售汇总与增量更正 | PASS | FAIL | 103.6 |
| C09 命令行工具连续迭代 | PASS | PASS | 125.2 |
| C10 发布判断与新证据修正 | INCONCLUSIVE | FAIL | 136.2 |
指标与缺测
{
"artifact_grades": {
"INCONCLUSIVE": 1,
"PASS": 9
},
"invocation_or_observer": {
"expected": 14,
"observed": 14,
"observed_sum_s": 1520.5619671810007,
"sum_s": 1520.5619671810007
},
"model_id_mismatch": 0,
"model_id_missing": 0,
"native_tool_calls": 72,
"observed_model_ids": {
"deepseek/deepseek-v4.1-flash-1017": 85
},
"observed_model_intervals": {
"expected": 85,
"observed": 85,
"observed_sum_s": 755.3867136230468,
"sum_s": 755.3867136230468
},
"observed_tmp_commands": 2,
"observed_tool_intervals": {
"expected": 72,
"observed": 72,
"observed_sum_s": 21.965413330078125,
"sum_s": 21.965413330078125
},
"raw_artifact_grades": {
"FAIL": 2,
"PASS": 8
},
"response_latency": {
"expected": 14,
"observed": 14,
"observed_sum_s": 873.5276961326599,
"sum_s": 873.5276961326599
},
"scope_observed_cases": 10,
"terminals": {
"completed": 14
},
"tmp_command_cases": [
"C08",
"C09"
],
"tool_distribution": {
"shell": 72
},
"tool_input_availability": {
"available": 65,
"partial": 7
},
"tool_missing_arguments": 0,
"tool_missing_results": 0,
"tool_output_availability": {
"available": 72
},
"user_turns": 14,
"visible_model_responses": 85
}采集与覆盖说明（manifest 原文）
- Relative to historical .68/.69 (teams-work-v1), this run uses a different teams-work-v2 environment. Main comparison uses same-day .69 v2 environment anchor.
- Six versions execute concurrently in separate 4C/8G sandboxes; cross-version provider contention and model nondeterminism are uncontrolled. One trial per case; no statistical ranking or causal attribution.
- Frozen Eval 28db1d8f2 and .69 response-only adapter; no OS source changes.
- Eval codeload returned HTTP404; identical frozen 1128-file source snapshot transferred and verified in sandbox. Initial incomplete archive upload retained; reassembled archive SHA256 32d89e797c08be115ae19e0d3a12a1242ad6bc55b54ecd05511a203f9416cb15.
- Old Eval requires optional upgrades; .80 removes TaskFlow upgrades projection (#3724). Public schema never required upgrades.
- C01 observer restart authorized only after native final reply + idle, preserving original intent/ACK/Session/command/operation; no input replay. Native reply timestamp remains latency endpoint; observation recovery delay reported separately.
- Adapter/driver diff: raw/resume-adaptation.diff; SHA256=ac954830f2c52a044f26e21c96b3310607fcac9fb2dd72e2e646018a02485579; adapter SHA256=ff8c2500e4c31223f5e8bcba3cc386e7668a84d422f1b72fc9323652442fa196; resume driver SHA256=4ddf83603407c96afa5420721fb2c41193182a2d3ef46c726311d859bef4dc05
- Content grades, native terminals, availability and public pagination probe recorded separately. Logical model token/call counts do not prove physical attempts or billing.
口径与限制
n=1；并发争用、模型随机性、跨日基线未受控；不能据此作统计排名或因果结论。输入终态、内容判分、范围遵守与采集完整度分开呈现。
固定基线为2026-11-07 Codex0.165.0 / OpenClaw2026.11.1；欢喜OS各版本单独新增试次。相同10题、14用户输入，liqian deepseek/deepseek-v4.1-flash-1017。 每次报告明确区分：输入→原生终态延迟、CLI/观察收尾、模型响应、工具步骤、内容判分、范围遵守与证据缺测。跨日固定基线不是同时随机对照。
- Relative to historical .68/.69 (teams-work-v1), this run uses a different teams-work-v2 environment. Main comparison uses same-day .69 v2 environment anchor.
- Six versions execute concurrently in separate 4C/8G sandboxes; cross-version provider contention and model nondeterminism are uncontrolled. One trial per case; no statistical ranking or causal attribution.
内容复核口径（comparisons/latest.md）：历史复核：C08 原检查器漏查子目录备份，原判保留、内容复核 PASS；C10 部署清单完整性与 JSON 层级题面有歧义，保留 INCONCLUSIVE。
主耗时不包含CLI终态后退出等待。模型/工具时间线允许重叠；不得相加当整体墙钟。单trial及跨日历史基线只支持探索性比较。