Current formal Eval observer includes request-bound terminal and tool-detail fixes; old baseline capture coverage differs.
Frozen Eval28db1d8 requires retired entryEmployeeVersion; response-only adapter validates complete release schema and stable identity, official request/ACK/terminal logic unchanged.
14 exact input native terminals completed; no cross-user-turn overlap by timestamp verification.
105 visible logical model responses, all requested model; auxiliary/provider physical call coverage remains partial.
100 native calls were shell; no call_tool or native Skill execution in this trial.
All100 arguments and result bodies available; original fullResultRef pagination first request503 preserved; ordinary public detail exact payload hashes verified100/100.
15 literal /tmp/ argument occurrences in7 cases, plus parent-directory listing observed; scope compliance not PASS.
Original grade9PASS/1FAIL; C08 recursive exact backup reviewPASS, C10 kept inconclusive across all baselines.
Preparation subphase times not captured remain null; phases overlap and must not be summed as serial wall time.
Frozen Eval, dataset, model and response-only adapter reused from .68; actual schema must match before input.
Fixed prior Codex/OpenClaw baseline; no new baseline trials.
All14 inputs completed;97 public logical model responses and85 shell calls. Direct-call benchmark does not exercise Skill/call_tool/attachment/group/TaskFlow-QA acceptance.
Original grader8PASS2FAIL; reviewed8PASS,C06FAIL(contract tasks array mismatch),C10 fixture ambiguity.
Ordinary public detail recovered85complete tool results after one actual.69 paged503; existing#3518.
Arguments:0fully missing,77available,8partial(truncation unknown). Direct details return identical text and source refs; full argument completeness for8partial entries is not established. Canonical source hashes not recomputed from business-text representation. All85tool RESULTS complete.
Relative to historical .68/.69 (teams-work-v1), this run uses a different teams-work-v2 environment. Main comparison uses same-day .69 v2 environment anchor.
Six versions execute concurrently in separate 4C/8G sandboxes; cross-version provider contention and model nondeterminism are uncontrolled. One trial per case; no statistical ranking or causal attribution.
Frozen Eval 28db1d8f2 and .69 response-only adapter; no OS source changes.
Eval codeload returned HTTP404; identical frozen 1128-file source snapshot transferred and verified in sandbox. Initial incomplete archive upload retained; reassembled archive SHA256 32d89e797c08be115ae19e0d3a12a1242ad6bc55b54ecd05511a203f9416cb15.
Content grades, native terminals, availability and public pagination probe recorded separately. Logical model token/call counts do not prove physical attempts or billing.
Relative to historical .68/.69 (teams-work-v1), this run uses a different teams-work-v2 environment. Main comparison uses same-day .69 v2 environment anchor.
Six versions execute concurrently in separate 4C/8G sandboxes; cross-version provider contention and model nondeterminism are uncontrolled. One trial per case; no statistical ranking or causal attribution.
Frozen Eval 28db1d8f2 and .69 response-only adapter; no OS source changes.
Eval codeload returned HTTP404; identical frozen 1128-file source snapshot transferred and verified in sandbox. Initial incomplete archive upload retained; reassembled archive SHA256 32d89e797c08be115ae19e0d3a12a1242ad6bc55b54ecd05511a203f9416cb15.
Content grades, native terminals, availability and public pagination probe recorded separately. Logical model token/call counts do not prove physical attempts or billing.
Relative to historical .68/.69 (teams-work-v1), this run uses a different teams-work-v2 environment. Main comparison uses same-day .69 v2 environment anchor.
Six versions execute concurrently in separate 4C/8G sandboxes; cross-version provider contention and model nondeterminism are uncontrolled. One trial per case; no statistical ranking or causal attribution.
Frozen Eval 28db1d8f2 and .69 response-only adapter; no OS source changes.
Eval codeload returned HTTP404; identical frozen 1128-file source snapshot transferred and verified in sandbox. Initial incomplete archive upload retained; reassembled archive SHA256 32d89e797c08be115ae19e0d3a12a1242ad6bc55b54ecd05511a203f9416cb15.
Content grades, native terminals, availability and public pagination probe recorded separately. Logical model token/call counts do not prove physical attempts or billing.
Relative to historical .68/.69 (teams-work-v1), this run uses a different teams-work-v2 environment. Main comparison uses same-day .69 v2 environment anchor.
Six versions execute concurrently in separate 4C/8G sandboxes; cross-version provider contention and model nondeterminism are uncontrolled. One trial per case; no statistical ranking or causal attribution.
Frozen Eval 28db1d8f2 and .69 response-only adapter; no OS source changes.
Eval codeload returned HTTP404; identical frozen 1128-file source snapshot transferred and verified in sandbox. Initial incomplete archive upload retained; reassembled archive SHA256 32d89e797c08be115ae19e0d3a12a1242ad6bc55b54ecd05511a203f9416cb15.
Content grades, native terminals, availability and public pagination probe recorded separately. Logical model token/call counts do not prove physical attempts or billing.
Relative to historical .68/.69 (teams-work-v1), this run uses a different teams-work-v2 environment. Main comparison uses same-day .69 v2 environment anchor.
Six versions execute concurrently in separate 4C/8G sandboxes; cross-version provider contention and model nondeterminism are uncontrolled. One trial per case; no statistical ranking or causal attribution.
Frozen Eval 28db1d8f2 and .69 response-only adapter; no OS source changes.
Eval codeload returned HTTP404; identical frozen 1128-file source snapshot transferred and verified in sandbox. Initial incomplete archive upload retained; reassembled archive SHA256 32d89e797c08be115ae19e0d3a12a1242ad6bc55b54ecd05511a203f9416cb15.
Old Eval requires optional upgrades; .80 removes TaskFlow upgrades projection (#3724). Public schema never required upgrades.
C01 observer restart authorized only after native final reply + idle, preserving original intent/ACK/Session/command/operation; no input replay. Native reply timestamp remains latency endpoint; observation recovery delay reported separately.
Content grades, native terminals, availability and public pagination probe recorded separately. Logical model token/call counts do not prove physical attempts or billing.
Relative to historical .68/.69 (teams-work-v1), this run uses a different teams-work-v2 environment. Main comparison uses same-day .69 v2 environment anchor.
Six versions execute concurrently in separate 4C/8G sandboxes; cross-version provider contention and model nondeterminism are uncontrolled. One trial per case; no statistical ranking or causal attribution.
Frozen Eval 28db1d8f2 and .69 response-only adapter; no OS source changes.
Eval codeload returned HTTP404; identical frozen 1128-file source snapshot transferred and verified in sandbox. Initial incomplete archive upload retained; reassembled archive SHA256 32d89e797c08be115ae19e0d3a12a1242ad6bc55b54ecd05511a203f9416cb15.
Old Eval requires optional upgrades; .80 removes TaskFlow upgrades projection (#3724). Public schema never required upgrades.
C01 observer restart authorized only after native final reply + idle, preserving original intent/ACK/Session/command/operation; no input replay. Native reply timestamp remains latency endpoint; observation recovery delay reported separately.
Content grades, native terminals, availability and public pagination probe recorded separately. Logical model token/call counts do not prove physical attempts or billing.
Relative to historical .68/.69 (teams-work-v1), this run uses a different teams-work-v2 environment. Main comparison uses same-day .69 v2 environment anchor.
Six versions execute concurrently in separate 4C/8G sandboxes; cross-version provider contention and model nondeterminism are uncontrolled. One trial per case; no statistical ranking or causal attribution.