cited abstract supplied

1/9 dimensions tested
Sonnet 5

Configuration: The claim's cited abstract is handed to the model before it answers. No search, no tools; the evidence document itself.
Source: gym.py scifact · contestants V_ev_*
Recorded runs: 300 scored outputs · $1.09 measured model spend · runs on 2026-08-31
Access: open harness · standard tools

FIG. 01

By dimension

Scores /10 · Paired p-values are exploratory and unadjusted

8.47Verify

Compared with baselinen=300 · verdict accuracy · evidence in hand · Δ +2.78 vs Sonnet 5 · 98W 15L 186T on 299 paired · p<.001 (unadjusted) · $0.0036/task · 2026-08-31

naked 0.569 at $0.0050 per task; the harness is cheaper because the naked model burns tokens hedging Test: exact McNemar on the tasks both cells answered.

Run provenance

Report path: outputs/scifact/results_matrix_sonnet5.json
Task pack: scifact-dev-300
Run: matrix-sonnet-5

—Ground

Not yet tested.

—Find

Not yet tested.

—Extract

Not yet tested.

—Analyze

Not yet tested.

—Compute

Not yet tested.

—Synthesize

Not yet tested.

—Review

Not yet tested.

—Hypothesize

Not yet tested.