cited abstract supplied

1/9 dimensions tested
Haiku 4.5

Configuration: The claim's cited abstract is handed to the model before it answers. No search, no tools; the evidence document itself.
Source: gym.py scifact · contestants V_ev_*
Recorded runs: 300 scored outputs · $0.46 measured model spend · runs on 2026-08-31
Access: open harness · standard tools

FIG. 01

By dimension

Scores /10 · Paired p-values are exploratory and unadjusted

8.43Verify

Compared with baselinen=300 · verdict accuracy · evidence in hand · Δ +3.83 vs Haiku 4.5 · 131W 16L 153T on 300 paired · p<.001 (unadjusted) · $0.0015/task · 2026-08-31

naked 0.460 at $0.0019 per task; the harness is cheaper because the naked model burns tokens hedging Test: exact McNemar on the tasks both cells answered.

Run provenance

Report path: outputs/scifact/results_matrix_haiku45.json
Task pack: scifact-dev-300
Run: matrix-haiku-4.5

—Ground

Not yet tested.

—Find

Not yet tested.

—Extract

Not yet tested.

—Analyze

Not yet tested.

—Compute

Not yet tested.

—Synthesize

Not yet tested.

—Review

Not yet tested.

—Hypothesize

Not yet tested.