All configurations / prompt skill
schema prompt
- Configuration
- The SciERC type definitions and output schema written into the prompt; zero-shot otherwise.
- Source
- contestants.py E1_*
- Recorded runs
- 100 scored outputs · $3.86 measured model spend · runs on 2026-08-31
- Access
- open prompt · standard tools
FIG. 01
By dimension
Scores /10 · Paired p-values are exploratory and unadjusted—Verify
Not yet tested.
—Ground
Not yet tested.
—Find
Not yet tested.
4.74Extract
Compared with baselinen=100 · mean(entity F1, relation F1) · Δ +0.26 vs Opus 5 · 62W 34L 4T on 100 paired · p=.006 (unadjusted) · $0.0386/task · 2026-08-31
entity F1 0.584, relation F1 0.365 Test: exact sign test on discordant pairs.
Run provenance
Report path: outputs/extract/results.json
Task pack: scierc-test-100
Run: extract-100
—Analyze
Not yet tested.
—Compute
Not yet tested.
—Synthesize
Not yet tested.
—Review
Not yet tested.
—Hypothesize
Not yet tested.