Results overview / Check a claim

Check a scientific claim

Given a claim and an abstract, decide whether the evidence supports the claim.

Sonnet 5·8 examples·One attempt with each setup·2026-09-15

No difference observed

All three conditions matched the published verdict on all eight claims. Evidence quality remains unresolved.

AI alone

8/8

verdicts matched

$0.0077 · 3.4s per case

With checklist

8/8

verdicts matched

$0.0059 · 3.1s per case

With skill

8/8

verdicts matched

$0.0076 · 3.2s per case

Skill tested: K-Dense: scientific-critical-thinking ↗. “AI alone” uses the task instructions and required tools; the other setups add a checklist or skill guidance. Everything else is held the same.

How this was scored, and what it does not establish

The score counts verdicts that match the dataset labels. It does not judge whether the supplied rationale is scientifically sufficient.

Eight public claims; one run per condition, with no measured verdict headroom.
Published rationale annotations omit some relevant evidence; label agreement is not independent scientific adjudication.
The task supplies abstracts and tests only a narrow part of the critical-thinking package.

SKILL.md supplied; no explicit reference-file reads recorded.

Next useful test: Review the evidence labels, then test harder claims from new sources.

Examples and actual answers

Open an example to see all three attempts.

HPV screening sensitivitySciFactAI aloneMatchedWith checklistMatchedWith skillMatched

Assess whether HPV-based screening is more sensitive over follow-up than cytology for detecting grade-2 cervical lesions.

Source dataset ↗ · Source cluster: scifact-source-8e49e639da36e853

AI alone

Recorded SUPPORT; matches the published claim label.

Recorded answer

{
  "verdict": "SUPPORT",
  "evidence": [
    {
      "doc_id": "9764256",
      "sentence_id": 13
    },
    {
      "doc_id": "9764256",
      "sentence_id": 16
    },
    {
      "doc_id": "27873158",
      "sentence_id": 21
    }
  ]
}

Recorded score: 1/1 · $0.0194 · 6.4s

With checklist

Recorded SUPPORT; matches the published claim label.

Recorded answer

{
  "verdict": "SUPPORT",
  "evidence": [
    {
      "doc_id": "9764256",
      "sentence_id": 0
    },
    {
      "doc_id": "9764256",
      "sentence_id": 13
    },
    {
      "doc_id": "27873158",
      "sentence_id": 0
    },
    {
      "doc_id": "27873158",
      "sentence_id": 15
    }
  ]
}

Recorded score: 1/1 · $0.0180 · 4.2s

With skill

Recorded SUPPORT; matches the published claim label.

Recorded answer

{
  "verdict": "SUPPORT",
  "evidence": [
    {
      "doc_id": "9764256",
      "sentence_id": 0
    },
    {
      "doc_id": "9764256",
      "sentence_id": 13
    },
    {
      "doc_id": "27873158",
      "sentence_id": 0
    }
  ]
}

Recorded score: 1/1 · $0.0264 · 3.5s

Recorded case verify-sf971 · Run four-track-canary-v3. Output excerpts are bounded; the full submitted answer was scored.

Tissue specificity of ribosome disordersSciFactAI aloneMatchedWith checklistMatchedWith skillMatched

Assess whether ribosome-related disorders produce largely nonspecific effects across cells and tissues.

Source dataset ↗ · Source cluster: scifact-source-71d5408dbc571823

AI alone

Recorded CONTRADICT; matches the published claim label.

Recorded answer

{
  "verdict": "CONTRADICT",
  "evidence": [
    {
      "doc_id": "12486491",
      "sentence_id": 1
    },
    {
      "doc_id": "12486491",
      "sentence_id": 4
    }
  ]
}

Recorded score: 1/1 · $0.0055 · 2.3s

With checklist

Recorded CONTRADICT; matches the published claim label.

Recorded answer

{
  "verdict": "CONTRADICT",
  "evidence": [
    {
      "doc_id": "12486491",
      "sentence_id": 1
    },
    {
      "doc_id": "12486491",
      "sentence_id": 4
    }
  ]
}

Recorded score: 1/1 · $0.0038 · 3.1s

With skill

Recorded CONTRADICT; matches the published claim label.

Recorded answer

{
  "verdict": "CONTRADICT",
  "evidence": [
    {
      "doc_id": "12486491",
      "sentence_id": 1
    },
    {
      "doc_id": "12486491",
      "sentence_id": 4
    }
  ]
}

Recorded score: 1/1 · $0.0050 · 2.8s

Recorded case verify-sf1049 · Run four-track-canary-v3. Output excerpts are bounded; the full submitted answer was scored.

Stem-cell purificationSciFactAI aloneMatchedWith checklistMatchedWith skillMatched

Assess whether blood-forming stem-cell isolation methods can reach 50% purity.

Source dataset ↗ · Source cluster: scifact-source-4dec666818d18b9e

AI alone

Recorded NEI; matches the published claim label.

Recorded answer

{
  "verdict": "NEI",
  "evidence": []
}

Recorded score: 1/1 · $0.0044 · 2.3s

With checklist

Recorded NEI; matches the published claim label.

Recorded answer

{
  "verdict": "NEI",
  "evidence": []
}

Recorded score: 1/1 · $0.0024 · 1.4s

With skill

Recorded NEI; matches the published claim label.

Recorded answer

{
  "verdict": "NEI",
  "evidence": []
}

Recorded score: 1/1 · $0.0034 · 2.6s

Recorded case verify-sf508 · Run four-track-canary-v3. Output excerpts are bounded; the full submitted answer was scored.

Cervical-cancer incidenceSciFactAI aloneMatchedWith checklistMatchedWith skillMatched

Assess a claim of declining cervical-cancer incidence.

Source dataset ↗ · Source cluster: scifact-source-dcc771eb41e19d23

AI alone

Recorded SUPPORT; matches the published claim label.

Recorded answer

{
  "verdict": "SUPPORT",
  "evidence": [
    {
      "doc_id": "36355784",
      "sentence_id": 6
    },
    {
      "doc_id": "36355784",
      "sentence_id": 7
    }
  ]
}

Recorded score: 1/1 · $0.0071 · 2.6s

With checklist

Recorded SUPPORT; matches the published claim label.

Recorded answer

{
  "verdict": "SUPPORT",
  "evidence": [
    {
      "doc_id": "36355784",
      "sentence_id": 6
    },
    {
      "doc_id": "36355784",
      "sentence_id": 7
    }
  ]
}

Recorded score: 1/1 · $0.0053 · 2.6s

With skill

Recorded SUPPORT; matches the published claim label.

Recorded answer

{
  "verdict": "SUPPORT",
  "evidence": [
    {
      "doc_id": "36355784",
      "sentence_id": 6
    },
    {
      "doc_id": "36355784",
      "sentence_id": 7
    }
  ]
}

Recorded score: 1/1 · $0.0060 · 1.6s

Recorded case verify-sf597 · Run four-track-canary-v3. Output excerpts are bounded; the full submitted answer was scored.

A receptor and airway inflammationSciFactAI aloneMatchedWith checklistMatchedWith skillMatched

Assess whether CX3CR1 on type-2 helper T cells restrains inflammation in the airways.

Source dataset ↗ · Source cluster: scifact-source-047fcbae4a7ee0d7

AI alone

Recorded CONTRADICT; matches the published claim label.

Recorded answer

{
  "verdict": "CONTRADICT",
  "evidence": [
    {
      "doc_id": "21366394",
      "sentence_id": 2
    },
    {
      "doc_id": "21366394",
      "sentence_id": 4
    }
  ]
}

Recorded score: 1/1 · $0.0054 · 4.0s

With checklist

Recorded CONTRADICT; matches the published claim label.

Recorded answer

{
  "verdict": "CONTRADICT",
  "evidence": [
    {
      "doc_id": "21366394",
      "sentence_id": 2
    },
    {
      "doc_id": "21366394",
      "sentence_id": 4
    }
  ]
}

Recorded score: 1/1 · $0.0036 · 3.3s

With skill

Recorded CONTRADICT; matches the published claim label.

Recorded answer

{
  "verdict": "CONTRADICT",
  "evidence": [
    {
      "doc_id": "21366394",
      "sentence_id": 2
    },
    {
      "doc_id": "21366394",
      "sentence_id": 4
    }
  ]
}

Recorded score: 1/1 · $0.0039 · 1.8s

Recorded case verify-sf219 · Run new-sources-and-analysis-v3. Output excerpts are bounded; the full submitted answer was scored.

Bacterial response to alcohol stressSciFactAI aloneMatchedWith checklistMatchedWith skillMatched

Assess whether alcohol stress lowers bacterial IBP expression.

Source dataset ↗ · Source cluster: scifact-source-25486d3e4985ae13

AI alone

Recorded NEI; matches the published claim label.

Recorded answer

{
  "verdict": "NEI",
  "evidence": []
}

Recorded score: 1/1 · $0.0044 · 3.0s

With checklist

Recorded NEI; matches the published claim label.

Recorded answer

{
  "verdict": "NEI",
  "evidence": []
}

Recorded score: 1/1 · $0.0026 · 3.0s

With skill

Recorded NEI; matches the published claim label.

Recorded answer

{
  "verdict": "NEI",
  "evidence": []
}

Recorded score: 1/1 · $0.0035 · 2.7s

Recorded case verify-sf388 · Run new-sources-and-analysis-v3. Output excerpts are bounded; the full submitted answer was scored.

Open access and citationsSciFactAI aloneMatchedWith checklistMatchedWith skillMatched

Assess whether open access is linked to a higher chance of being cited than conventional publication.

Source dataset ↗ · Source cluster: scifact-source-a7d883a17dd4e4dd

AI alone

Recorded SUPPORT; matches the published claim label.

Recorded answer

{
  "verdict": "SUPPORT",
  "evidence": [
    {
      "doc_id": "27768226",
      "sentence_id": 2
    },
    {
      "doc_id": "27768226",
      "sentence_id": 10
    },
    {
      "doc_id": "27768226",
      "sentence_id": 11
    }
  ]
}

Recorded score: 1/1 · $0.0087 · 2.0s

With checklist

Recorded SUPPORT; matches the published claim label.

Recorded answer

{
  "verdict": "SUPPORT",
  "evidence": [
    {
      "doc_id": "27768226",
      "sentence_id": 2
    },
    {
      "doc_id": "27768226",
      "sentence_id": 10
    },
    {
      "doc_id": "27768226",
      "sentence_id": 11
    }
  ]
}

Recorded score: 1/1 · $0.0069 · 2.6s

With skill

Recorded SUPPORT; matches the published claim label.

Recorded answer

{
  "verdict": "SUPPORT",
  "evidence": [
    {
      "doc_id": "27768226",
      "sentence_id": 2
    },
    {
      "doc_id": "27768226",
      "sentence_id": 10
    },
    {
      "doc_id": "27768226",
      "sentence_id": 11
    }
  ]
}

Recorded score: 1/1 · $0.0076 · 3.4s

Recorded case verify-sf130 · Run new-sources-and-analysis-v3. Output excerpts are bounded; the full submitted answer was scored.

Tumour growth and metabolismSciFactAI aloneMatchedWith checklistMatchedWith skillMatched

Assess whether aPKCz promotes tumour growth through changes in glutamine metabolism.

Source dataset ↗ · Source cluster: scifact-source-5bea185d0e992a50

AI alone

Recorded CONTRADICT; matches the published claim label.

Recorded answer

{
  "verdict": "CONTRADICT",
  "evidence": [
    {
      "doc_id": "17755060",
      "sentence_id": 1
    },
    {
      "doc_id": "17755060",
      "sentence_id": 3
    },
    {
      "doc_id": "17755060",
      "sentence_id": 5
    }
  ]
}

Recorded score: 1/1 · $0.0071 · 4.5s

With checklist

Recorded CONTRADICT; matches the published claim label.

Recorded answer

{
  "verdict": "CONTRADICT",
  "evidence": [
    {
      "doc_id": "17755060",
      "sentence_id": 1
    },
    {
      "doc_id": "17755060",
      "sentence_id": 3
    },
    {
      "doc_id": "17755060",
      "sentence_id": 5
    }
  ]
}

Recorded score: 1/1 · $0.0044 · 4.2s

With skill

Recorded CONTRADICT; matches the published claim label.

Recorded answer

{
  "verdict": "CONTRADICT",
  "evidence": [
    {
      "doc_id": "17755060",
      "sentence_id": 1
    },
    {
      "doc_id": "17755060",
      "sentence_id": 3
    },
    {
      "doc_id": "17755060",
      "sentence_id": 5
    }
  ]
}

Recorded score: 1/1 · $0.0053 · 7.3s

Recorded case verify-sf1382 · Run new-sources-and-analysis-v3. Output excerpts are bounded; the full submitted answer was scored.

← Results overview Download experiment data