Results overview / Choose papers

Choose relevant papers

Choose five relevant papers from a supplied list of 20.

Sonnet 5·4 examples·One attempt with each setup·2026-09-15

No difference observed

All three conditions reached 43.8% mean recall. Perfect selection from these lists could reach only 50%.

AI alone

43.8%

known sources found

$0.0302 · 3.1s per case

With checklist

43.8%

known sources found

$0.0314 · 4.0s per case

With skill

43.8%

known sources found

$0.0336 · 3.6s per case

Skill tested: K-Dense: literature-review ↗. “AI alone” uses the task instructions and required tools; the other setups add a checklist or skill guidance. Everything else is held the same.

How this was scored, and what it does not establish

Recall is the share of known cited sources recovered in the five selected papers, averaged across cases. The supplied candidate lists limit how high this score can go.

Two of four candidate pools contain no known target source.
The target set contains published cited sources, not exhaustive relevance judgments.
This tests fixed-list reranking; searching databases and writing a literature review were not exercised.

SKILL.md supplied; no explicit reference-file reads recorded.

Next useful test: Improve candidate coverage and test new research questions with reviewed relevance judgments.

Examples and actual answers

Open an example to see all three attempts.

HPV screening sensitivitySciFactAI alone75.0%With checklist75.0%With skill75.0%

Choose five candidate papers for this question: Assess whether HPV-based screening is more sensitive over follow-up than cytology for detecting grade-2 cervical lesions.

The supplied list has 20 candidates. Its maximum possible recall of the known target sources is 100%.

Source dataset ↗ · Source cluster: scifact-source-8e49e639da36e853

AI alone

Selected 3 of 4 known cited sources.

Output excerpt

{
  "paper_ids": [
    "46695481",
    "9764256",
    "27873158",
    "27446873",
    "7650066"
  ]
}

Selected papers

{
  "46695481": "Human papillomavirus and Papanicolaou tests to screen for cervical cancer.",
  "9764256": "Human papillomavirus testing for the detection of high-grade cervical intraepithelial neoplasia and cancer: final results of the POBASCAM randomised controlled trial.",
  "27873158": "Efficacy of human papillomavirus testing for the detection of invasive cervical cancers and cervical intraepithelial neoplasia: a randomised controlled trial.",
  "27446873": "Rate of cervical cancer, severe intraepithelial neoplasia, and adenocarcinoma in situ in primary HPV DNA screening with cytology triage: randomised study within organised screening…",
  "7650066": "Long-term follow-up of cervical disease in women screened by cytology and HPV testing: results from the HART study"
}

Recorded score: 75.0% · $0.0388 · 3.9s

With checklist

Selected 3 of 4 known cited sources.

Output excerpt

{
  "paper_ids": [
    "46695481",
    "9764256",
    "27446873",
    "27873158",
    "7650066"
  ]
}

Selected papers

{
  "46695481": "Human papillomavirus and Papanicolaou tests to screen for cervical cancer.",
  "9764256": "Human papillomavirus testing for the detection of high-grade cervical intraepithelial neoplasia and cancer: final results of the POBASCAM randomised controlled trial.",
  "27446873": "Rate of cervical cancer, severe intraepithelial neoplasia, and adenocarcinoma in situ in primary HPV DNA screening with cytology triage: randomised study within organised screening…",
  "27873158": "Efficacy of human papillomavirus testing for the detection of invasive cervical cancers and cervical intraepithelial neoplasia: a randomised controlled trial.",
  "7650066": "Long-term follow-up of cervical disease in women screened by cytology and HPV testing: results from the HART study"
}

Recorded score: 75.0% · $0.0388 · 3.7s

With skill

Selected 3 of 4 known cited sources.

Output excerpt

{
  "paper_ids": [
    "46695481",
    "9764256",
    "27873158",
    "27446873",
    "6561200"
  ]
}

Selected papers

{
  "46695481": "Human papillomavirus and Papanicolaou tests to screen for cervical cancer.",
  "9764256": "Human papillomavirus testing for the detection of high-grade cervical intraepithelial neoplasia and cancer: final results of the POBASCAM randomised controlled trial.",
  "27873158": "Efficacy of human papillomavirus testing for the detection of invasive cervical cancers and cervical intraepithelial neoplasia: a randomised controlled trial.",
  "27446873": "Rate of cervical cancer, severe intraepithelial neoplasia, and adenocarcinoma in situ in primary HPV DNA screening with cytology triage: randomised study within organised screening…",
  "6561200": "Efficacy of HPV DNA testing with cytology triage and/or repeat HPV DNA testing in primary cervical cancer screening."
}

Recorded score: 75.0% · $0.0527 · 5.0s

Recorded case find-sf971 · Run four-track-canary-v3. Output excerpts are bounded; the full submitted answer was scored.

Tissue specificity of ribosome disordersSciFactAI alone0.0%With checklist0.0%With skill0.0%

Choose five candidate papers for this question: Assess whether ribosome-related disorders produce largely nonspecific effects across cells and tissues.

The supplied list has 20 candidates. Its maximum possible recall of the known target sources is 0%.

Source dataset ↗ · Source cluster: scifact-source-71d5408dbc571823

AI alone

Selected 0 of 1 known cited sources. No known target source was present in the candidate list.

Output excerpt

{
  "paper_ids": [
    "8925851",
    "24737389",
    "26112624",
    "7487927",
    "31304956"
  ]
}

Selected papers

{
  "8925851": "Review article",
  "24737389": "Growth control and ribosomopathies.",
  "26112624": "The complexity of human ribosome biogenesis revealed by systematic nucleolar screening of Pre-rRNA processing factors.",
  "7487927": "The Ribosome Biogenesis Factor Nol11 Is Required for Optimal rDNA Transcription and Craniofacial Development in Xenopus ",
  "31304956": "Cranial neural crest and the building of the vertebrate head"
}

Recorded score: 0.0% · $0.0259 · 2.5s

With checklist

Selected 0 of 1 known cited sources. No known target source was present in the candidate list.

Output excerpt

{
  "paper_ids": [
    "8925851",
    "24737389",
    "7487927",
    "26112624",
    "31304956"
  ]
}

Selected papers

{
  "8925851": "Review article",
  "24737389": "Growth control and ribosomopathies.",
  "7487927": "The Ribosome Biogenesis Factor Nol11 Is Required for Optimal rDNA Transcription and Craniofacial Development in Xenopus ",
  "26112624": "The complexity of human ribosome biogenesis revealed by systematic nucleolar screening of Pre-rRNA processing factors.",
  "31304956": "Cranial neural crest and the building of the vertebrate head"
}

Recorded score: 0.0% · $0.0267 · 4.1s

With skill

Selected 0 of 1 known cited sources. No known target source was present in the candidate list.

Output excerpt

{
  "paper_ids": [
    "8925851",
    "24737389",
    "7487927",
    "26112624",
    "31304956"
  ]
}

Selected papers

{
  "8925851": "Review article",
  "24737389": "Growth control and ribosomopathies.",
  "7487927": "The Ribosome Biogenesis Factor Nol11 Is Required for Optimal rDNA Transcription and Craniofacial Development in Xenopus ",
  "26112624": "The complexity of human ribosome biogenesis revealed by systematic nucleolar screening of Pre-rRNA processing factors.",
  "31304956": "Cranial neural crest and the building of the vertebrate head"
}

Recorded score: 0.0% · $0.0254 · 2.8s

Recorded case find-sf1049 · Run four-track-canary-v3. Output excerpts are bounded; the full submitted answer was scored.

Stem-cell purificationSciFactAI alone0.0%With checklist0.0%With skill0.0%

Choose five candidate papers for this question: Assess whether blood-forming stem-cell isolation methods can reach 50% purity.

The supplied list has 20 candidates. Its maximum possible recall of the known target sources is 0%.

Source dataset ↗ · Source cluster: scifact-source-4dec666818d18b9e

AI alone

Selected 0 of 1 known cited sources. No known target source was present in the candidate list.

Output excerpt

{
  "paper_ids": [
    "25516011",
    "40234452",
    "11900630",
    "18374364",
    "13116880"
  ]
}

Selected papers

{
  "25516011": "Purification and characterization of mouse hematopoietic stem cells.",
  "40234452": "HES-1 preserves purified hematopoietic stem cells ex vivo and accumulates side population cells in vivo.",
  "11900630": "Hematopoietic stem cells and other hematopoietic cells show broad resistance to chemotherapeutic agents in vivo when overexpressing bcl-2.",
  "18374364": "In vivo proliferation and cell cycle kinetics of long-term self-renewing hematopoietic stem cells.",
  "13116880": "Hematopoietic stem cell: self-renewal versus differentiation."
}

Recorded score: 0.0% · $0.0221 · 2.6s

With checklist

Selected 0 of 1 known cited sources. No known target source was present in the candidate list.

Output excerpt

{
  "paper_ids": [
    "25516011",
    "40234452",
    "11900630",
    "13116880",
    "34982259"
  ]
}

Selected papers

{
  "25516011": "Purification and characterization of mouse hematopoietic stem cells.",
  "40234452": "HES-1 preserves purified hematopoietic stem cells ex vivo and accumulates side population cells in vivo.",
  "11900630": "Hematopoietic stem cells and other hematopoietic cells show broad resistance to chemotherapeutic agents in vivo when overexpressing bcl-2.",
  "13116880": "Hematopoietic stem cell: self-renewal versus differentiation.",
  "34982259": "Of lineage and legacy: the development of mammalian hematopoietic stem cells"
}

Recorded score: 0.0% · $0.0255 · 5.3s

With skill

Selected 0 of 1 known cited sources. No known target source was present in the candidate list.

Output excerpt

{
  "paper_ids": [
    "25516011",
    "40234452",
    "11900630",
    "13116880",
    "18374364"
  ]
}

Selected papers

{
  "25516011": "Purification and characterization of mouse hematopoietic stem cells.",
  "40234452": "HES-1 preserves purified hematopoietic stem cells ex vivo and accumulates side population cells in vivo.",
  "11900630": "Hematopoietic stem cells and other hematopoietic cells show broad resistance to chemotherapeutic agents in vivo when overexpressing bcl-2.",
  "13116880": "Hematopoietic stem cell: self-renewal versus differentiation.",
  "18374364": "In vivo proliferation and cell cycle kinetics of long-term self-renewing hematopoietic stem cells."
}

Recorded score: 0.0% · $0.0240 · 5.1s

Recorded case find-sf508 · Run four-track-canary-v3. Output excerpts are bounded; the full submitted answer was scored.

Cervical-cancer incidenceSciFactAI alone100.0%With checklist100.0%With skill100.0%

Choose five candidate papers for this question: Assess a claim of declining cervical-cancer incidence.

The supplied list has 20 candidates. Its maximum possible recall of the known target sources is 100%.

Source dataset ↗ · Source cluster: scifact-source-dcc771eb41e19d23

AI alone

Selected 3 of 3 known cited sources.

Output excerpt

{
  "paper_ids": [
    "25742130",
    "36355784",
    "12779444",
    "52188256",
    "27446873"
  ]
}

Selected papers

{
  "25742130": "Mass screening programmes and trends in cervical cancer in Finland and the Netherlands.",
  "36355784": "The effect of mass screening on incidence and mortality of squamous and adenocarcinoma of cervix uteri.",
  "12779444": "Effect of screening on cervical cancer mortality in England and Wales: analysis of trends with an age period cohort model.",
  "52188256": "Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries.",
  "27446873": "Rate of cervical cancer, severe intraepithelial neoplasia, and adenocarcinoma in situ in primary HPV DNA screening with cytology triage: randomised study within organised screening…"
}

Recorded score: 100.0% · $0.0341 · 3.2s

With checklist

Selected 3 of 3 known cited sources.

Output excerpt

{
  "paper_ids": [
    "25742130",
    "36355784",
    "12779444",
    "52188256",
    "27446873"
  ]
}

Selected papers

{
  "25742130": "Mass screening programmes and trends in cervical cancer in Finland and the Netherlands.",
  "36355784": "The effect of mass screening on incidence and mortality of squamous and adenocarcinoma of cervix uteri.",
  "12779444": "Effect of screening on cervical cancer mortality in England and Wales: analysis of trends with an age period cohort model.",
  "52188256": "Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries.",
  "27446873": "Rate of cervical cancer, severe intraepithelial neoplasia, and adenocarcinoma in situ in primary HPV DNA screening with cytology triage: randomised study within organised screening…"
}

Recorded score: 100.0% · $0.0347 · 2.8s

With skill

Selected 3 of 3 known cited sources.

Output excerpt

{
  "paper_ids": [
    "25742130",
    "36355784",
    "12779444",
    "52188256",
    "27446873"
  ]
}

Selected papers

{
  "25742130": "Mass screening programmes and trends in cervical cancer in Finland and the Netherlands.",
  "36355784": "The effect of mass screening on incidence and mortality of squamous and adenocarcinoma of cervix uteri.",
  "12779444": "Effect of screening on cervical cancer mortality in England and Wales: analysis of trends with an age period cohort model.",
  "52188256": "Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries.",
  "27446873": "Rate of cervical cancer, severe intraepithelial neoplasia, and adenocarcinoma in situ in primary HPV DNA screening with cytology triage: randomised study within organised screening…"
}

Recorded score: 100.0% · $0.0326 · 1.7s

Recorded case find-sf597 · Run four-track-canary-v3. Output excerpts are bounded; the full submitted answer was scored.

← Results overview Download experiment data