Validation / Papers

Each paper below gives Cuvette its data and a question. For each known value, the list shows the value that each model got and if the two match.

We thank the authors of each paper and tutorial. Their published data and methods make this validation possible.

These results come from the final run of 9 October 2026. Claude Opus, Sonnet and Haiku did three runs for each paper. qwen3:8b did one run for each paper. The scorer did not know which model made each run. 53 of 71 papers are in the final run. The other 18 papers show one earlier run of Haiku, or no run, and are not in the totals.

**Table 1 |** Totals of the final run of 9 October 2026. Each run counts.
Model	Papers	Runs	Values that match	Correct in the final answer
Opus claude-opus-5-5	53	159	1411 of 1416	1146 of 1152
Sonnet claude-sonnet-5-5	53	159	1410 of 1416	1143 of 1152
Haiku claude-haiku-5-5	53	159	1410 of 1416	1133 of 1152
qwen3:8b a small open model, through Ollama	53	53	376 of 472	240 of 384

Session record: The full log of a run: each message, step and output.
Tolerance: The largest accepted difference from the known value, set before the run.
Match: A number in the session record is inside the tolerance.
In the final answer: The model also stated the value in its final answer.
Models: Claude Opus, Sonnet and Haiku (Anthropic). qwen3:8b, a small open model that runs on our computer through Ollama.

Match and no match compare the number in the session record with the known value. In the final answer shows if the model also stated the value.

Imaging

Research paper

Caicedo 2019

Caicedo JC, Roth J, Goodman A, et al.

Evaluation of Deep Learning Strategies for Nucleus Segmentation in Fluorescence Images. Cytometry Part A 95(9):952-965 (2019). doi:10.1002/cyto.a.23863

Evaluation of deep learning strategies for nucleus segmentation in fluorescence images

Cellpose (Python), through the cellpose adapter

Opus1 of 1 values match, in each of 3 runs

1 of 1

In the final answer: 1 of 1, in each of 3 runs

Sonnet1 of 1 values match, in each of 3 runs

1 of 1

In the final answer: 0 to 1 of 1, over 3 runs

2 runs ended with a time limit

Haiku1 of 1 values match, in each of 3 runs

1 of 1

In the final answer: 1 of 1, in each of 3 runs

qwen3:8b1 of 1 values match

1 of 1

In the final answer: 1 of 1

Show each value

Known values and run values for Caicedo 2019, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Nuclei in the ground-truth masks of the 50 test images.Printed in the paper	`5720`	exact	`5720` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`5720` match in 3 of 3 runsIn the final answer: yes in 1 of 3 runs	`5720` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`5720` matchIn the final answer: yes
Fraction of true nuclei missed at IoU 0.7, CellProfiler advanced pipeline. (reference)Printed in the paper	`0.155`	± 0.05	`0.1452` Reference value. Not scored	`0.1452` Reference value. Not scored	`0.1837` Reference value. Not scored	`0.1837` Reference value. Not scored
Fraction of true nuclei missed at IoU 0.7, CellProfiler basic pipeline. (reference)Printed in the paper	`0.201`	± 0.05	`0.2017` Reference value. Not scored	`0.2017` Reference value. Not scored	`0.2` Reference value. Not scored	`0.1837` Reference value. Not scored

Open the runs

Research paper

Ljosa 2012

Ljosa V, Sokolnicki KL, Carpenter AE.

Annotated high-throughput microscopy image sets for validation. Nature Methods 9(7):637 (2012). doi:10.1038/nmeth.2083

Also: Bray MA, Fraser AN, Hasaka TP, Carpenter AE. Workflow and metrics for image quality control in large-scale high-content screens. Journal of Biomolecular Screening 17(2):266-274 (2012). doi:10.1177/1087057111420292

Broad Bioimage Benchmark Collection, image set BBBC005 (synthetic cells with known counts)

CellProfiler (headless), through the cellprofiler adapter

Opus3 of 3 values match, in each of 3 runs

3 of 3

In the final answer: 2 of 2, in each of 3 runs

Sonnet3 of 3 values match, in each of 3 runs

3 of 3

In the final answer: 2 of 2, in each of 3 runs

Haiku3 of 3 values match, in each of 3 runs

3 of 3

In the final answer: 2 of 2, in each of 3 runs

qwen3:8b3 of 3 values match

3 of 3

In the final answer: 2 of 2

Show each value

Known values and run values for Ljosa 2012, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Detected to true count ratio on the 100 sampled images, published counts.We calculated it with pandas, on the published CellProfiler counts in BBBC005_results_bray.csv. Independent check: yes	`0.9379`	± 0.02	`0.936` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9398842` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9398842` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9399` matchIn the final answer: yes
Mean absolute count error on the 100 sampled images, published counts.We calculated it with pandas, on the published CellProfiler counts in BBBC005_results_bray.csv. Independent check: yes	`3.11`	± 0.1	`3.01` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.01` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.01` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.01` matchIn the final answer: yes
Sum of true cell counts over the 100 sampled images.We calculated it with Python, from the C values in the file names. Independent check: yes	`5007`	exact	`5007` match in 3 of 3 runsNot asked in the question	`5007` match in 3 of 3 runsNot asked in the question	`5007` match in 3 of 3 runsNot asked in the question	`5007` matchNot asked in the question

Open the runs

Research paper

Mahbod 2024

Mahbod A, Polak C, Feldmann K, et al.

NuInsSeg: A fully annotated dataset for nuclei instance segmentation in H&E-stained histological images. Scientific Data 11:295 (2024). doi:10.1038/s41597-024-03117-2

NuInsSeg, nuclei instance segmentation in H&E images

scikit-image (Python), through the scikit-image adapter

Opus3 of 3 values match, in each of 3 runs

3 of 3

In the final answer: 1 of 1, in each of 3 runs

Sonnet3 of 3 values match, in each of 3 runs

3 of 3

In the final answer: 1 of 1, in each of 3 runs

Haiku3 of 3 values match, in each of 3 runs

3 of 3

In the final answer: 1 of 1, in each of 3 runs

qwen3:8b3 of 3 values match

3 of 3

In the final answer: 1 of 1

Show each value

Known values and run values for Mahbod 2024, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Mean pixel Dice, Otsu on the hematoxylin channel.We calculated it with scikit-image 0.26.0. Independent check: yes	`0.538`	± 0.02	`0.5380504` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5380504` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5380504` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5380504` matchIn the final answer: yes
Image patches in the dataset.Printed in the paper	`665`	exact	`665` match in 3 of 3 runsNot asked in the question	`665` match in 3 of 3 runsNot asked in the question	`665` match in 3 of 3 runsNot asked in the question	`665` matchNot asked in the question
Nuclei in the released label masks.We calculated it with Python count of unique non-zero labels in the released label masks. Independent check: yes	`35138`	exact	`35138` match in 3 of 3 runsNot asked in the question	`35138` match in 3 of 3 runsNot asked in the question	`35138` match in 3 of 3 runsNot asked in the question	`35138` matchNot asked in the question

Open the runs

Research paper

Manders 1993

Manders EMM, Verbeek FJ, Aten JA.

Measurement of co-localization of objects in dual-colour confocal images. Journal of Microscopy 169(3):375-382 (1993). doi:10.1111/j.1365-2818.1993.tb03313.x

overlap coefficients on designed image pairs

scikit-image and NumPy (Python), through the image-assays adapter, which repeats the Fiji Coloc 2 calculations.

Opus6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 5 of 5, in each of 3 runs

Sonnet6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 5 of 5, in each of 3 runs

Haiku6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 5 of 5, in each of 3 runs

qwen3:8b6 of 6 values match

6 of 6

In the final answer: 5 of 5

Show each value

Known values and run values for Manders 1993, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Pearson r, A with BPrinted in the paper. Independent check: yes	`0.72`	± 0.01	`0.7212973` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7212973` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7212973` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7212973` matchIn the final answer: yes
Pearson r, A with CPrinted in the paper. Independent check: yes	`0.44`	± 0.01	`0.4425947` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4425947` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4425947` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4425947` matchIn the final answer: yes
Pearson r, A with DPrinted in the paper. Independent check: yes	`0.16`	± 0.01	`0.163892` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.163892` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.163892` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.163892` matchIn the final answer: yes
Manders M1, A with BPrinted in the official tutorial. Independent check: yes	`0.75`	± 0.0001	`0.75` match in 3 of 3 runsNot asked in the question	`0.75` match in 3 of 3 runsNot asked in the question	`0.75` match in 3 of 3 runsNot asked in the question	`0.75` matchNot asked in the question
Manders M1, A with CPrinted in the official tutorial. Independent check: yes	`0.5`	± 0.0001	`0.5` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5` matchIn the final answer: yes
Manders M2, A with DPrinted in the official tutorial. Independent check: yes	`0.25`	± 0.0001	`0.25` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.25` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.25` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.25` matchIn the final answer: yes

Open the runs

Research paper

Ram 2021

Ram S, Vizcarra P, Whalen P, et al.

Pixelwise H-score: A novel digital image analysis-based metric to quantify membrane biomarker expression from immunohistochemistry images. PLOS ONE 16(9): e0245638 (2021). doi:10.1371/journal.pone.0245638

pixelwise H-score, pathologist and digital H-scores

ihc-scoring adapter (pandas, SciPy). The paper used QuPath, HALO and Visiopharm.

No run yetno result file has this paper

No result file has this paper. It is not in the totals.

Show each value

Known values and run values for Ram 2021, earlier run of 9 October 2026
Value	Known value	Tolerance	Haiku
Cases scored for P-cadherinPrinted in the paper	`30`	exact	not in this run
Cases scored for PD-L1Printed in the paper	`24`	exact	not in this run
Cases scored for 5T4Printed in the paper	`21`	exact	not in this run
H-score of PD-L1 case 1 from its percents (0, 20, 75 and 5 percent at 0, 1+, 2+, 3+)Printed in the paper. Independent check: yes	`185`	± 0.5	not in this run
Spearman rho, P-cadherin, pathologist H-score and QuPath H-scorePrinted in the paper	`0.39`	± 0.01	not in this run
Spearman rho, P-cadherin, pathologist H-score and HALO H-scorePrinted in the paper	`0.5`	± 0.01	not in this run
Spearman rho, PD-L1, pathologist H-score and QuPath H-scorePrinted in the paper	`0.74`	± 0.01	not in this run
Spearman rho, PD-L1, pathologist H-score and HALO H-score (the paper prints 0.69, the data give 0.701)Printed in the paper	`0.69`	± 0.015	not in this run
Spearman rho, 5T4, pathologist H-score and QuPath H-scorePrinted in the paper	`0.79`	± 0.01	not in this run
Spearman rho, 5T4, pathologist H-score and HALO H-scorePrinted in the paper	`0.75`	± 0.01	not in this run
Weighted kappa (quadratic, cut points 100 and 200), P-cadherin, pathologist and QuPathWe calculated it with scikit-learn 1.9.1 cohen_kappa_score. Independent check: yes	`0.010989`	± 0.001	not in this run
ICC(2,1), P-cadherin, pathologist and QuPathWe calculated it with pingouin 0.7.0 intraclass_corr. Independent check: yes	`0.149255`	± 0.002	not in this run

Open the runs

Research paper

Rees 2019

Rees P, Wills JW, Brown MR, et al.

The origin of heterogeneous nanoparticle uptake by cells. Nature Communications 10:2341 (2019). doi:10.1038/s41467-019-10112-4

nanoparticle uptake per cell

scikit-image (Python), through the image-assays adapter. The paper used CellProfiler 2.2.

Opus5 of 5 values match, in each of 3 runs

5 of 5

In the final answer: 4 of 4, in each of 3 runs

Sonnet5 of 5 values match, in each of 3 runs

5 of 5

In the final answer: 4 of 4, in each of 3 runs

Haiku5 of 5 values match, in each of 3 runs

5 of 5

In the final answer: 4 of 4, in each of 3 runs

qwen3:8b5 of 5 values match

5 of 5

In the final answer: 4 of 4

Show each value

Known values and run values for Rees 2019, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Fields measuredPrinted in the paper	`3`	exact	`3` match in 3 of 3 runsNot asked in the question	`3` match in 3 of 3 runsNot asked in the question	`3` match in 3 of 3 runsNot asked in the question	`3` matchNot asked in the question
Nuclei in the three fields (CellProfiler 229, 216, 137)Printed in the paper. Independent check: yes	`582`	± 35	`566` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`566` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`566` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`566` matchIn the final answer: yes
Nanoparticle-loaded vesicles in the three fields (CellProfiler 189, 141, 131)Printed in the paper. Independent check: yes	`461`	± 35	`459` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`459` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`459` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`459` matchIn the final answer: yes
Mean vesicles per cell, three fields pooled (387 / 572)Printed in the paper. Independent check: yes	`0.6766`	± 0.12	`0.63953` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.63953` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.63953` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.63953` matchIn the final answer: yes
Fraction of cells with at least one vesicle, pooled (250 / 572)Printed in the paper. Independent check: yes	`0.4371`	± 0.08	`0.46512` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.46512` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4651163` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.46512` matchIn the final answer: yes
Cells after edge exclusion (CellProfiler Propagation 222, 215, 135); the watershed cells differ (reference)Printed in the paper	`572`	± 35	`566` Reference value. Not scored	`566` Reference value. Not scored	`566` Reference value. Not scored	`566` Reference value. Not scored

Open the runs

Tutorial or software test data

Schindelin 2012

Schindelin J, Arganda-Carreras I, Frise E, et al.

Fiji: an open-source platform for biological-image analysis. Nature Methods 9(7):676-682 (2012). doi:10.1038/nmeth.2019

Fiji, with the ImageJ "Blobs" sample image

scikit-image (Python), through the scikit-image adapter

Opus1 of 1 values match, in each of 3 runs

1 of 1

In the final answer: 1 of 1, in each of 3 runs

Sonnet1 of 1 values match, in each of 3 runs

1 of 1

In the final answer: 1 of 1, in each of 3 runs

Haiku1 of 1 values match, in each of 3 runs

1 of 1

In the final answer: 1 of 1, in each of 3 runs

qwen3:8b1 of 1 values match

1 of 1

In the final answer: 1 of 1

Show each value

Known values and run values for Schindelin 2012, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Particles, edge particles included.We calculated it with ImageJ 1.53 (Default threshold, Analyze Particles). Independent check: yes	`64`	exact	`64` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`64` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`64` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`64` matchIn the final answer: yes

Open the runs

Flow and mass cytometry

Research paper

Aulicino 2022

Aulicino F et al.

Highly efficient CRISPR-mediated large DNA docking and multiplexed prime editing using a single baculovirus. Nucleic Acids Research (2022). doi:10.1093/nar/gkac587

Data: doi:10.6084/m9.figshare.20110364

baculovirus delivery of large DNA inserts for CRISPR docking

flowCore (R), through the flowcore adapter

No run yetno result file has this paper

No result file has this paper. It is not in the totals.

Show each value

Known values and run values for Aulicino 2022, earlier run of 9 October 2026
Value	Known value	Tolerance	Haiku
Mean percent of mCherry-positive cells after the HDR design, day 10 (the paper prints about 5%)Printed in the paper	`5`	± 1	not in this run
Mean percent of mCherry-positive cells after the HITI-2c design, day 10 (the paper prints about 20%)Printed in the paper	`20`	± 1.5	not in this run
MCherry cutoff: 99.9th percentile of the pooled untransfected control events, PE-CF594-AWe calculated it with population_stats of the flowcore adapter. Independent check: yes	`3791.17`	± 0.5	not in this run
Mean percent mCherry-positive, HDR, three replicatesWe calculated it with compare_groups of the flowcore adapter. Independent check: yes	`4.419`	± 0.01	not in this run
SD of the percent mCherry-positive, HDRWe calculated it with compare_groups of the flowcore adapter. Independent check: yes	`0.146`	± 0.01	not in this run
Mean percent mCherry-positive, HITI-2c, three replicatesWe calculated it with compare_groups of the flowcore adapter. Independent check: yes	`20.903`	± 0.01	not in this run
SD of the percent mCherry-positive, HITI-2cWe calculated it with compare_groups of the flowcore adapter. Independent check: yes	`2.256`	± 0.02	not in this run
Welch t test p value, HITI-2c against HDRWe calculated it with compare_groups of the flowcore adapter. Independent check: yes	`0.006018`	± 0.0001	not in this run

Open the runs

Tutorial or software test data

Hahne 2009

Hahne F, LeMeur N, Brinkman RR, et al.

flowCore: a Bioconductor package for high throughput flow cytometry. BMC Bioinformatics 10:106 (2009). doi:10.1186/1471-2105-10-106

the flowCore compensation data set (compdata)

flowCore (R), through the flowcore adapter

No run yetno result file has this paper

No result file has this paper. It is not in the totals.

Show each value

Known values and run values for Hahne 2009, earlier run of 9 October 2026
Value	Known value	Tolerance	Haiku
Spillover of the FL1-H control into FL2-HPrinted in the official tutorial	`0.242022`	± 5e-05	not in this run
Spillover of the FL1-H control into FL3-HPrinted in the official tutorial	`0.032084`	± 0.0004	not in this run
Spillover of the FL1-H control into FL4-HPrinted in the official tutorial	`0.001128`	± 5e-05	not in this run
Spillover of the FL2-H control into FL1-HPrinted in the official tutorial	`0.007722`	± 5e-05	not in this run
Spillover of the FL2-H control into FL3-HPrinted in the official tutorial	`0.140788`	± 5e-05	not in this run
Spillover of the FL2-H control into FL4-HPrinted in the official tutorial	`0.002633`	± 0.0002	not in this run
Spillover of the FL3-H control into FL1-HPrinted in the official tutorial	`0.015081`	± 5e-05	not in this run
Spillover of the FL3-H control into FL2-HPrinted in the official tutorial	`0.17559`	± 5e-05	not in this run
Spillover of the FL3-H control into FL4-HPrinted in the official tutorial	`0.229594`	± 5e-05	not in this run
Spillover of the FL4-H control into FL1-HPrinted in the official tutorial	`0.000759`	± 5e-05	not in this run
Spillover of the FL4-H control into FL2-HPrinted in the official tutorial	`0.000962`	± 5e-05	not in this run
Spillover of the FL4-H control into FL3-HPrinted in the official tutorial	`0.003219`	± 5e-05	not in this run

Open the runs

Research paper

Nowicka 2019

Nowicka M, Krieg C, Crowell HL, et al.

CyTOF workflow: differential discovery in high-throughput high-dimensional cytometry datasets. F1000Research 6:748, version 3 (2019). doi:10.12688/f1000research.11622.3

Also: Weber LM, Nowicka M, Soneson C, Robinson MD. diffcyt: Differential discovery in high-dimensional cytometry via high-resolution clustering. Commun Biol 2:183 (2019). doi:10.1038/s42003-019-0415-5

CyTOF workflow, differential abundance of blood cell types after BCR-XL stimulation

FlowSOM, diffcyt and lme4 (R), through the cytof adapter

No run yetno result file has this paper

No result file has this paper. It is not in the totals.

Show each value

Known values and run values for Nowicka 2019, earlier run of 9 October 2026
Value	Known value	Tolerance	Haiku
Cells in the file of donor 2 with BCR-XLPrinted in the paper. Independent check: yes	`16675`	exact	not in this run
Cells in the file of donor 1 ReferencePrinted in the paper. Independent check: yes	`2739`	exact	not in this run
Percent of NK cells in the Reference file of donor 1Printed in the paper. Independent check: yes	`14.3`	± 0.05	not in this run
Percent of CD4 T-cells in the Reference file of donor 1Printed in the paper. Independent check: yes	`44.7`	± 0.05	not in this run
P value of the NK cells, paired modelPrinted in the paper. Independent check: yes	`4.5e-13`	± 1e-14	not in this run
P value of the B-cells IgM-, paired modelPrinted in the paper. Independent check: yes	`2.2e-11`	± 1e-12	not in this run
Adjusted p value of the CD8 T-cells, paired modelPrinted in the paper. Independent check: yes	`0.0019`	± 0.0001	not in this run
Cell types with adjusted p at or below 0.05, paired modelPrinted in the paper. Independent check: yes	`6`	exact	not in this run
Cell types with adjusted p at or below 0.05, model without the pairingPrinted in the paper. Independent check: yes	`3`	exact	not in this run
P value of the B-cells IgM+, model without the pairingPrinted in the paper. Independent check: yes	`0.0135`	± 0.0005	not in this run

Open the runs

Tutorial or software test data

White 2021

White S, Quinn J, Enzor J, et al.

FlowKit: A Python Toolkit for Integrated Manual and Automated Cytometry Analysis Workflows. Frontiers in Immunology 12:768541 (2021). doi:10.3389/fimmu.2021.768541

FlowKit, a Python toolkit for flow cytometry gating

FlowKit (Python), through the flowkit adapter

Opus12 of 12 values match, in each of 3 runs

12 of 12

In the final answer: 10 of 10, in each of 3 runs

Sonnet12 of 12 values match, in each of 3 runs

12 of 12

In the final answer: 9 to 10 of 10, over 3 runs

Haiku12 of 12 values match, in each of 3 runs

12 of 12

In the final answer: 10 of 10, in each of 3 runs

qwen3:8b8 of 12 values match

8 of 12

In the final answer: 4 of 10

Show each value

Known values and run values for White 2021, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Events in the filePrinted in the official tutorial. Independent check: yes	`290172`	exact	`290172` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`290172` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`290172` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`290172` matchIn the final answer: yes
CD3+ events, FlowKit, matrix of the workspacePrinted in the official tutorial. Independent check: yes	`133670`	exact	`133670` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`133670` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`133670` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`133670` matchIn the final answer: yes
CD3+ events, FlowJo count in the workspaceWe calculated it with FlowKit 1.3.2, read from the workspace file. Independent check: yes	`133860`	exact	`133860` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`133860` match in 3 of 3 runsIn the final answer: yes in 1 of 3 runs	`133860` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`133860` matchIn the final answer: no
CD4+ events, FlowKitPrinted in the official tutorial. Independent check: yes	`82484`	exact	`82484` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`82484` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`82484` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`82484` matchIn the final answer: yes
CD4+ events, FlowJoWe calculated it with FlowKit 1.3.2, read from the workspace file. Independent check: yes	`82636`	exact	`82636` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`82636` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`82636` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`82636` matchIn the final answer: no
CD8+ events, FlowKitPrinted in the official tutorial. Independent check: yes	`47165`	exact	`47165` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`47165` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`47165` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`47165` matchIn the final answer: yes
CD8+ events, FlowJoWe calculated it with FlowKit 1.3.2, read from the workspace file. Independent check: yes	`47241`	exact	`47241` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`47241` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`47241` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`47241` matchIn the final answer: no
Largest difference FlowKit minus FlowJo, percent of FlowJo count (CD4+)We calculated it with FlowKit 1.3.2. Independent check: yes	`-0.184`	± 0.02	`-0.1839392` match in 3 of 3 runsNot asked in the question	`-0.1839392` match in 3 of 3 runsNot asked in the question	`-0.1839392` match in 3 of 3 runsNot asked in the question	`-0.1839392` matchNot asked in the question
CD3+ events, no compensationWe calculated it with FlowKit 1.3.2. Independent check: yes	`135381`	exact	`135381` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`135381` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`135381` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`133860` no matchIn the final answer: no
CD4+ events, no compensationWe calculated it with FlowKit 1.3.2. Independent check: yes	`85369`	exact	`85369` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`85369` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`85369` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`82636` no matchIn the final answer: no
CD8+ events, no compensationWe calculated it with FlowKit 1.3.2. Independent check: yes	`47694`	exact	`47694` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`47694` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`47694` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`47241` no matchIn the final answer: no
CD8+ IFNg+ events, no compensation (2 with the matrix)We calculated it with FlowKit 1.3.2. Independent check: yes	`672`	exact	`672` match in 3 of 3 runsNot asked in the question	`672` match in 3 of 3 runsNot asked in the question	`672` match in 3 of 3 runsNot asked in the question	`690` no matchNot asked in the question

Open the runs

Mass spectrometry

Research paper

Alexandrov 2019

Alexandrov T, Ovchinnikova K, Palmer A, et al.

METASPACE: a community-populated knowledge base of spatial metabolomes in health and disease. bioRxiv preprint (2019). doi:10.1101/539478

Also: Palmer A, et al. FDR-controlled metabolite annotation for high-resolution imaging mass spectrometry. Nature Methods 14:57-60 (2017). doi:10.1038/nmeth.4072

METASPACE, a community knowledge base of spatial metabolomes

METASPACE (metaspace2020 Python client), through the metaspace adapter

Opus5 of 5 values match, in each of 3 runs

5 of 5

In the final answer: 4 of 4, in each of 3 runs

Sonnet5 of 5 values match, in each of 3 runs

5 of 5

In the final answer: 4 of 4, in each of 3 runs

Haiku5 of 5 values match, in each of 3 runs

5 of 5

In the final answer: 2 to 4 of 4, over 3 runs

qwen3:8b4 of 5 values match

4 of 5

In the final answer: 2 of 4

Show each value

Known values and run values for Alexandrov 2019, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Dataset 10h14m23s, LipidMaps annotations at FDR 10%.We calculated it with METASPACE, queried with metaspace2020 2.0.9 on 2026-10-07. Check with the same program	`414`	exact	`414` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`414` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`414` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`414` matchIn the final answer: yes
Dataset 10h14m23s, HMDB annotations at FDR 10%.We calculated it with METASPACE, queried with metaspace2020 2.0.9 on 2026-10-07. Check with the same program	`263`	exact	`263` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`263` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`263` match in 3 of 3 runsIn the final answer: yes in 2 of 3 runs	`263` matchIn the final answer: no
Dataset 12h31m33s, LipidMaps annotations at FDR 10%.We calculated it with METASPACE, queried with metaspace2020 2.0.9 on 2026-10-07. Check with the same program	`191`	exact	`191` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`191` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`191` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`191` matchIn the final answer: yes
Dataset 12h31m33s, HMDB annotations at FDR 10%.We calculated it with METASPACE, queried with metaspace2020 2.0.9 on 2026-10-07. Check with the same program	`123`	exact	`123` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`123` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`123` match in 3 of 3 runsIn the final answer: yes in 2 of 3 runs	`144` no matchIn the final answer: no
Dataset 10h14m23s, LipidMaps annotations at FDR 5%.We calculated it with METASPACE, queried with metaspace2020 2.0.9 on 2026-10-07. Check with the same program	`144`	exact	`144` match in 3 of 3 runsNot asked in the question	`144` match in 3 of 3 runsNot asked in the question	`144` match in 3 of 3 runsNot asked in the question	`144` matchNot asked in the question

Open the runs

Tutorial or software test data

Bemis 2015

Bemis KD, Harry A, Eberlin LS, et al.

Cardinal: an R package for statistical analysis of mass spectrometry-based imaging experiments. Bioinformatics 31(14):2418-2420 (2015). doi:10.1093/bioinformatics/btv146

Cardinal, an R package for statistical analysis of MS imaging experiments

Cardinal (R, Bioconductor), through the cardinal adapter

Opus5 of 5 values match, in each of 3 runs

5 of 5

In the final answer: 5 of 5, in each of 3 runs

Sonnet5 of 5 values match, in each of 3 runs

5 of 5

In the final answer: 5 of 5, in each of 3 runs

Haiku4 of 5 values match, in each of 3 runs

4 of 5

In the final answer: 4 of 5, in each of 3 runs

qwen3:8b5 of 5 values match

5 of 5

In the final answer: 5 of 5

Show each value

Known values and run values for Bemis 2015, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Spectra in pig206.Printed in the official tutorial	`4959`	exact	`4959` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`4959` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`4959` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`4959` matchIn the final answer: yes
M/z values in pig206.Printed in the official tutorial	`10200`	exact	`10200` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10200` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10200` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10200` matchIn the final answer: yes
Peaks after peak processing.Printed in the official tutorial	`687`	± 10	`687` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`687` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`727` no match in 3 of 3 runsIn the final answer: no in 3 of 3 runs	`687` matchIn the final answer: yes
Segments in the chosen model (s=32).Printed in the official tutorial	`6`	± 1	`6` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`6` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`6` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`6` matchIn the final answer: yes
Segments in the s=64 model.Printed in the official tutorial	`3`	± 1	`3` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3` matchIn the final answer: yes

Open the runs

Tutorial or software test data

Lazear 2023

Lazear MR.

Sage: An Open-Source Tool for Fast Proteomics Searching and Quantification at Scale. J Proteome Res 22(11):3652-3659 (2023). doi:10.1021/acs.jproteome.3c00486

Sage, an open-source tool for fast proteomics searching and quantification at scale

Sage (command line), through the sage adapter

Opus4 to 5 of 6 values match, over 3 runs

4 to 5 of 6

In the final answer: 1 of 3, in each of 3 runs

Sonnet4 to 5 of 6 values match, over 3 runs

4 to 5 of 6

In the final answer: 1 to 2 of 3, over 3 runs

Haiku6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 3 of 3, in each of 3 runs

qwen3:8b5 of 6 values match

5 of 6

In the final answer: 2 of 3

Show each value

Known values and run values for Lazear 2023, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
PSMs in the result tableWe calculated it with Sage 0.14.6 (release binary v0.14.7). Check with the same program	`303`	exact	`303` match in 3 of 3 runsNot asked in the question	`303` match in 3 of 3 runsNot asked in the question	`303` match in 3 of 3 runsNot asked in the question	`303` matchNot asked in the question
Decoy PSMs in the result tableWe calculated it with Sage 0.14.6 (release binary v0.14.7). Check with the same program	`43`	exact	`43` match in 3 of 3 runsNot asked in the question	`43` match in 3 of 3 runsNot asked in the question	`43` match in 3 of 3 runsNot asked in the question	`43` matchNot asked in the question
Target PSMs at 1 percent PSM FDRWe calculated it with Sage 0.14.6 (release binary v0.14.7). Independent check: yes	`207`	exact	`206` no match in 2 of 3 runsIn the final answer: no in 3 of 3 runs	`206` no match in 2 of 3 runsIn the final answer: yes in 1 of 3 runs	`207` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`207` matchIn the final answer: yes
Distinct peptides among the PSMs at 1 percentWe calculated it with Sage 0.14.6 (release binary v0.14.7). Independent check: yes	`47`	exact	`47` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`47` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`47` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`47` matchIn the final answer: yes
Decoy PSMs among the kept PSMsWe calculated it with Sage 0.14.6 (release binary v0.14.7). Independent check: yes	`1`	exact	`1` match in 3 of 3 runsNot asked in the question	`1` match in 3 of 3 runsNot asked in the question	`1` match in 3 of 3 runsNot asked in the question	`1` matchNot asked in the question
Target PSMs at 5 percent PSM FDRWe calculated it with Sage 0.14.6 (release binary v0.14.7). Independent check: yes	`242`	exact	`244` no match in 3 of 3 runsIn the final answer: no in 3 of 3 runs	`244` no match in 3 of 3 runsIn the final answer: no in 3 of 3 runs	`242` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`260` no matchIn the final answer: no

Open the runs

Research paper

Palmer 2017

Palmer A, Phapale P, Chernyavsky I, et al.

FDR-controlled metabolite annotation for high-resolution imaging mass spectrometry. Nature Methods 14(1):57-60 (2017). doi:10.1038/nmeth.4072

FDR-controlled metabolite annotation for high-resolution imaging MS

METASPACE (metaspace2020 Python client), through the metaspace adapter

Opusno scored values

Sonnetno scored values

Haikuno scored values

qwen3:8bno scored values

This paper has no scored values. Its values are reference values only.

Show each value

Known values and run values for Palmer 2017, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Sum formulas annotated in all three serial sections of animal a2. (reference)Printed in the paper	`51`	± 8	`48` Reference value. Not scored	no numberReference value. Not scored	no numberReference value. Not scored	no numberReference value. Not scored
Unique formulas in sections a2s1, a2s2 and a2s3, computed from the MAF table. (reference)We calculated it with Count of the MTBLS313 annotation table (MAF file) by the benchmark authors. Independent check: yes	`66`	exact	`66` Reference value. Not scored	`61` Reference value. Not scored	`61` Reference value. Not scored	`61` Reference value. Not scored
Total sum formulas annotated in all datasets. (reference)Printed in the paper	`103`	± 20	`100` Reference value. Not scored	`61` Reference value. Not scored	`100` Reference value. Not scored	`61` Reference value. Not scored

Open the runs

Tutorial or software test data

Röst 2016

Röst HL, Sachsenberg T, Aiche S, et al.

OpenMS: a flexible open-source software platform for mass spectrometry data analysis. Nature Methods 13(9):741-748 (2016). doi:10.1038/nmeth.3959

OpenMS, an open-source platform for mass spectrometry data analysis

pyOpenMS (Python interface to OpenMS), through the pyopenms adapter

Opus5 of 5 values match, in each of 3 runs

5 of 5

In the final answer: 2 of 2, in each of 3 runs

Sonnet5 of 5 values match, in each of 3 runs

5 of 5

In the final answer: 2 of 2, in each of 3 runs

Haiku5 of 5 values match, in each of 3 runs

5 of 5

In the final answer: 2 of 2, in each of 3 runs

qwen3:8b5 of 5 values match

5 of 5

In the final answer: 1 of 2

Show each value

Known values and run values for Röst 2016, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Spectra in PStd_050_1We calculated it with pyOpenMS 3.6.0. Independent check: yes	`7173`	exact	`7173` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7173` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7173` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7173` matchIn the final answer: no
MS1 spectra in PStd_050_1We calculated it with pyOpenMS 3.6.0. Independent check: yes	`7173`	exact	`7173` match in 3 of 3 runsNot asked in the question	`7173` match in 3 of 3 runsNot asked in the question	`7173` match in 3 of 3 runsNot asked in the question	`7173` matchNot asked in the question
Mass traces in PStd_050_1 with the paper parametersWe calculated it with pyOpenMS 3.6.0. Check with the same program	`1880`	± 100	`1880` match in 3 of 3 runsNot asked in the question	`1880` match in 3 of 3 runsNot asked in the question	`1880` match in 3 of 3 runsNot asked in the question	`1880` matchNot asked in the question
Elution peaks in PStd_050_1We calculated it with pyOpenMS 3.6.0. Independent check: yes	`2255`	± 120	`2255` match in 3 of 3 runsNot asked in the question	`2255` match in 3 of 3 runsNot asked in the question	`2255` match in 3 of 3 runsNot asked in the question	`2255` matchNot asked in the question
Features in PStd_050_1We calculated it with pyOpenMS 3.6.0. Independent check: yes	`1887`	± 100	`1887` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1887` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1887` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1887` matchIn the final answer: yes

Open the runs

Tutorial or software test data

Schramm 2012

Schramm T, Hester Z, Klinkert I, et al.

imzML - a common data format for the flexible exchange and processing of mass spectrometry imaging data. Journal of Proteomics 75(16):5106-5110 (2012). doi:10.1016/j.jprot.2012.07.026

imzML, a common data format for mass spectrometry imaging data

pyimzML (Python), through the pyimzml adapter

Opus7 of 7 values match, in each of 3 runs

7 of 7

In the final answer: 4 of 4, in each of 3 runs

Sonnet7 of 7 values match, in each of 3 runs

7 of 7

In the final answer: 4 of 4, in each of 3 runs

Haiku7 of 7 values match, in each of 3 runs

7 of 7

In the final answer: 4 of 4, in each of 3 runs

qwen3:8b5 of 7 values match

5 of 7

In the final answer: 3 of 4

Show each value

Known values and run values for Schramm 2012, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Spectra (pixels) in each file.Printed in the official tutorial	`9`	exact	`9` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`9` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`9` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`9` matchIn the final answer: yes
Minimum m/z.We calculated it with pyimzML 1.5.5. Independent check: yes	`100.0833`	± 0.0001	`100.0833` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`100.0833` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`100.0833` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`100.0833` matchIn the final answer: yes
Maximum m/z.We calculated it with pyimzML 1.5.5. Independent check: yes	`799.9167`	± 0.0001	`799.9167` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`799.9167` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`799.9167` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`799.9167` matchIn the final answer: yes
Total ion current of the strongest pixel.We calculated it with pyimzML 1.5.5 and numpy. Independent check: yes	`243.5395`	± 0.001	`243.5395` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`243.5395` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`243.5395` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`100.0833` no matchIn the final answer: no
Mean total ion current per pixel.We calculated it with pyimzML 1.5.5 and numpy. Independent check: yes	`161.1444`	± 0.001	`161.1444` match in 3 of 3 runsNot asked in the question	`161.1444` match in 3 of 3 runsNot asked in the question	`161.1444` match in 3 of 3 runsNot asked in the question	`100.0833` no matchNot asked in the question
Size of the continuous ibd file in bytes.We calculated it with file size of the downloaded file. Independent check: yes	`335976`	exact	`335976` match in 3 of 3 runsNot asked in the question	`335976` match in 3 of 3 runsNot asked in the question	`335976` match in 3 of 3 runsNot asked in the question	`335976` matchNot asked in the question
Size of the processed ibd file in bytes.We calculated it with file size of the downloaded file. Independent check: yes	`604744`	exact	`604744` match in 3 of 3 runsNot asked in the question	`604744` match in 3 of 3 runsNot asked in the question	`604744` match in 3 of 3 runsNot asked in the question	`604744` matchNot asked in the question

Open the runs

Research paper

Thaqi 2026

Thaqi G, Chiang DM, Wudy SI, et al.

Bovine Corpus Luteum Proteomics during Different Reproductive and Physiological Stages. Scientific Data 13: 826 (2026). doi:10.1038/s41597-026-07515-6

Data: doi:10.5281/zenodo.18680101

bovine corpus luteum proteomics

limma (R), through the limma-proteomics adapter

No run yetno result file has this paper

No result file has this paper. It is not in the totals.

Show each value

Known values and run values for Thaqi 2026, earlier run of 9 October 2026
Value	Known value	Tolerance	Haiku
Protein groups after the removal of the flagged rowsPrinted in the paper	`3783`	exact	not in this run
Proteins higher in stage I than in stage IVPrinted in the paper	`282`	exact	not in this run
Proteins higher in stage IV than in stage IPrinted in the paper	`268`	exact	not in this run
Proteins higher in stage III than in stage VPrinted in the paper	`240`	exact	not in this run
Proteins higher in stage V than in stage IIIPrinted in the paper	`240`	exact	not in this run
Proteins higher in stage IV than in stage VIPrinted in the paper	`98`	exact	not in this run
Proteins higher in stage VI than in stage IVPrinted in the paper	`131`	exact	not in this run
Proteins higher in stage V than in stage VIPrinted in the paper	`74`	exact	not in this run
Proteins higher in stage VI than in stage VPrinted in the paper	`109`	exact	not in this run
Proteins higher in stage VI than in stage VIIPrinted in the paper	`219`	exact	not in this run
Proteins higher in stage VII than in stage VIPrinted in the paper	`240`	exact	not in this run
Proteins higher in stage VII than in stage XPrinted in the paper	`138`	exact	not in this run
Proteins higher in stage X than in stage VIIPrinted in the paper	`125`	exact	not in this run
Proteins tested after the peptide and valid value filtersWe calculated it with prepare_protein_table of the limma-proteomics adapter. Independent check: yes	`1908`	exact	not in this run

Open the runs

Tutorial or software test data

Thevenot 2015

Thevenot EA, Roux A, Xu Y, et al.

Analysis of the human adult urinary metabolome variations with age, body mass index, and gender by implementing a comprehensive workflow for univariate and OPLS statistical analyses. Journal of Proteome Research 14(8):3322-3335 (2015). doi:10.1021/acs.jproteome.5b00354

ropls, urine metabolome of 183 adults

ropls (R), through the metabolomics-stats adapter

No run yetno result file has this paper

No result file has this paper. It is not in the totals.

Show each value

Known values and run values for Thevenot 2015, earlier run of 9 October 2026
Value	Known value	Tolerance	Haiku
Number of samplesPrinted in the official tutorial	`183`	exact	not in this run
Cumulative R2X of the PCA (standard scaling)Printed in the official tutorial	`0.501`	± 0.002	not in this run
Number of PCA components that ropls keepsPrinted in the official tutorial	`8`	exact	not in this run
Cumulative R2X of the PLS-DA of genderPrinted in the official tutorial	`0.275`	± 0.002	not in this run
Cumulative R2Y of the PLS-DA of genderPrinted in the official tutorial	`0.73`	± 0.005	not in this run
Cumulative Q2 of the PLS-DA of genderPrinted in the official tutorial	`0.584`	± 0.005	not in this run
RMSEE of the PLS-DA of genderPrinted in the official tutorial	`0.262`	± 0.003	not in this run
Number of PLS-DA componentsPrinted in the official tutorial	`3`	exact	not in this run
Cumulative Q2 of the OPLS-DA of genderPrinted in the official tutorial	`0.602`	± 0.005	not in this run
Number of orthogonal components of the OPLS-DAPrinted in the official tutorial	`2`	exact	not in this run
Permutation p value of Q2 with 20 permutations (the smallest possible value)Printed in the official tutorial	`0.05`	± 0.005	not in this run
R2Y of the OPLS-DA trained on the odd rowsPrinted in the official tutorial	`0.825`	± 0.005	not in this run
Q2 of the OPLS-DA trained on the odd rowsPrinted in the official tutorial	`0.608`	± 0.005	not in this run
RMSEP of the OPLS-DA on the test rowsPrinted in the official tutorial	`0.341`	± 0.003	not in this run
Correct calls in the test set of 91 samplesPrinted in the official tutorial	`77`	exact	not in this run
R2Y of the OPLS regression of agePrinted in the official tutorial	`0.476`	± 0.005	not in this run
Q2 of the OPLS regression of agePrinted in the official tutorial	`0.31`	± 0.005	not in this run
RMSEE of the OPLS regression of age (years)Printed in the official tutorial	`7.53`	± 0.02	not in this run
Metabolites with an adjusted p below 0.05 (Welch test)We calculated it with SciPy (checks/check_metab.py in the adapter) and R t.test (check.R). Independent check: yes	`42`	exact	not in this run
Significant metabolites that are higher in menWe calculated it with SciPy and R. Independent check: yes	`11`	exact	not in this run
Significant metabolites that are lower in menWe calculated it with SciPy and R. Independent check: yes	`31`	exact	not in this run

Open the runs

Research paper

Zhu 2020

Zhu Y, Orre LM, Zhou Tran Y, et al.

DEqMS: a method for accurate variance estimation in differential protein expression analysis. Molecular and Cellular Proteomics 19(6): 1047-1057 (2020). doi:10.1074/mcp.TIR119.001646

Data: doi:10.1074/mcp.M113.031591

DEqMS, label-free spike-in benchmark

limma (R), through the limma-proteomics adapter

No run yetno result file has this paper

No result file has this paper. It is not in the totals.

Show each value

Known values and run values for Zhu 2020, earlier run of 9 October 2026
Value	Known value	Tolerance	Haiku
Protein groups after the removal of Reverse and Contaminant rowsPrinted in the paper	`6566`	exact	not in this run
Proteins with at least two values in each conditionPrinted in the paper	`5022`	exact	not in this run
E. coli proteins higher at the higher E. coli amount, limma with trend, adjusted p below 0.01 (the paper prints 1230; the tolerance of 5 is the known difference of this data version)Printed in the paper	`1230`	± 5	not in this run
Human proteins higher at the higher E. coli amount, limma with trend, adjusted p below 0.01Printed in the paper	`13`	exact	not in this run
True positives of DEqMS in the paper (a different method; the tool does not run it) (reference)Printed in the paper	`1237`	± 0.5	not in this run
False positives of DEqMS in the paper (a different method) (reference)Printed in the paper	`11`	± 0.5	not in this run

Open the runs

Genomics

Tutorial or software test data

Andrews 2010

Andrews S.

FastQC: a quality control tool for high throughput sequence data. Babraham Bioinformatics, software documentation (2010). https://www.bioinformatics.babraham.ac.uk/projects/fastqc/

FastQC, official example reports

FastQC (Java command line), through the fastqc adapter

Opus7 of 7 values match, in each of 3 runs

7 of 7

In the final answer: 3 of 3, in each of 3 runs

Sonnet7 of 7 values match, in each of 3 runs

7 of 7

In the final answer: 3 of 3, in each of 3 runs

Haiku7 of 7 values match, in each of 3 runs

7 of 7

In the final answer: 3 of 3, in each of 3 runs

qwen3:8b7 of 7 values match

7 of 7

In the final answer: 3 of 3

Show each value

Known values and run values for Andrews 2010, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Good file total sequencesPrinted in the official tutorial	`250000`	exact	`250000` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`250000` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`250000` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`250000` matchIn the final answer: yes
Bad file total sequencesPrinted in the official tutorial	`395288`	exact	`395288` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`395288` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`395288` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`395288` matchIn the final answer: yes
Sequence length, both filesPrinted in the official tutorial	`40`	exact	`40` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`40` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`40` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`40` matchIn the final answer: yes
Good file %GCPrinted in the official tutorial	`45`	± 1	`45` match in 3 of 3 runsNot asked in the question	`45` match in 3 of 3 runsNot asked in the question	`45` match in 3 of 3 runsNot asked in the question	`45` matchNot asked in the question
Bad file %GCPrinted in the official tutorial	`47`	± 1	`47` match in 3 of 3 runsNot asked in the question	`47` match in 3 of 3 runsNot asked in the question	`47` match in 3 of 3 runsNot asked in the question	`47` matchNot asked in the question
Good file modules not PASSPrinted in the official tutorial	`0`	exact	`0` match in 3 of 3 runsNot asked in the question	`0` match in 3 of 3 runsNot asked in the question	`0` match in 3 of 3 runsNot asked in the question	`0` matchNot asked in the question
Bad file modules with WARNINGPrinted in the official tutorial	`4`	exact	`4` match in 3 of 3 runsNot asked in the question	`4` match in 3 of 3 runsNot asked in the question	`4` match in 3 of 3 runsNot asked in the question	`4` matchNot asked in the question

Open the runs

Research paper

Argelaguet 2018

Argelaguet R, Velten B, Arnol D, et al.

Multi-Omics Factor Analysis: a framework for unsupervised integration of multi-omics data sets. Molecular Systems Biology 14(6): e8124 (2018). doi:10.15252/msb.20178124

Data: doi:10.1172/JCI93801

MOFA on chronic lymphocytic leukemia

MOFA+ (mofapy2), through the mofa adapter

No run yetno result file has this paper

No result file has this paper. It is not in the totals.

Show each value

Known values and run values for Argelaguet 2018, earlier run of 9 October 2026
Value	Known value	Tolerance	Haiku
Factors that MOFA keeps (minimum explained variance 2%)Printed in the paper	`10`	exact	not in this run
Variance of the drug response view explained by all factors, percent (the paper prints 41; the tolerance of 7 covers MOFA+ with one start against MOFA with the best of 25)Printed in the paper	`41`	± 7	not in this run
Variance of the mRNA view explained by all factors, percentPrinted in the paper	`38`	± 3	not in this run
Variance of the methylation view explained by all factors, percentPrinted in the paper	`24`	± 3	not in this run
Variance of the mutation view explained by all factors, percent (the paper prints 24; the tolerance of 6 covers the difference of the two programs)Printed in the paper	`24`	± 6	not in this run
Number of the factor that aligns with IGHV statusPrinted in the paper	`1`	exact	not in this run
Number of the factor that aligns with trisomy 12Printed in the paper	`2`	exact	not in this run

Open the runs

Research paper

Astapova 2021

Astapova O, Seger C, Hammes SR.

Ligand Binding Prolongs Androgen Receptor Protein Half-Life by Reducing its Degradation. Journal of the Endocrine Society 5(5):bvab035 (2021). doi:10.1210/jendso/bvab035

DHT and gene expression in KGN granulosa cells

DESeq2 (R), through the deseq2 adapter

Opus4 of 4 values match, in each of 3 runs

4 of 4

In the final answer: 3 of 3, in each of 3 runs

Sonnet4 of 4 values match, in each of 3 runs

4 of 4

In the final answer: 3 of 3, in each of 3 runs

Haiku4 of 4 values match, in each of 3 runs

4 of 4

In the final answer: 3 of 3, in each of 3 runs

qwen3:8b4 of 4 values match

4 of 4

In the final answer: 3 of 3

Show each value

Known values and run values for Astapova 2021, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Genes with padj < 0.05, DHT vs vehicle, paired designPrinted in the paper	`173`	± 2	`173` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`173` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`173` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`173` matchIn the final answer: yes
Genes up with DHT at padj < 0.05Printed in the paper	`125`	± 2	`125` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`125` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`125` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`125` matchIn the final answer: yes
Genes down with DHT at padj < 0.05Printed in the paper	`48`	± 2	`48` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`48` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`48` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`48` matchIn the final answer: yes
Genes in the count tableWe calculated it with R read.delim on the GEO count file (check_deseq2.R). Independent check: yes	`59087`	exact	`59087` match in 3 of 3 runsNot asked in the question	`59087` match in 3 of 3 runsNot asked in the question	`59087` match in 3 of 3 runsNot asked in the question	`59087` matchNot asked in the question

Open the runs

Tutorial or software test data

Danecek 2021

Danecek P, Bonfield JK, Liddle J, et al.

Twelve years of SAMtools and BCFtools. GigaScience 10(2):giab008 (2021). doi:10.1093/gigascience/giab008

Twelve years of SAMtools and BCFtools

bcftools (command line), through the bcftools adapter

Opus6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 5 of 5, in each of 3 runs

Sonnet6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 5 of 5, in each of 3 runs

Haiku6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 5 of 5, in each of 3 runs

qwen3:8b6 of 6 values match

6 of 6

In the final answer: 4 of 5

Show each value

Known values and run values for Danecek 2021, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
SNP recordsWe calculated it with bcftools 1.24 (htslib 1.24), bcftools stats. Independent check: yes	`1955`	exact	`1955` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1955` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1955` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1955` matchIn the final answer: yes
Indel recordsWe calculated it with bcftools 1.24 (htslib 1.24), bcftools stats. Independent check: yes	`248`	exact	`248` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`248` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`248` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`248` matchIn the final answer: yes
Total recordsWe calculated it with bcftools 1.24 (htslib 1.24), bcftools stats. Independent check: yes	`2210`	exact	`2210` match in 3 of 3 runsNot asked in the question	`2210` match in 3 of 3 runsNot asked in the question	`2210` match in 3 of 3 runsNot asked in the question	`2210` matchNot asked in the question
Ts/Tv ratio of the SNPsWe calculated it with bcftools 1.24 (htslib 1.24), bcftools stats. Independent check: yes	`2.2`	± 0.01	`2.2` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.2` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.2` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.2` matchIn the final answer: yes
Records in chr20:1000000-1050000We calculated it with bcftools 1.24 (htslib 1.24), bcftools stats -r. Independent check: yes	`1137`	exact	`1137` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1137` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1137` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1137` matchIn the final answer: no
Records with INFO/AF at least 0.05We calculated it with bcftools 1.24 (htslib 1.24), bcftools view -i. Independent check: yes	`328`	exact	`328` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`328` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`328` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`328` matchIn the final answer: yes

Open the runs

Research paper

El Masri 2026

El Masri D et al.

Early-life microbiota in the LIMIT cohort: unveiling meconium microbiota types beyond maternal lifestyle. Gut Microbes 18(1) (2026). doi:10.1080/19490976.2026.2712811

LIMIT cohort, meconium microbiota, MaAsLin 2

MaAsLin 2 (R), through the microbiome adapter

No run yetno result file has this paper

No result file has this paper. It is not in the totals.

Show each value

Known values and run values for El Masri 2026, earlier run of 9 October 2026
Value	Known value	Tolerance	Haiku
Meconium samples in the studyPrinted in the paper. Independent check: yes	`168`	exact	not in this run
Samples in the model after the filter for vaginal delivery and the removal of missing valuesPrinted in the paper. Independent check: yes	`132`	exact	not in this run
Associations with q at or below 0.05Printed in the paper. Independent check: yes	`1`	exact	not in this run
Adjusted p value (q) of the most significant OTUPrinted in the paper. Independent check: yes	`0.043`	± 0.0005	not in this run
Samples that contain the most significant OTUPrinted in the paper. Independent check: yes	`7`	exact	not in this run
OTUs tested after the prevalence limit (not printed in the paper) (reference)We calculated it with MaAsLin2 1.26.0 through the microbiome adapter. Independent check: yes	`65`	exact	not in this run
P value of the Kruskal-Wallis test of Shannon diversity by GWG categoryWe calculated it with vegan 2.7.6 and R wilcox and kruskal tests through the microbiome adapter. Independent check: yes	`0.9917`	± 0.005	not in this run
PERMANOVA R2 of GWG (Bray-Curtis, relative abundances)We calculated it with vegan 2.7.6 adonis2 through the microbiome adapter. Independent check: yes	`0.0104`	± 0.0005	not in this run
PERMANOVA F of GWGWe calculated it with vegan 2.7.6 adonis2 through the microbiome adapter. Independent check: yes	`0.678`	± 0.005	not in this run
PERMANOVA p of GWG, 999 permutationsWe calculated it with vegan 2.7.6 adonis2, 999 permutations. Independent check: yes	`0.91`	± 0.06	not in this run

Open the runs

Research paper

Korsunsky 2019

Korsunsky I, Millard N, Fan J, et al.

Fast, sensitive and accurate integration of single-cell data with Harmony. Nature Methods 16(12):1289-1296 (2019). doi:10.1038/s41592-019-0619-0

Harmony, integration of Jurkat and 293T cell line data sets

scanpy and harmonypy (Python), through the scanpy and harmony adapters

Opus5 of 5 values match, in each of 3 runs

5 of 5

In the final answer: 5 of 5, in each of 3 runs

Sonnet5 of 5 values match, in each of 3 runs

5 of 5

In the final answer: 5 of 5, in each of 3 runs

Haiku5 of 5 values match, in each of 3 runs

5 of 5

In the final answer: 5 of 5, in each of 3 runs

qwen3:8b4 of 5 values match

4 of 5

In the final answer: 3 of 5

Show each value

Known values and run values for Korsunsky 2019, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Median iLISI of the data sets before HarmonyPrinted in the paper. Independent check: yes	`1.01`	± 0.1	`1.002393` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.006729` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.005921` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.006138` matchIn the final answer: no
Median iLISI of the data sets after HarmonyPrinted in the paper. Independent check: yes	`1.59`	± 0.1	`1.611606` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.611606` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.617978` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.528935` matchIn the final answer: no
Median cLISI of the cell lines before HarmonyPrinted in the paper. Independent check: yes	`1`	± 0.02	`1` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1` matchIn the final answer: yes
Median cLISI of the cell lines after HarmonyPrinted in the paper. Independent check: yes	`1`	± 0.02	`1` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1` matchIn the final answer: yes
Cells of the pure Jurkat data set after quality controlPrinted in the paper. Independent check: yes	`3255`	exact	`3255` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3255` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3255` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	no matchIn the final answer: yes
Cells of the pure 293T data set after quality control (paper count) (reference)Printed in the paper. Independent check: yes	`2859`	exact	`2885` Reference value. Not scored	`2885` Reference value. Not scored	`2885` Reference value. Not scored	no numberReference value. Not scored
Jurkat cells in the 50:50 mix (paper count) (reference)Printed in the paper. Independent check: yes	`1799`	exact	`1773` Reference value. Not scored	`1673` Reference value. Not scored	`1773` Reference value. Not scored	no numberReference value. Not scored
293T cells in the 50:50 mix (paper count) (reference)Printed in the paper. Independent check: yes	`1565`	exact	`1615` Reference value. Not scored	`1673` Reference value. Not scored	`1615` Reference value. Not scored	no numberReference value. Not scored

Open the runs

Tutorial or software test data

Li 2009

Li H, Handsaker B, Wysoker A, et al.

The Sequence Alignment/Map format and SAMtools. Bioinformatics 25(16):2078-2079 (2009). doi:10.1093/bioinformatics/btp352

SAMtools, example alignment

samtools (command line), through the samtools adapter

Opus7 of 7 values match, in each of 3 runs

7 of 7

In the final answer: 3 of 3, in each of 3 runs

Sonnet7 of 7 values match, in each of 3 runs

7 of 7

In the final answer: 3 of 3, in each of 3 runs

Haiku7 of 7 values match, in each of 3 runs

7 of 7

In the final answer: 3 of 3, in each of 3 runs

qwen3:8b5 of 7 values match

5 of 7

In the final answer: 3 of 3

Show each value

Known values and run values for Li 2009, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Total readsWe calculated it with samtools 1.24 through pysam 0.24.1, flagstat. Independent check: yes	`3307`	exact	`3307` match in 3 of 3 runsNot asked in the question	`3307` match in 3 of 3 runsNot asked in the question	`3307` match in 3 of 3 runsNot asked in the question	`3307` matchNot asked in the question
Mapped readsWe calculated it with samtools 1.24 through pysam 0.24.1, flagstat. Independent check: yes	`3271`	exact	`3271` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3271` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3271` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3271` matchIn the final answer: yes
Properly paired readsWe calculated it with samtools 1.24 through pysam 0.24.1, flagstat. Independent check: yes	`3144`	exact	`3144` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3144` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3144` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3144` matchIn the final answer: yes
SingletonsWe calculated it with samtools 1.24 through pysam 0.24.1, flagstat. Independent check: yes	`127`	exact	`127` match in 3 of 3 runsNot asked in the question	`127` match in 3 of 3 runsNot asked in the question	`127` match in 3 of 3 runsNot asked in the question	`127` matchNot asked in the question
Reads mapped to seq1We calculated it with samtools 1.24 through pysam 0.24.1, idxstats. Independent check: yes	`1482`	exact	`1482` match in 3 of 3 runsNot asked in the question	`1482` match in 3 of 3 runsNot asked in the question	`1482` match in 3 of 3 runsNot asked in the question	`1653` no matchNot asked in the question
Reads mapped to seq2We calculated it with samtools 1.24 through pysam 0.24.1, idxstats. Independent check: yes	`1789`	exact	`1789` match in 3 of 3 runsNot asked in the question	`1789` match in 3 of 3 runsNot asked in the question	`1789` match in 3 of 3 runsNot asked in the question	`1654` no matchNot asked in the question
Reads in seq2:100-200We calculated it with samtools 1.24 through pysam 0.24.1, view -c. Independent check: yes	`79`	exact	`79` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`79` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`79` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`79` matchIn the final answer: yes

Open the runs

Research paper

Li 2014

Li W, Xu H, Xiao T, et al.

MAGeCK enables robust identification of essential genes from genome-scale CRISPR/Cas9 knockout screens. Genome Biology 15:554 (2014). doi:10.1186/s13059-014-0554-4

MAGeCK, vemurafenib screen in A375 melanoma cells

MAGeCK (Python and C++), through the mageck adapter

No run yetno result file has this paper

No result file has this paper. It is not in the totals.

Show each value

Known values and run values for Li 2014, earlier run of 9 October 2026
Value	Known value	Tolerance	Haiku
Rank of CDH13 among the positively selected genes, vemurafenib against DMSO, day 14Printed in the paper	`9`	exact	not in this run
Rank of PPT1 among the positively selected genes, day 14Printed in the paper	`14`	exact	not in this run
Rank of TADA1 among the positively selected genes, day 14 (the paper gives rank 11 as the largest rank of NF1, NF2, MED12, CUL3, TADA1 and TADA2B)Printed in the paper	`11`	exact	not in this run
Rank of RREB1 among the negatively selected genes, day 14Printed in the paper	`1`	exact	not in this run
Rank of EGFR among the negatively selected genes, day 7Printed in the paper	`6`	exact	not in this run
False discovery rate of EGFR, negative selection, day 7Printed in the paper	`0.025`	± 0.005	not in this run
False discovery rate of CDH13, positive selection, day 14 (paper 0.017; MAGeCK 0.5.9.5 gives 0.028) (reference)Printed in the paper	`0.017`	± 0.015	not in this run
Genes ranked (paper 17,419; the cleaned supplement has 17,396 gene symbols) (reference)Printed in the paper	`17419`	± 40	not in this run

Open the runs

Research paper

Liao 2025

Liao X, Koehnken Sawall J, Seeley R, et al.

An adipo-osteoprogenitor population in the endosteal niche contributes to bone and fat formation in adult mouse bone marrow. PNAS 122(47):e2502436122 (2025). doi:10.1073/pnas.2502436122

adipo-osteoprogenitors in mouse bone marrow, single-cell quality control

scanpy (Python), through the scanpy adapter

Opus5 of 5 values match, in each of 3 runs

5 of 5

In the final answer: 4 of 4, in each of 3 runs

Sonnet5 of 5 values match, in each of 3 runs

5 of 5

In the final answer: 4 of 4, in each of 3 runs

Haiku4 to 5 of 5 values match, over 3 runs

4 to 5 of 5

In the final answer: 3 to 4 of 4, over 3 runs

qwen3:8b3 of 5 values match

3 of 5

In the final answer: 2 of 4

Show each value

Known values and run values for Liao 2025, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Cells in the pulse matrixPrinted in the paper	`10608`	exact	`10608` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10608` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10608` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10608` matchIn the final answer: yes
Cells in the chase matrixPrinted in the paper	`10046`	exact	`10046` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10046` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10046` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10046` matchIn the final answer: yes
Pulse cells after quality controlPrinted in the paper	`9612`	exact	`9612` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`9612` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`9612` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10046` no matchIn the final answer: no
Chase cells after quality controlPrinted in the paper	`8887`	exact	`8887` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`8887` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`8330` no match in 1 of 3 runsIn the final answer: yes in 2 of 3 runs	`8330` no matchIn the final answer: no
Genes in each matrixWe calculated it with Python gzip read of features.tsv.gz (check_qc.py). Independent check: yes	`32286`	exact	`32286` match in 3 of 3 runsNot asked in the question	`32286` match in 3 of 3 runsNot asked in the question	`32286` match in 3 of 3 runsNot asked in the question	`32286` matchNot asked in the question

Open the runs

Tutorial or software test data

Love 2014

Love MI, Huber W, Anders S.

Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2. Genome Biology 15:550 (2014). doi:10.1186/s13059-014-0550-8

Also: Love MI, Anders S, Kim V, Huber W. RNA-Seq workflow: gene-level exploratory analysis and differential expression. F1000Research 4:1070 (2016) doi:10.12688/f1000research.7035.2

DESeq2, airway smooth muscle cells with dexamethasone

DESeq2 (R), through the deseq2 adapter

Opus5 of 5 values match, in each of 3 runs

5 of 5

In the final answer: 4 of 4, in each of 3 runs

Sonnet5 of 5 values match, in each of 3 runs

5 of 5

In the final answer: 4 of 4, in each of 3 runs

Haiku5 of 5 values match, in each of 3 runs

5 of 5

In the final answer: 4 of 4, in each of 3 runs

qwen3:8b5 of 5 values match

5 of 5

In the final answer: 4 of 4

Show each value

Known values and run values for Love 2014, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Genes after pre-filter, rnaseqGene workflow on data(gse)Printed in the official tutorial	`16637`	± 10	`16637` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`16637` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`16637` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`16637` matchIn the final answer: yes
Genes with padj < 0.1, rnaseqGene workflow on data(gse)Printed in the official tutorial	`4381`	± 10	`4381` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`4381` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`4381` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`4381` matchIn the final answer: yes
Up-regulated genes at padj < 0.1, rnaseqGene workflow on data(gse)Printed in the official tutorial	`2362`	± 10	`2362` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2362` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2362` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2362` matchIn the final answer: yes
Down-regulated genes at padj < 0.1, rnaseqGene workflow on data(gse)Printed in the official tutorial	`2019`	± 3	`2019` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2019` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2019` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2019` matchIn the final answer: yes
Genes with padj < 0.05 (alpha 0.05), rnaseqGene workflow on data(gse)Printed in the official tutorial	`3602`	± 10	`3602` match in 3 of 3 runsNot asked in the question	`3602` match in 3 of 3 runsNot asked in the question	`3602` match in 3 of 3 runsNot asked in the question	`3602` matchNot asked in the question
Same experiment, read counts on Ensembl 75 genes: genes after pre-filter (reference)We calculated it with the rnaseqGene 1.36.0 workflow code, DESeq2 1.52.0 and R 4.6.1 (check_workflow.R). Check with the same program	`16139`	± 10	`16637` Reference value. Not scored	`16637` Reference value. Not scored	`16637` Reference value. Not scored	`16637` Reference value. Not scored
Same experiment, read counts on Ensembl 75 genes: genes with padj < 0.1 (reference)We calculated it with the rnaseqGene 1.36.0 workflow code, DESeq2 1.52.0 and R 4.6.1 (check_workflow.R). Check with the same program	`4905`	± 10	`4381` Reference value. Not scored	`4381` Reference value. Not scored	`4381` Reference value. Not scored	`4381` Reference value. Not scored
Same experiment, read counts on Ensembl 75 genes: up at padj < 0.1 (reference)We calculated it with the rnaseqGene 1.36.0 workflow code, DESeq2 1.52.0 and R 4.6.1 (check_workflow.R). Check with the same program	`2607`	± 10	`2648.605` Reference value. Not scored	`2648.605` Reference value. Not scored	`2648.605` Reference value. Not scored	`2648.605` Reference value. Not scored
Same experiment, read counts on Ensembl 75 genes: down at padj < 0.1 (reference)We calculated it with the rnaseqGene 1.36.0 workflow code, DESeq2 1.52.0 and R 4.6.1 (check_workflow.R). Check with the same program	`2298`	± 10	`2362` Reference value. Not scored	`2362` Reference value. Not scored	`2362` Reference value. Not scored	`2362` Reference value. Not scored
Same experiment, read counts on Ensembl 75 genes: genes with padj < 0.05 (alpha 0.05) (reference)We calculated it with the rnaseqGene 1.36.0 workflow code, DESeq2 1.52.0 and R 4.6.1 (check_workflow.R). Check with the same program	`4081`	± 10	`4381` Reference value. Not scored	`4381` Reference value. Not scored	`4381` Reference value. Not scored	`4381` Reference value. Not scored

Open the runs

Tutorial or software test data

Quinlan 2010

Quinlan AR, Hall IM.

BEDTools: a flexible suite of utilities for comparing genomic features. Bioinformatics 26(6):841-842 (2010). doi:10.1093/bioinformatics/btq033

BEDTools

bedtools (command line), through the bedtools adapter

Opus4 of 4 values match, in each of 3 runs

4 of 4

In the final answer: 3 of 3, in each of 3 runs

Sonnet4 of 4 values match, in each of 3 runs

4 of 4

In the final answer: 3 of 3, in each of 3 runs

Haiku4 of 4 values match, in each of 3 runs

4 of 4

In the final answer: 3 of 3, in each of 3 runs

qwen3:8b4 of 4 values match

4 of 4

In the final answer: 3 of 3

Show each value

Known values and run values for Quinlan 2010, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Exons that overlap a repeatWe calculated it with bedtools 2.31.1 intersect -u. Independent check: yes	`126`	exact	`126` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`126` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`126` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`126` matchIn the final answer: yes
Overlapping exon and repeat pairsWe calculated it with bedtools 2.31.1 intersect -wo. Independent check: yes	`129`	exact	`129` match in 3 of 3 runsNot asked in the question	`129` match in 3 of 3 runsNot asked in the question	`129` match in 3 of 3 runsNot asked in the question	`129` matchNot asked in the question
Overlap bases, any overlapWe calculated it with bedtools 2.31.1 intersect -wo, sum of the last column. Independent check: yes	`33713`	exact	`33713` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`33713` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`33713` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`33713` matchIn the final answer: yes
AluY intervals after merge, distance 100We calculated it with bedtools 2.31.1 merge -d 100 on sorted input. Independent check: yes	`11270`	exact	`11270` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`11270` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`11270` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`11270` matchIn the final answer: yes

Open the runs

Tutorial or software test data

Ross-Innes 2012

Ross-Innes CS, Stark R, Teschendorff AE, et al.

Differential oestrogen receptor binding is associated with clinical outcome in breast cancer. Nature 481:389-393 (2012). doi:10.1038/nature10730

Also: Love MI, Huber W, Anders S. Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2. Genome Biology 15:550 (2014). doi:10.1186/s13059-014-0550-8

DiffBind, estrogen receptor binding in tamoxifen-resistant breast cancer cells

DiffBind, DESeq2 and edgeR (R), through the diffbind adapter

No run yetno result file has this paper

No result file has this paper. It is not in the totals.

Show each value

Known values and run values for Ross-Innes 2012, earlier run of 9 October 2026
Value	Known value	Tolerance	Haiku
Consensus sites (chromosome 18)Printed in the official tutorial	`2845`	exact	not in this run
Differential peaks, resistant against responsive, tissue as blocking factor, DESeq2, FDR 0.05Printed in the official tutorial	`783`	exact	not in this run
Peaks that gain signal in the resistant samples, tissue as blocking factorWe calculated it with DiffBind 3.22.2 (dba.report) and DESeq2 1.52.0 called directly (checks/check_direct.R of the adapter). Check with the same program	`188`	exact	not in this run
Peaks that lose signal in the resistant samples, tissue as blocking factorWe calculated it with DiffBind 3.22.2 (dba.report) and DESeq2 1.52.0 called directly (checks/check_direct.R of the adapter). Check with the same program	`595`	exact	not in this run
Differential peaks without a blocking factor (the design ~Condition of the vignette)Printed in the official tutorial	`246`	exact	not in this run

Open the runs

Research paper

Wang 2023

Wang Y, Zhang Y, Jiang M, et al.

Bioinformatics Prediction and Experimental Validation Identify a Novel Cuproptosis-Related Gene Signature in Human Synovial Inflammation during Osteoarthritis Progression. Biomolecules 13(1):127 (2023). doi:10.3390/biom13010127

small molecules for four cuproptosis hub genes in osteoarthritis

gseapy (Python), through the enrichment adapter

Opus14 of 14 values match, in each of 3 runs

14 of 14

In the final answer: 13 of 13, in each of 3 runs

Sonnet14 of 14 values match, in each of 3 runs

14 of 14

In the final answer: 13 of 13, in each of 3 runs

Haiku14 of 14 values match, in each of 3 runs

14 of 14

In the final answer: 13 of 13, in each of 3 runs

qwen3:8b14 of 14 values match

14 of 14

In the final answer: 13 of 13

Show each value

Known values and run values for Wang 2023, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Genes of the list in the top termPrinted in the paper	`3`	exact	`3` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3` matchIn the final answer: yes
Size of the top termPrinted in the paper	`1578`	exact	`1578` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1578` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1578` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1578` matchIn the final answer: yes
Raw p of the top termPrinted in the paper	`0.001845348`	± 1e-06	`0.001845384` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.001845384` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.001845384` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.001845384` matchIn the final answer: yes
Adjusted p of the top termPrinted in the paper	`0.1118661`	± 1e-05	`0.1118672` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1118672` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1118672` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1118672` matchIn the final answer: yes
Odds ratio of the top termPrinted in the paper	`35.08762`	± 0.001	`35.08762` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`35.08762` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`35.08762` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`35.08762` matchIn the final answer: yes
Size of the second termPrinted in the paper	`24`	exact	`24` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`24` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`24` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`24` matchIn the final answer: yes
Raw p of the second termPrinted in the paper	`0.00479166`	± 1e-06	`0.004791726` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.004791726` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.004791726` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.004791726` matchIn the final answer: yes
Size of the third termPrinted in the paper	`30`	exact	`30` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`30` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`30` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`30` matchIn the final answer: yes
Raw p of the third termPrinted in the paper	`0.005986886`	± 1e-06	`0.005986962` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.005986962` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.005986962` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.005986962` matchIn the final answer: yes
Size of the fourth termPrinted in the paper	`649`	exact	`649` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`649` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`649` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`649` matchIn the final answer: yes
Raw p of the fourth termPrinted in the paper	`0.006039675`	± 1e-06	`0.006039754` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.006039754` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.006039754` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.006039754` matchIn the final answer: yes
Size of the fifth termPrinted in the paper	`37`	exact	`37` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`37` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`37` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`37` matchIn the final answer: yes
Raw p of the fifth termPrinted in the paper	`0.007379957`	± 1e-06	`0.007380042` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.007380042` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.007380042` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.007380042` matchIn the final answer: yes
Number of gene sets testedWe calculated it with gseapy 1.3.1. Independent check: yes	`4026`	exact	`4026` match in 3 of 3 runsNot asked in the question	`4026` match in 3 of 3 runsNot asked in the question	`4026` match in 3 of 3 runsNot asked in the question	`4026` matchNot asked in the question

Open the runs

Research paper

Weber 2023

Weber LM, Saha A, Datta A, et al.

nnSVG for the scalable identification of spatially variable genes using nearest-neighbor Gaussian processes. Nature Communications 14:4059 (2023). doi:10.1038/s41467-023-39748-z

nnSVG, spatially variable genes in the human prefrontal cortex

Squidpy (Python), through the squidpy adapter

Opus4 of 4 values match, in each of 3 runs

4 of 4

In the final answer: 4 of 4, in each of 3 runs

Sonnet4 of 4 values match, in each of 3 runs

4 of 4

In the final answer: 4 of 4, in each of 3 runs

Haiku4 of 4 values match, in each of 3 runs

4 of 4

In the final answer: 3 to 4 of 4, over 3 runs

qwen3:8b3 of 4 values match

3 of 4

In the final answer: 3 of 4

Show each value

Known values and run values for Weber 2023, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Spots in the tissuePrinted in the paper. Independent check: yes	`3639`	exact	`3639` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3639` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3639` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3639` matchIn the final answer: yes
Number of the two layer genes MOBP and SNAP25 within the top 100 genes by Moran's IPrinted in the paper. Independent check: yes	`2`	exact	`2` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2` match in 3 of 3 runsIn the final answer: yes in 2 of 3 runs	`2` matchIn the final answer: yes
Moran's I rank of MOBPWe calculated it with squidpy 1.8.3. Independent check: yes	`4`	± 2	`4` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`4` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`4` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2` matchIn the final answer: yes
Moran's I rank of SNAP25We calculated it with squidpy 1.8.3. Independent check: yes	`19`	± 3	`19` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`19` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`19` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2` no matchIn the final answer: no
Genes that pass the filter (paper count) (reference)Printed in the paper. Independent check: yes	`3396`	exact	`3309` Reference value. Not scored	`3309` Reference value. Not scored	`3309` Reference value. Not scored	`3309` Reference value. Not scored

Open the runs

Tutorial or software test data

Wolf 2018

Wolf FA, Angerer P, Theis FJ.

SCANPY: large-scale single-cell gene expression data analysis. Genome Biology 19:15 (2018). doi:10.1186/s13059-017-1382-0

scanpy, PBMC 3k clustering tutorial

scanpy (Python), through the scanpy adapter

Opus6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 2 of 2, in each of 3 runs

Sonnet6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 2 of 2, in each of 3 runs

Haiku6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 2 of 2, in each of 3 runs

qwen3:8b6 of 6 values match

6 of 6

In the final answer: 2 of 2

Show each value

Known values and run values for Wolf 2018, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Cells at load.Printed in the official tutorial	`2700`	exact	`2700` match in 3 of 3 runsNot asked in the question	`2700` match in 3 of 3 runsNot asked in the question	`2700` match in 3 of 3 runsNot asked in the question	`2700` matchNot asked in the question
Genes at load.Printed in the official tutorial	`32738`	exact	`32738` match in 3 of 3 runsNot asked in the question	`32738` match in 3 of 3 runsNot asked in the question	`32738` match in 3 of 3 runsNot asked in the question	`32738` matchNot asked in the question
Genes after the minimum cells filter.Printed in the official tutorial	`13714`	exact	`13714` match in 3 of 3 runsNot asked in the question	`13714` match in 3 of 3 runsNot asked in the question	`13714` match in 3 of 3 runsNot asked in the question	`13714` matchNot asked in the question
Cells after quality control.Printed in the official tutorial	`2638`	exact	`2638` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2638` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2638` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2638` matchIn the final answer: yes
Highly variable genes, default flavor.Printed in the official tutorial	`1838`	± 20	`1838` match in 3 of 3 runsNot asked in the question	`1838` match in 3 of 3 runsNot asked in the question	`1838` match in 3 of 3 runsNot asked in the question	`1838` matchNot asked in the question
Leiden clusters at resolution 0.7.We calculated it with scanpy 1.12.4. Independent check: yes	`7`	± 1	`7` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7` matchIn the final answer: yes

Open the runs

Tutorial or software test data

Yavorska 2017

Yavorska OO, Burgess S.

MendelianRandomization: an R package for performing Mendelian randomization analyses using summarized data. International Journal of Epidemiology 46(6):1734-1739 (2017). doi:10.1093/ije/dyx034

Also: Patel A, Ye T, Xue H, Lin Z, Xu S, Woolf B, Mason AM, Burgess S. MendelianRandomization v0.9.0: updates to an R package for performing Mendelian randomization analyses using summarized data. Wellcome Open Res 8:449 (2023). doi:10.12688/wellcomeopenres.19995.1

MendelianRandomization, LDL cholesterol and coronary heart disease

MendelianRandomization (R), through the mendelianrandomization adapter

No run yetno result file has this paper

No result file has this paper. It is not in the totals.

Show each value

Known values and run values for Yavorska 2017, earlier run of 9 October 2026
Value	Known value	Tolerance	Haiku
Number of variantsPrinted in the official tutorial	`28`	exact	not in this run
Inverse-variance weighted estimate (log odds ratio per unit of LDL cholesterol)Printed in the official tutorial	`2.834`	± 0.01	not in this run
Standard error of the inverse-variance weighted estimate (random effects)Printed in the official tutorial	`0.53`	± 0.01	not in this run
Lower limit of the 95% confidence interval of the inverse-variance weighted estimatePrinted in the official tutorial	`1.796`	± 0.02	not in this run
Upper limit of the 95% confidence interval of the inverse-variance weighted estimatePrinted in the official tutorial	`3.873`	± 0.02	not in this run
Cochran's Q (27 degrees of freedom)Printed in the official tutorial	`99.5304`	± 0.5	not in this run
I-squared of the variant estimates, percentPrinted in the official tutorial	`72.9`	± 0.5	not in this run
F statistic of the instruments (equals the number of variants to the printed digit; the scorer can match the variant count) (reference)Printed in the official tutorial	`28`	± 0.5	not in this run
Standard error of the MR-Egger estimatePrinted in the official tutorial	`0.77`	± 0.01	not in this run
MR-Egger estimatePrinted in the official tutorial	`3.253`	± 0.01	not in this run
MR-Egger interceptPrinted in the official tutorial	`-0.011`	± 0.002	not in this run
P value of the MR-Egger interceptPrinted in the official tutorial	`0.451`	± 0.01	not in this run
Weighted median estimatePrinted in the official tutorial	`2.683`	± 0.01	not in this run

Open the runs

Phylogenetics

Research paper

Kanetis 2022

Kanetis LI, Taliadoros D, Makris G, et al.

A Novel Seimatosporium and Other Sporocadaceae Species Associated with Grapevine Trunk Diseases in Cyprus. Plants 11(20):2733 (2022). doi:10.3390/plants11202733

Sporocadaceae from grapevine trunk diseases in Cyprus

IQ-TREE 3.1.4 (command line), through the phylo adapter

Opus3 of 3 values match, in each of 3 runs

3 of 3

In the final answer: 2 of 2, in each of 3 runs

Sonnet3 of 3 values match, in each of 3 runs

3 of 3

In the final answer: 2 of 2, in each of 3 runs

Haiku3 of 3 values match, in each of 3 runs

3 of 3

In the final answer: 2 of 2, in each of 3 runs

qwen3:8b2 of 3 values match

2 of 3

In the final answer: 0 of 2

Show each value

Known values and run values for Kanetis 2022, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Log-likelihood of the ML tree, concatenated alignment, TIM2e+I+G4Printed in the paper	`-15895.47`	± 0.1	`-15895.48` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-15895.48` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-15895.48` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-16151.58` no matchIn the final answer: no
Sites in the concatenated alignmentPrinted in the paper	`2375`	exact	`2375` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2375` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2375` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2375` matchIn the final answer: no
Sequences in the concatenated alignment (54 Sporocadaceae and the outgroup)We calculated it with IQ-TREE 2.0.8 input summary (check_iqtree.sh). Independent check: yes	`55`	exact	`55` match in 3 of 3 runsNot asked in the question	`55` match in 3 of 3 runsNot asked in the question	`55` match in 3 of 3 runsNot asked in the question	`55` matchNot asked in the question
Log-likelihood with the model that ModelFinder of IQ-TREE 3 picks (TIM2+F+R3) (reference)We calculated it with IQ-TREE 3.1.4 with -m MFP (check_iqtree.sh). Check with the same program	`-15891.37`	± 0.1	`-15895.48` Reference value. Not scored	`-15895.48` Reference value. Not scored	`-15895.48` Reference value. Not scored	`-16151.58` Reference value. Not scored

Open the runs

Tutorial or software test data

Minh 2020

Minh BQ, Schmidt HA, Chernomor O, et al.

IQ-TREE 2: new models and efficient methods for phylogenetic inference in the genomic era. Molecular Biology and Evolution 37(5):1530-1534 (2020). doi:10.1093/molbev/msaa015

IQ-TREE 2, example.phy tutorial data

IQ-TREE 3.1.4 and MAFFT (command line), through the phylo adapter

Opus1 of 1 values match, in each of 3 runs

1 of 1

In the final answer: 1 of 1, in each of 3 runs

Sonnet1 of 1 values match, in each of 3 runs

1 of 1

In the final answer: 1 of 1, in each of 3 runs

Haiku1 of 1 values match, in each of 3 runs

1 of 1

In the final answer: 1 of 1, in each of 3 runs

qwen3:8b1 of 1 values match

1 of 1

In the final answer: 1 of 1

Show each value

Known values and run values for Minh 2020, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Log-likelihood of the ML treeWe calculated it with IQ-TREE 3.1.4 (iqtree3), seed 1. Independent check: yes	`-21152.54`	± 1	`-21152.54` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-21152.54` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-21152.54` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-21152.54` matchIn the final answer: yes

Open the runs

Neuroscience

Research paper

Gouwens 2019

Gouwens NW, Sorensen SA, Berg J, et al.

Classification of electrophysiological and morphological neuron types in the mouse visual cortex. Nature Neuroscience 22(7):1182-1195 (2019). doi:10.1038/s41593-019-0417-0

Allen Cell Types electrophysiology features of three cells

pyABF, h5py and SciPy (Python), through the patch-clamp adapter

No run yetno result file has this paper

No result file has this paper. It is not in the totals.

Show each value

Known values and run values for Gouwens 2019, earlier run of 9 October 2026
Value	Known value	Tolerance	Haiku
Cell 567987585: rheobase, pAPrinted in the paper	`230`	exact	not in this run
Cell 567987585: F-I slope, Hz per pAPrinted in the paper	`1.0717`	± 0.005	not in this run
Cell 567987585: input resistance, MOhmPrinted in the paper	`129.84`	± 0.5	not in this run
Cell 567987585: membrane time constant, msPrinted in the paper	`8.26`	± 0.3	not in this run
Cell 567987585: upstroke to downstroke ratioPrinted in the paper	`1.1458`	± 0.003	not in this run
Cell 595511209: rheobase, pAPrinted in the paper	`70`	exact	not in this run
Cell 595511209: F-I slope, Hz per pAPrinted in the paper	`0.1152`	± 0.001	not in this run
Cell 595511209: input resistance, MOhmPrinted in the paper	`187.19`	± 0.5	not in this run
Cell 595511209: membrane time constant, msPrinted in the paper	`32.56`	± 0.6	not in this run
Cell 595511209: upstroke to downstroke ratioPrinted in the paper	`3.8746`	± 0.004	not in this run
Cell 563220277: rheobase, pAPrinted in the paper	`90`	exact	not in this run
Cell 563220277: F-I slope, Hz per pAPrinted in the paper	`0.2075`	± 0.001	not in this run
Cell 563220277: input resistance, MOhmPrinted in the paper	`128.52`	± 0.5	not in this run
Cell 563220277: upstroke to downstroke ratioPrinted in the paper	`3.2259`	± 0.004	not in this run
Cell 563220277: membrane time constant, ms (the tool gives no fit for one of the three sweeps) (reference)Printed in the paper	`14.87`	± 0.5	not in this run

Open the runs

Tutorial or software test data

Gramfort 2013

Gramfort A, Luessi M, Larson E, et al.

MEG and EEG data analysis with MNE-Python. Frontiers in Neuroscience 7:267 (2013). doi:10.3389/fnins.2013.00267

MNE software, sample dataset (auditory and visual stimuli)

MNE-Python, through the mne adapter

Opus14 of 14 values match, in each of 3 runs

14 of 14

In the final answer: 12 of 12, in each of 3 runs

Sonnet14 of 14 values match, in each of 3 runs

14 of 14

In the final answer: 12 of 12, in each of 3 runs

Haiku14 of 14 values match, in each of 3 runs

14 of 14

In the final answer: 12 of 12, in each of 3 runs

qwen3:8b14 of 14 values match

14 of 14

In the final answer: 1 of 12

Show each value

Known values and run values for Gramfort 2013, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Events found on STI 014.Printed in the official tutorial. Independent check: yes	`319`	exact	`319` match in 3 of 3 runsNot asked in the question	`319` match in 3 of 3 runsNot asked in the question	`319` match in 3 of 3 runsNot asked in the question	`319` matchNot asked in the question
Events with id 1 (auditory/left).We calculated it with MNE-Python 1.13.2. Independent check: yes	`72`	exact	`72` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`72` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`72` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`72` matchIn the final answer: no
Events with id 2 (auditory/right).We calculated it with MNE-Python 1.13.2. Independent check: yes	`73`	exact	`73` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`73` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`73` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`73` matchIn the final answer: no
Events with id 3 (visual/left).We calculated it with MNE-Python 1.13.2. Independent check: yes	`73`	exact	`73` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`73` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`73` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`73` matchIn the final answer: no
Events with id 4 (visual/right).We calculated it with MNE-Python 1.13.2. Independent check: yes	`70`	exact	`70` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`70` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`70` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`70` matchIn the final answer: no
Events with id 5 (smiley).We calculated it with MNE-Python 1.13.2. Independent check: yes	`15`	exact	`15` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`15` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`15` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`15` matchIn the final answer: no
Events with id 32 (buttonpress).We calculated it with MNE-Python 1.13.2. Independent check: yes	`16`	exact	`16` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`16` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`16` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`16` matchIn the final answer: no
Epochs dropped by the limits, all six ids.Printed in the official tutorial. Independent check: yes	`10`	exact	`10` match in 3 of 3 runsNot asked in the question	`10` match in 3 of 3 runsNot asked in the question	`10` match in 3 of 3 runsNot asked in the question	`10` matchNot asked in the question
Epochs kept, all six ids.We calculated it with MNE-Python 1.13.2. Independent check: yes	`309`	exact	`309` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`309` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`309` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`309` matchIn the final answer: no
Auditory/left epochs kept.We calculated it with MNE-Python 1.13.2. Independent check: yes	`68`	exact	`68` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`68` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`68` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`68` matchIn the final answer: no
N100 latency (s), global field power of good gradiometers, auditory/left.We calculated it with MNE-Python 1.13.2. Independent check: yes	`0.0932`	± 0.007	`0.09323776` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.09323776` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.09323776` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.09323776` matchIn the final answer: yes
N100 global field power amplitude in gradiometers (fT/cm).We calculated it with MNE-Python 1.13.2. Independent check: yes	`42.1`	± 2	`42.07276` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`42.07276` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`42.07276` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`42.10613` matchIn the final answer: no
Latency (s) of the strongest gradiometer channel.We calculated it with MNE-Python 1.13.2. Independent check: yes	`0.0866`	± 0.007	`0.08657792` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.08657792` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.08657792` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.08` matchIn the final answer: no
Amplitude (fT/cm) of the strongest gradiometer channel, absolute value.We calculated it with MNE-Python 1.13.2. Independent check: yes	`199`	± 8	`198.7991` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`198.7991` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`198.7991` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`203` matchIn the final answer: no

Open the runs

Research paper

Jahncke 2025

Jahncke JN, Schnell E, Wright KM.

Distinct functional domains of Dystroglycan regulate inhibitory synapse formation and maintenance in cerebellar Purkinje cells. Communications Biology 8(1):878 (2025). doi:10.1038/s42003-025-08323-1

Data: doi:10.6084/m9.figshare.29083331

mIPSCs of Purkinje cells in Dag1 conditional knockout mice

pyABF, Neo and SciPy (Python), through the patch-clamp adapter

No run yetno result file has this paper

No result file has this paper. It is not in the totals.

Show each value

Known values and run values for Jahncke 2025, earlier run of 9 October 2026
Value	Known value	Tolerance	Haiku
Control cellsPrinted in the paper	`16`	exact	not in this run
CKO cellsPrinted in the paper	`16`	exact	not in this run
Animals in each groupPrinted in the paper	`6`	exact	not in this run
P value of the mIPSC frequency, cKO against Ctrl, cells as observations (the paper prints 0.018; the tolerance covers a different detection method)Printed in the paper	`0.018`	± 0.015	not in this run
P value of the mIPSC amplitude, cKO against Ctrl, cells as observations (the paper prints 0.017)Printed in the paper	`0.017`	± 0.015	not in this run
P value of the 10-90% rise time difference (the paper prints 0.38) (reference)Printed in the paper	`0.38`	± 0.2	not in this run
P value of the decay time constant difference (the paper prints 0.36) (reference)Printed in the paper	`0.36`	± 0.25	not in this run
Mean decay time constant of the Ctrl cells, ms (the paper prints 7.87)Printed in the paper	`7.87`	± 0.5	not in this run
Mean frequency of the Ctrl cells, Hz (not printed in the paper)We calculated it with the patch-clamp adapter. Check with the same program	`2.97`	± 0.6	not in this run
Mean frequency of the cKO cells, Hz (not printed in the paper)We calculated it with the patch-clamp adapter. Check with the same program	`1.76`	± 0.5	not in this run
Mean amplitude of the Ctrl cells, pA (not printed in the paper)We calculated it with the patch-clamp adapter. Check with the same program	`51.9`	± 7	not in this run
Mean amplitude of the cKO cells, pA (not printed in the paper)We calculated it with the patch-clamp adapter. Check with the same program	`38.3`	± 6	not in this run
P value of the amplitude with the animal as the unit (6 against 6 animals): not significant at 0.05 (reference)We calculated it with the patch-clamp adapter. Independent check: yes	`0.067`	± 0.03	not in this run

Open the runs

Structural biology

Tutorial or software test data

Gowers 2016

Gowers RJ, Linke M, Barnoud J, et al.

MDAnalysis: a Python package for the rapid analysis of molecular dynamics simulations. Proceedings of the 15th Python in Science Conference, pages 98-105 (2016). doi:10.25080/Majora-629e541a-00e

Also: Michaud-Agrawal N, Denning EJ, Woolf TB, Beckstein O. MDAnalysis: a toolkit for the analysis of molecular dynamics simulations. Journal of Computational Chemistry 32:2319-2327 (2011). doi:10.1002/jcc.21787

MDAnalysis, adenylate kinase (AdK) trajectory

MDAnalysis and Bio.PDB (Python), through the mdanalysis adapter

Opus7 of 7 values match, in each of 3 runs

7 of 7

In the final answer: 7 of 7, in each of 3 runs

Sonnet7 of 7 values match, in each of 3 runs

7 of 7

In the final answer: 7 of 7, in each of 3 runs

Haiku7 of 7 values match, in each of 3 runs

7 of 7

In the final answer: 7 of 7, in each of 3 runs

qwen3:8b7 of 7 values match

7 of 7

In the final answer: 7 of 7

Show each value

Known values and run values for Gowers 2016, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Atoms in the AdK systemPrinted in the official tutorial. Independent check: yes	`3341`	exact	`3341` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3341` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3341` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3341` matchIn the final answer: yes
Frames in the AdK trajectoryPrinted in the official tutorial. Independent check: yes	`98`	exact	`98` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`98` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`98` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`98` matchIn the final answer: yes
Backbone RMSD of the last frame against frame 0, in angstromPrinted in the official tutorial. Independent check: yes	`6.8203`	± 0.01	`6.820322` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`6.820322` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`6.820322` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`6.820322` matchIn the final answer: yes
Radius of gyration of the protein, last frame, in angstromPrinted in the official tutorial. Independent check: yes	`19.592`	± 0.02	`19.59158` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`19.59158` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`19.59158` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`19.59158` matchIn the final answer: yes
Residues in chain A of 4AKEWe calculated it with Bio.PDB (Biopython 1.88). Independent check: yes	`214`	exact	`214` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`214` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`214` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`214` matchIn the final answer: yes
Residues in chain B of 4AKEWe calculated it with Bio.PDB (Biopython 1.88). Independent check: yes	`214`	exact	`214` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`214` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`214` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`214` matchIn the final answer: yes
CA RMSD after fit of 4AKE chain A on 1AKE chain A, in angstromWe calculated it with Bio.PDB (Biopython 1.88), Superimposer. Independent check: yes	`7.131`	± 0.02	`7.1307` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7.1307` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7.1307` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7.1307` matchIn the final answer: yes

Open the runs

Chemistry

Research paper

Chure 2024

Chure G, Cremer J.

hplc-py: A Python Utility For Rapid Quantification of Complex Chemical Chromatograms. Journal of Open Source Software 9(94):6270 (2024). doi:10.21105/joss.06270

peak integration and a lactose calibration with hplc-py

hplc-py (Python), through the chromatography adapter

Opus12 of 12 values match, in each of 3 runs

12 of 12

In the final answer: 11 of 11, in each of 3 runs

Sonnet12 of 12 values match, in each of 3 runs

12 of 12

In the final answer: 11 of 11, in each of 3 runs

Haiku12 of 12 values match, in each of 3 runs

12 of 12

In the final answer: 11 of 11, in each of 3 runs

qwen3:8b6 of 12 values match

6 of 12

In the final answer: 6 of 11

Show each value

Known values and run values for Chure 2024, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Number of peaks between 10 and 20 minutesWe calculated it with hplc-py 0.2.10. Check with the same program	`6`	exact	`6` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`6` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`6` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`6` matchIn the final answer: yes
Retention time of the first peak, in minutesPrinted in the paper	`10.9`	± 0.005	`10.9` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10.9` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10.9` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10.9` matchIn the final answer: yes
Area of the first peakPrinted in the paper	`2.8e6`	± 0.05e6	`2805655` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2805655` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2805655` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2789693` matchIn the final answer: yes
Fitted amplitude of the first peakPrinted in the paper	`2.3e4`	± 0.05e4	`23380.46` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`22935` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`22935` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`23247.44` matchIn the final answer: yes
Fitted scale of the first peakPrinted in the paper	`0.16`	± 0.005	`0.1587619` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1587619` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1587619` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.15833` matchIn the final answer: yes
Fitted skew of the first peakPrinted in the paper	`0.7`	± 0.01	`0.6917996` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6858291` no matchIn the final answer: no
Retention time of the third peak, in minutesPrinted in the paper	`14.45`	± 0.005	`14.45` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`14.45` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`14.45` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`14.45` matchIn the final answer: yes
Area of the first peak, full precisionWe calculated it with hplc-py 0.2.10. Check with the same program	`2805655`	± 300	`2805655` match in 3 of 3 runsNot asked in the question	`2805655` match in 3 of 3 runsNot asked in the question	`2805655` match in 3 of 3 runsNot asked in the question	`2789693` no matchNot asked in the question
Calibration slope, area per mMWe calculated it with numpy least squares, in the adapter. Independent check: yes	`156586.7`	± 160	`156586.7` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`156586.7` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`156586.7` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`75508` no matchIn the final answer: no
Calibration intercept, in area unitsWe calculated it with numpy least squares, in the adapter. Independent check: yes	`14624.94`	± 150	`14624.94` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`14624.94` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`14624.94` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`14891.04` no matchIn the final answer: no
R squared of the calibrationWe calculated it with numpy least squares, in the adapter. Independent check: yes	`0.99887`	± 5e-05	`0.9988679` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9988679` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9988679` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1` no matchIn the final answer: no
Concentration of the unknown lactose sample, in mMWe calculated it with the chromatography adapter. Independent check: yes	`1.8994`	± 0.005	`1.899415` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.899415` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.899415` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.984174` no matchIn the final answer: no

Open the runs

Research paper

Delaney 2004

Delaney JS.

ESOL: estimating aqueous solubility directly from molecular structure. J Chem Inf Comput Sci 44(3):1000-1005 (2004). doi:10.1021/ci034243x

Also: Lipinski CA, Lombardo F, Dominy BW, Feeney PJ. Experimental and computational approaches to estimate solubility and permeability in drug discovery and development settings. Advanced Drug Delivery Reviews 23:3-25 (1997). doi:10.1016/S0169-409X(96)00423-1

ESOL aqueous solubility, rule of five and descriptors with RDKit

RDKit (Python), through the rdkit adapter

Opus6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 4 of 4, in each of 3 runs

Sonnet6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 4 of 4, in each of 3 runs

Haiku6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 4 of 4, in each of 3 runs

qwen3:8b4 of 6 values match

4 of 6

In the final answer: 2 of 4

Show each value

Known values and run values for Delaney 2004, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Molecules parsed.We calculated it with RDKit 2026.03.6. Independent check: yes	`1128`	exact	`1128` match in 3 of 3 runsNot asked in the question	`1128` match in 3 of 3 runsNot asked in the question	`1128` match in 3 of 3 runsNot asked in the question	`1128` matchNot asked in the question
Molecules with zero rule-of-five violations.We calculated it with RDKit 2026.03.6. Check with the same program	`1018`	± 5	`1018` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1018` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1018` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1018` matchIn the final answer: yes
Molecules with two or more violations.We calculated it with RDKit 2026.03.6. Independent check: yes	`12`	± 3	`12` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`12` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`12` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`12` matchIn the final answer: yes
Molecules with logP above 5.We calculated it with RDKit 2026.03.6, Crippen logP. Check with the same program	`97`	± 5	`97` match in 3 of 3 runsNot asked in the question	`97` match in 3 of 3 runsNot asked in the question	`97` match in 3 of 3 runsNot asked in the question	`97` matchNot asked in the question
R squared of the ESOL equation (Crippen logP) against measured values.We calculated it with RDKit 2026.03.6. Independent check: yes	`0.752`	± 0.03	`0.7519385` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7519385` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7519385` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	no matchIn the final answer: no
RMSE of the ESOL equation against measured values, in log units.We calculated it with RDKit 2026.03.6. Independent check: yes	`1.1`	± 0.1	`1.099372` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.099372` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.099372` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	no matchIn the final answer: no

Open the runs

Microbiology

Research paper

Kovacs 2026

Kovács ÁB et al.

Genome-wide association study of Mycoplasma anserisalpingitidis strains for antibiotic susceptibility. Scientific Reports (2026). doi:10.1038/s41598-026-39804-w

antibiotic susceptibility of Mycoplasma anserisalpingitidis

AMR (R), through the amr adapter

No run yetno result file has this paper

No result file has this paper. It is not in the totals.

Show each value

Known values and run values for Kovacs 2026, earlier run of 9 October 2026
Value	Known value	Tolerance	Haiku
MIC50 of tylvalosin in ug/mlPrinted in the paper	`0.5`	± 0.001	not in this run
MIC90 of tylvalosin in ug/mlPrinted in the paper	`2`	± 0.001	not in this run
MIC50 of oxytetracycline in ug/mlPrinted in the paper	`16`	± 0.001	not in this run
MIC90 of oxytetracycline in ug/mlPrinted in the paper	`64`	± 0.001	not in this run
MIC50 of doxycycline in ug/mlPrinted in the paper	`5`	± 0.001	not in this run
MIC90 of doxycycline in ug/mlPrinted in the paper	`10`	± 0.001	not in this run
MIC50 of tiamulin in ug/mlPrinted in the paper	`0.625`	± 0.001	not in this run
MIC90 of tiamulin in ug/mlPrinted in the paper	`2.5`	± 0.001	not in this run
MIC50 of tylosin in ug/mlPrinted in the paper	`16`	± 0.001	not in this run
MIC90 of tylosin in ug/ml (the paper prints 64 or more)Printed in the paper	`64`	± 0.001	not in this run
MIC50 of lincomycin in ug/mlPrinted in the paper	`2`	± 0.001	not in this run
MIC50 of spectinomycin in ug/mlPrinted in the paper	`8`	± 0.001	not in this run
MIC50 of tilmicosin in ug/ml (the paper prints 64 or more)Printed in the paper	`64`	± 0.001	not in this run
MIC90 of tilmicosin in ug/ml (the paper prints 64 or more)Printed in the paper	`64`	± 0.001	not in this run
MIC50 of enrofloxacin as printed in the paper (more than 10; does not reproduce, computed 5) (reference)Printed in the paper	`10`	± 0.001	not in this run

Open the runs

Research paper

Kresken 2021

Kresken M, Pfeifer Y, Werner G.

Comparative in vitro activity of piperacillin-tazobactam and temocillin against third-generation cephalosporin-resistant, carbapenem-susceptible Escherichia coli and Klebsiella pneumoniae. GMS Infectious Diseases 9:Doc08 (2021). doi:10.3205/id000077

Data: doi:10.5061/dryad.931zcrjkc

piperacillin/tazobactam and temocillin against cephalosporin-resistant Enterobacterales

AMR (R), through the amr adapter

No run yetno result file has this paper

No result file has this paper. It is not in the totals.

Show each value

Known values and run values for Kresken 2021, earlier run of 9 October 2026
Value	Known value	Tolerance	Haiku
Percent susceptible, Escherichia coli, 3GC-resistant, piperacillin/tazobactamPrinted in the paper	`79.3`	± 0.06	not in this run
MIC50 in mg/L, Escherichia coli, 3GC-resistant, piperacillin/tazobactamPrinted in the paper	`2`	± 0.001	not in this run
MIC90 in mg/L, Escherichia coli, 3GC-resistant, piperacillin/tazobactamPrinted in the paper	`32`	± 0.001	not in this run
Percent susceptible, Escherichia coli, 3GC-resistant, temocillinPrinted in the paper	`94.8`	± 0.06	not in this run
MIC50 in mg/L, Escherichia coli, 3GC-resistant, temocillinPrinted in the paper	`8`	± 0.001	not in this run
MIC90 in mg/L, Escherichia coli, 3GC-resistant, temocillinPrinted in the paper	`16`	± 0.001	not in this run
Percent susceptible, Escherichia coli, 3GC-susceptible, piperacillin/tazobactamPrinted in the paper	`93.3`	± 0.06	not in this run
MIC50 in mg/L, Escherichia coli, 3GC-susceptible, piperacillin/tazobactamPrinted in the paper	`1`	± 0.001	not in this run
MIC90 in mg/L, Escherichia coli, 3GC-susceptible, piperacillin/tazobactamPrinted in the paper	`4`	± 0.001	not in this run
Percent susceptible, Escherichia coli, 3GC-susceptible, temocillinPrinted in the paper	`100`	± 0.06	not in this run
MIC50 in mg/L, Escherichia coli, 3GC-susceptible, temocillinPrinted in the paper	`4`	± 0.001	not in this run
MIC90 in mg/L, Escherichia coli, 3GC-susceptible, temocillinPrinted in the paper	`16`	± 0.001	not in this run
Percent susceptible, Klebsiella pneumoniae, 3GC-resistant, piperacillin/tazobactamPrinted in the paper	`57.1`	± 0.06	not in this run
MIC50 in mg/L, Klebsiella pneumoniae, 3GC-resistant, piperacillin/tazobactamPrinted in the paper	`8`	± 0.001	not in this run
MIC90 in mg/L, Klebsiella pneumoniae, 3GC-resistant, piperacillin/tazobactamPrinted in the paper	`128`	± 0.001	not in this run
Percent susceptible, Klebsiella pneumoniae, 3GC-resistant, temocillinPrinted in the paper	`90.5`	± 0.06	not in this run
MIC50 in mg/L, Klebsiella pneumoniae, 3GC-resistant, temocillinPrinted in the paper	`8`	± 0.001	not in this run
MIC90 in mg/L, Klebsiella pneumoniae, 3GC-resistant, temocillinPrinted in the paper	`16`	± 0.001	not in this run
Percent susceptible, Klebsiella pneumoniae, 3GC-susceptible, piperacillin/tazobactamPrinted in the paper	`93.3`	± 0.06	not in this run
MIC50 in mg/L, Klebsiella pneumoniae, 3GC-susceptible, piperacillin/tazobactamPrinted in the paper	`1`	± 0.001	not in this run
MIC90 in mg/L, Klebsiella pneumoniae, 3GC-susceptible, piperacillin/tazobactamPrinted in the paper	`8`	± 0.001	not in this run
Percent susceptible, Klebsiella pneumoniae, 3GC-susceptible, temocillinPrinted in the paper	`100`	± 0.06	not in this run
MIC50 in mg/L, Klebsiella pneumoniae, 3GC-susceptible, temocillinPrinted in the paper	`2`	± 0.001	not in this run
MIC90 in mg/L, Klebsiella pneumoniae, 3GC-susceptible, temocillinPrinted in the paper	`8`	± 0.001	not in this run

Open the runs

Research paper

Makrai 2023

Makrai L, Fodróczy B, Nagy SÁ, et al.

Annotated dataset for deep-learning-based bacterial colony detection. Scientific Data 10:497 (2023). doi:10.1038/s41597-023-02404-8

annotated bacterial colony photos

scikit-image (Python), through the image-assays adapter. The paper counted by hand annotation (COCO Annotator and Make Sense).

Opus2 of 2 values match, in each of 3 runs

2 of 2

In the final answer: 1 of 1, in each of 3 runs

Sonnet2 of 2 values match, in each of 3 runs

2 of 2

In the final answer: 1 of 1, in each of 3 runs

Haiku2 of 2 values match, in each of 3 runs

2 of 2

In the final answer: 1 of 1, in each of 3 runs

qwen3:8b2 of 2 values match

2 of 2

In the final answer: 1 of 1

Show each value

Known values and run values for Makrai 2023, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Photos countedPrinted in the paper	`6`	exact	`6` match in 3 of 3 runsNot asked in the question	`6` match in 3 of 3 runsNot asked in the question	`6` match in 3 of 3 runsNot asked in the question	`6` matchNot asked in the question
Colonies on the six photos, printed counts 14, 14, 14, 14, 16, 16Printed in the paper. Independent check: yes	`88`	± 9	`88` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`88` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`88` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`88` matchIn the final answer: yes

Open the runs

Research paper

Nair 2024

Nair RR, Andersson DI, Warsi OM.

Antibiotic resistance begets more resistance: chromosomal resistance mutations mitigate fitness costs conferred by multi-resistant clinical plasmids. Microbiology Spectrum 12(5): e0420623 (2024). doi:10.1128/spectrum.04206-23

Data: doi:10.6084/m9.figshare.25400194

growth cost of clinical plasmids in resistant E. coli

growthcurver (R), through the growthcurver adapter

Opus20 of 20 values match, in each of 3 runs

20 of 20

In the final answer: 17 of 17, in each of 3 runs

Sonnet20 of 20 values match, in each of 3 runs

20 of 20

In the final answer: 17 of 17, in each of 3 runs

Haiku20 of 20 values match, in each of 3 runs

20 of 20

In the final answer: 17 of 17, in each of 3 runs

qwen3:8b4 of 20 values match

4 of 20

In the final answer: 0 of 17

1 run ended with a time limit

Show each value

Known values and run values for Nair 2024, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Wells fittedPrinted in the paper	`70`	exact	`70` match in 3 of 3 runsNot asked in the question	`70` match in 3 of 3 runsNot asked in the question	`70` match in 3 of 3 runsNot asked in the question	`70` matchNot asked in the question
Growth rate r of well W01 (host, block 1), per hourPrinted in the paper	`0.371065`	± 5e-06	`0.3710652` match in 3 of 3 runsNot asked in the question	`0.3710652` match in 3 of 3 runsNot asked in the question	`0.3710652` match in 3 of 3 runsNot asked in the question	`0.3710652` matchNot asked in the question
Growth rate r of well W24 (P5, block 3), per hourPrinted in the paper	`0.479446`	± 5e-06	`0.4794457` match in 3 of 3 runsNot asked in the question	`0.4794457` match in 3 of 3 runsNot asked in the question	`0.4794457` match in 3 of 3 runsNot asked in the question	`0.4794457` matchNot asked in the question
Mean relative growth rate, P1We calculated it with compare_growth of the growthcurver adapter. Independent check: yes	`1.005299`	± 0.0005	`1.005299` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.005299` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.005299` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5683573` no matchIn the final answer: no
Mean relative growth rate, P2We calculated it with compare_growth of the growthcurver adapter. Independent check: yes	`1.027756`	± 0.0005	`1.027756` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.027756` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.027756` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.445726` no matchIn the final answer: no
Mean relative growth rate, P3We calculated it with compare_growth of the growthcurver adapter. Independent check: yes	`1.032948`	± 0.0005	`1.032948` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.032948` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.032948` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.445726` no matchIn the final answer: no
Mean relative growth rate, P4We calculated it with compare_growth of the growthcurver adapter. Independent check: yes	`0.965689`	± 0.0005	`0.9656891` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9656891` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9656891` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5683573` no matchIn the final answer: no
Mean relative growth rate, P5We calculated it with compare_growth of the growthcurver adapter. Independent check: yes	`1.072459`	± 0.0005	`1.072459` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.072459` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.072459` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.445726` no matchIn the final answer: no
Mean relative growth rate, P6We calculated it with compare_growth of the growthcurver adapter. Independent check: yes	`1.008294`	± 0.0005	`1.008294` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.008294` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.008294` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.445726` no matchIn the final answer: no
Mean relative growth rate, P7We calculated it with compare_growth of the growthcurver adapter. Independent check: yes	`0.961198`	± 0.0005	`0.9611976` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9611976` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9611976` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5683573` no matchIn the final answer: no
Mean relative growth rate, P8We calculated it with compare_growth of the growthcurver adapter. Independent check: yes	`0.97545`	± 0.0005	`0.9754496` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9754496` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9754496` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5683573` no matchIn the final answer: no
Mean relative growth rate, P9We calculated it with compare_growth of the growthcurver adapter. Independent check: yes	`1.042573`	± 0.0005	`1.042573` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.042573` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.042573` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.445726` no matchIn the final answer: no
Mean relative growth rate, P10We calculated it with compare_growth of the growthcurver adapter. Independent check: yes	`0.990162`	± 0.0005	`0.9901624` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9901624` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9901624` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5683573` no matchIn the final answer: no
Mean relative growth rate, P11We calculated it with compare_growth of the growthcurver adapter. Independent check: yes	`0.946554`	± 0.0005	`0.9465535` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9465535` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9465535` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5683573` no matchIn the final answer: no
Mean relative growth rate, P14We calculated it with compare_growth of the growthcurver adapter. Independent check: yes	`1.082055`	± 0.0005	`1.082055` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.082055` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.082055` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.445726` no matchIn the final answer: no
95% CI lower bound of the relative growth rate, P11We calculated it with compare_growth of the growthcurver adapter. Independent check: yes	`0.742365`	± 0.0005	`0.7423649` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7423649` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7423649` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5683573` no matchIn the final answer: no
95% CI upper bound of the relative growth rate, P11We calculated it with compare_growth of the growthcurver adapter. Independent check: yes	`1.150742`	± 0.0005	`1.150742` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.150742` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.150742` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.445726` no matchIn the final answer: no
P value of the one-sample t test against 1, P5We calculated it with compare_growth of the growthcurver adapter. Independent check: yes	`0.243208`	± 0.0005	`0.2432082` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.2432082` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.2432082` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.2881358` no matchIn the final answer: no
P value of the one-sample t test against 1, P11We calculated it with compare_growth of the growthcurver adapter. Independent check: yes	`0.507637`	± 0.0005	`0.5076367` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5076367` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5076367` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5074041` matchIn the final answer: no
Mean of the 12 plasmid meansWe calculated it with compare_growth of the growthcurver adapter. Independent check: yes	`1.009203`	± 0.0005	`1.009203` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.009203` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.009203` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.445726` no matchIn the final answer: no
Average relative growth rate of STR-R as printed in the Results (does not reproduce from the source data; computed 1.0092) (reference)Printed in the paper	`0.996`	± 0.0005	`1` Reference value. Not scored	`1` Reference value. Not scored	`1` Reference value. Not scored	`0.5683573` Reference value. Not scored

Open the runs

Bench assays

Research paper

Aledo 2022

Aledo JC.

renz: An R package for the analysis of enzyme kinetic data. BMC Bioinformatics 23:182 (2022). doi:10.1186/s12859-022-04729-4

Michaelis-Menten fit of the ONPG data in renz

drc (R), through the drc adapter. The paper used the R package renz.

Opus2 of 2 values match, in each of 3 runs

2 of 2

In the final answer: 2 of 2, in each of 3 runs

Sonnet2 of 2 values match, in each of 3 runs

2 of 2

In the final answer: 2 of 2, in each of 3 runs

Haiku2 of 2 values match, in each of 3 runs

2 of 2

In the final answer: 2 of 2, in each of 3 runs

qwen3:8b2 of 2 values match

2 of 2

In the final answer: 2 of 2

Show each value

Known values and run values for Aledo 2022, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Km (mM), nonlinear fitPrinted in the paper	`2.4793`	± 0.005	`2.479264` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.479264` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.479264` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.479264` matchIn the final answer: yes
Vmax (mM/min), nonlinear fitPrinted in the paper	`0.18407`	± 0.0005	`0.1840735` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1840735` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1840735` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1840735` matchIn the final answer: yes

Open the runs

Research paper

Cooley 2020

Cooley R, Kara N, Hui NS, et al.

Development of a cell-free split-luciferase biochemical assay as a tool for screening for inhibitors of challenging protein-protein interaction targets. Wellcome Open Research 5:20 (2020). doi:10.12688/wellcomeopenres.15675.1

Data: doi:10.17605/OSF.IO/MGKQV

Z' factor of a split-luciferase assay on ten plates

drc adapter, normalize_plate (R arithmetic). The paper used Excel.

Opus19 of 19 values match, in each of 3 runs

19 of 19

In the final answer: 19 of 19, in each of 3 runs

Sonnet19 of 19 values match, in each of 3 runs

19 of 19

In the final answer: 19 of 19, in each of 3 runs

Haiku19 of 19 values match, in each of 3 runs

19 of 19

In the final answer: 19 of 19, in each of 3 runs

qwen3:8b9 of 19 values match

9 of 19

In the final answer: 0 of 19

1 run ended with no final answer

Show each value

Known values and run values for Cooley 2020, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Z' plate 1, 10 µlPrinted in the paper	`0.5413`	± 0.004	`0.5413071` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5413071` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5413071` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5218797` no matchIn the final answer: no
Z' plate 2, 10 µlPrinted in the paper	`0.6102`	± 0.004	`0.6102128` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6102128` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6102128` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6429603` no matchIn the final answer: no
Z' plate 3, 10 µlPrinted in the paper	`0.7275`	± 0.004	`0.7275382` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7275382` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7275382` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7282149` matchIn the final answer: no
Z' plate 4, 10 µlPrinted in the paper	`0.7302`	± 0.002	`0.7302386` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7302386` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7302386` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7282149` matchIn the final answer: no
Z' plate 5, 10 µlPrinted in the paper	`0.6643`	± 0.004	`0.6642998` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6642998` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6642998` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6708963` no matchIn the final answer: no
Z' plate 6, 10 µlPrinted in the paper	`0.6319`	± 0.004	`0.6318892` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6318892` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6318892` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6429603` no matchIn the final answer: no
Z' plate 7, 10 µlPrinted in the paper	`0.7666`	± 0.004	`0.7666218` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7666218` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7666218` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7864944` no matchIn the final answer: no
Z' plate 8, 10 µlPrinted in the paper	`0.7415`	± 0.004	`0.7415492` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7415492` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7415492` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.746225` no matchIn the final answer: no
Z' plate 9, 10 µlPrinted in the paper	`0.621`	± 0.004	`0.6209891` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6209891` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.621` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6429603` no matchIn the final answer: no
Z' plate 10, 10 µlPrinted in the paper	`0.7217`	± 0.004	`0.7216555` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7216555` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7216555` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7170989` no matchIn the final answer: no
Z' plate 2, 20 µlPrinted in the paper	`0.5219`	± 0.004	`0.5218797` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5218797` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5218797` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5218797` matchIn the final answer: no
Z' plate 4, 20 µlPrinted in the paper	`0.7865`	± 0.004	`0.7864944` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7864944` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7864944` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7864944` matchIn the final answer: no
Z' plate 6, 20 µlPrinted in the paper	`0.643`	± 0.004	`0.6429603` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6429603` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.643` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6429603` matchIn the final answer: no
Z' plate 7, 20 µlPrinted in the paper	`0.6709`	± 0.004	`0.6708963` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6708963` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6708963` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6708963` matchIn the final answer: no
Z' plate 8, 20 µlPrinted in the paper	`0.685`	± 0.004	`0.6850474` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6850474` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.685` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6850474` matchIn the final answer: no
Z' plate 9, 20 µlPrinted in the paper	`0.8005`	± 0.004	`0.8004509` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.8004509` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.8004509` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.8004509` matchIn the final answer: no
Z' plate 10, 20 µlPrinted in the paper	`0.7282`	± 0.004	`0.7282149` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7282149` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7282149` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7282149` matchIn the final answer: no
Mean of the positive wells, plate 1, 10 µlPrinted in the paper	`2471.2`	± 0.6	`2471.2` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2471.2` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2471.2` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1643.543` no matchIn the final answer: no
SD of the positive wells, plate 1, 10 µlPrinted in the paper	`268.1`	± 0.6	`268.1216` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`268.1216` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`268.1` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`247.8378` no matchIn the final answer: no
Z' plate 3, 20 µl, as printed (does not reproduce; computed 0.746) (reference)Printed in the paper	`0.65`	± 0.005	`0.6429603` Reference value. Not scored	`0.6429603` Reference value. Not scored	`0.643` Reference value. Not scored	`0.6429603` Reference value. Not scored
Z' plate 5, 20 µl, as printed (does not reproduce; computed 0.852) (reference)Printed in the paper	`0.83`	± 0.005	`0.8524462` Reference value. Not scored	`0.8524462` Reference value. Not scored	`0.852` Reference value. Not scored	`0.8524462` Reference value. Not scored

Open the runs

Research paper

Desveaux 2026

Desveaux JM, Faudry E, Contreras-Martel C, et al.

Neutralizing human monoclonal antibodies that target the PcrV component of the type III secretion system of Pseudomonas aeruginosa act through distinct mechanisms. eLife (2026). doi:10.7554/eLife.105195

IC50 of anti-PcrV antibodies

drc (R), through the drc adapter. The paper used R 4.3.2 with drc.

Opus9 of 9 values match, in each of 3 runs

9 of 9

In the final answer: 8 of 8, in each of 3 runs

Sonnet9 of 9 values match, in each of 3 runs

9 of 9

In the final answer: 8 of 8, in each of 3 runs

Haiku9 of 9 values match, in each of 3 runs

9 of 9

In the final answer: 8 of 8, in each of 3 runs

qwen3:8b9 of 9 values match

9 of 9

In the final answer: 8 of 8

Show each value

Known values and run values for Desveaux 2026, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
IC50 of MEDI3902, injection assay (µg/mL)Printed in the paper	`0.11725`	± 0.0006	`0.11725` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1172502` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1172502` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1172502` matchIn the final answer: yes
IC50 of 30-B8, injection assay (µg/mL)Printed in the paper	`0.021307`	± 0.0001	`0.021307` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.02130746` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.02130746` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.02130746` matchIn the final answer: yes
IC50 of P3D6, injection assay (µg/mL)Printed in the paper	`3.6534`	± 0.006	`3.653446` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.653446` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.653446` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.653446` matchIn the final answer: yes
IC50 of P3D6, cytotoxicity assay (µg/mL)Printed in the paper	`11.792`	± 0.06	`11.79184` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`11.79184` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`11.79184` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`11.79184` matchIn the final answer: yes
IC50 of 30-B8 against variant V2 (µg/mL)Printed in the paper	`0.012782`	± 6e-05	`0.012782` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.01278244` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.01278244` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.01278244` matchIn the final answer: yes
IC50 of 30-B8 against variant V3 (µg/mL)Printed in the paper	`0.011202`	± 6e-05	`0.011202` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.01120237` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.01120237` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.01120237` matchIn the final answer: yes
IC50 of 30-B8 against variant V4 (µg/mL)Printed in the paper	`0.01305`	± 6e-05	`0.01305` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.0130497` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.0130497` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.0130497` matchIn the final answer: yes
IC50 of 30-B8 against variant V5 (µg/mL)Printed in the paper	`0.010452`	± 6e-05	`0.010452` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.01045157` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.01045157` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.01045157` matchIn the final answer: yes
IC50 of 30-B8, cytotoxicity assay, three-parameter model (µg/mL)We calculated it with drc adapter fit_dose_response (drc 4.0.0 LL.3). Independent check: yes	`6.6996`	± 0.05	`6.699555` match in 3 of 3 runsNot asked in the question	`6.699555` match in 3 of 3 runsNot asked in the question	`6.699555` match in 3 of 3 runsNot asked in the question	`6.699555` matchNot asked in the question
IC50 of 30-B8, cytotoxicity assay, as printed (µg/mL; reproduces only with a 4PL) (reference)Printed in the paper	`0.0452`	± 0.0002	`0.0451` Reference value. Not scored	`0.0563` Reference value. Not scored	`0.06375493` Reference value. Not scored	`0.06375493` Reference value. Not scored

Open the runs

Research paper

Hildyard 2021

Hildyard JCW, Wells DJ, Piercy RJ.

Identification of qPCR reference genes suitable for normalising gene expression in the developing mouse embryo. Wellcome Open Research 6:197 (version 2) (2021). doi:10.12688/wellcomeopenres.16972.2

qPCR reference genes for the developing mouse embryo

NormqPCR (R), through the qpcr adapter. The paper used the geNorm and NormFinder Excel add-ins, Excel and Prism 8.

Opus18 of 18 values match, in each of 3 runs

18 of 18

In the final answer: 16 of 16, in each of 3 runs

Sonnet18 of 18 values match, in each of 3 runs

18 of 18

In the final answer: 16 of 16, in each of 3 runs

Haiku16 to 18 of 18 values match, over 3 runs

16 to 18 of 18

In the final answer: 14 to 16 of 16, over 3 runs

qwen3:8b10 of 18 values match

10 of 18

In the final answer: 0 of 16

Show each value

Known values and run values for Hildyard 2021, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Number of samplesPrinted in the paper	`44`	exact	`44` match in 3 of 3 runsNot asked in the question	`44` match in 3 of 3 runsNot asked in the question	`44` match in 3 of 3 runsNot asked in the question	`44` matchNot asked in the question
GeNorm average M of the best pair HTATSF1 and CDC40, all samplesPrinted in the paper	`0.217549`	± 0.0005	`0.2175489` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.2175489` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.2175489` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.2175489` matchIn the final answer: no
GeNorm average M of all 15 genes (step that removes HPRT1)Printed in the paper	`0.527538`	± 0.0005	`0.5275384` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5275384` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5275384` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5275384` matchIn the final answer: no
GeNorm average M of 13 genes (step that removes GAPDH)Printed in the paper	`0.440995`	± 0.0005	`0.4409948` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4409948` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4409948` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4409948` matchIn the final answer: no
GeNorm average M of 3 genes (step that removes ACTB)Printed in the paper	`0.227746`	± 0.0005	`0.2277458` match in 3 of 3 runsNot asked in the question	`0.2277458` match in 3 of 3 runsNot asked in the question	`0.2277458` match in 3 of 3 runsNot asked in the question	`0.2277458` matchNot asked in the question
Genes with geNorm average M below 0.5Printed in the paper	`14`	exact	`14` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`14` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`14` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`14` matchIn the final answer: no
GeNorm pairwise variation V2/3Printed in the paper	`0.06864`	± 0.0003	`0.06863984` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.06863984` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.06863984` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.06863984` matchIn the final answer: no
NormFinder stability, no groups, AP3D1 (most stable)Printed in the paper	`0.134827`	± 0.0005	`0.1348269` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1348269` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1348269` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1348269` matchIn the final answer: no
NormFinder stability, no groups, HPRT1 (least stable)Printed in the paper	`0.480987`	± 0.0005	`0.480987` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.480987` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.480987` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.480987` matchIn the final answer: no
NormFinder stability grouped by tissue, UBC (most stable)Printed in the paper	`0.081032`	± 0.0005	`0.08103166` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.08103166` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.06863984` no match in 1 of 3 runsIn the final answer: yes in 2 of 3 runs	`0.06863984` no matchIn the final answer: no
NormFinder stability grouped by tissue, B2M (least stable)Printed in the paper	`0.180476`	± 0.0005	`0.1804759` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1804759` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1831813` no match in 1 of 3 runsIn the final answer: yes in 2 of 3 runs	`0.1831813` no matchIn the final answer: no
GeNorm average M of the best pair, head samplesPrinted in the paper	`0.114216`	± 0.0005	`0.1142158` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1142158` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1142158` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1142158` matchIn the final answer: no
Standard curve slope, GAPDHPrinted in the paper	`-3.4557`	± 0.0006	`-3.455719` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-3.455719` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-3.455719` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0` no matchIn the final answer: no
Standard curve slope, HTATSF1Printed in the paper	`-3.1799`	± 0.0006	`-3.179886` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-3.179886` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-3.179886` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0` no matchIn the final answer: no
Standard curve slope, UBCPrinted in the paper	`-3.3788`	± 0.0006	`-3.37875` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-3.37875` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-3.37875` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0` no matchIn the final answer: no
Efficiency, CYC1Printed in the paper	`1.945026`	± 0.0005	`1.945012` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.945012` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.945012` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2` no matchIn the final answer: no
Efficiency, PAK1IP1Printed in the paper	`2.012002`	± 0.0005	`2.011994` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.011994` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.011994` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2` no matchIn the final answer: no
Standard curve slope, ACTB, from the dataWe calculated it with fit_efficiency (base R lm). Independent check: yes	`-3.2993`	± 0.0006	`-3.299324` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-3.299324` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-3.299324` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0` no matchIn the final answer: no
Standard curve slope, ACTB, as printed (does not reproduce; the data give -3.2993) (reference)Printed in the paper	`-3.2293`	± 0.0006	`-3.233121` Reference value. Not scored	`-3.233121` Reference value. Not scored	`-3.233121` Reference value. Not scored	`0` Reference value. Not scored

Open the runs

Research paper

Klindworth 2013

Klindworth A, Pruesse E, Schweer T, et al.

Evaluation of general 16S ribosomal RNA gene PCR primers for classical and next-generation sequencing-based diversity studies. Nucleic Acids Research 41(1):e1 (2013). doi:10.1093/nar/gks808

Also: Brosius J, Palmer ML, Kennedy PJ, Noller HF. Complete nucleotide sequence of a 16S ribosomal RNA gene from Escherichia coli. PNAS 75(10):4801-4805 (1978). doi:10.1073/pnas.75.10.4801

general 16S rRNA gene primers on the E. coli rrnB operon

Biopython and primer3-py, through the cloning adapter

Opus11 of 11 values match, in each of 3 runs

11 of 11

In the final answer: 11 of 11, in each of 3 runs

Sonnet11 of 11 values match, in each of 3 runs

11 of 11

In the final answer: 11 of 11, in each of 3 runs

Haiku11 of 11 values match, in each of 3 runs

11 of 11

In the final answer: 11 of 11, in each of 3 runs

qwen3:8b9 of 11 values match

9 of 11

In the final answer: 9 of 11

Show each value

Known values and run values for Klindworth 2013, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Number of products from pair 1We calculated it with the cloning adapter, simulate_pcr. Independent check: yes	`1`	exact	`1` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1` matchIn the final answer: yes
Start of the pair 1 product on J01695.2We calculated it with the cloning adapter, simulate_pcr. Independent check: yes	`1608`	exact	`1608` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1608` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1608` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1608` matchIn the final answer: yes
End of the pair 1 product on J01695.2We calculated it with the cloning adapter, simulate_pcr. Independent check: yes	`2072`	exact	`2072` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2072` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2072` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2072` matchIn the final answer: yes
Length of the pair 1 product, first to last baseWe calculated it with the cloning adapter, simulate_pcr. Independent check: yes	`465`	exact	`465` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`465` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`465` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`465` matchIn the final answer: yes
Start of the pair 1 forward primer in the E. coli 16S numberingPrinted in the paper	`341`	exact	`341` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`341` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`341` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`465` no matchIn the final answer: no
Number of products from pair 2We calculated it with the cloning adapter, simulate_pcr. Independent check: yes	`1`	exact	`1` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1` matchIn the final answer: yes
Start of the pair 2 product on J01695.2We calculated it with the cloning adapter, simulate_pcr. Independent check: yes	`1275`	exact	`1275` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1275` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1275` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1275` matchIn the final answer: yes
End of the pair 2 product on J01695.2We calculated it with the cloning adapter, simulate_pcr. Independent check: yes	`2193`	exact	`2193` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2193` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2193` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2193` matchIn the final answer: yes
Length of the pair 2 product, first to last baseWe calculated it with the cloning adapter, simulate_pcr. Independent check: yes	`919`	exact	`919` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`919` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`919` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`919` matchIn the final answer: yes
Start of the pair 2 forward primer in the E. coli 16S numberingPrinted in the paper	`8`	exact	`8` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`8` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`8` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2` no matchIn the final answer: no
Melting temperature of CCTACGGGAGGCAGCAG, in degrees CWe calculated it with primer3-py 2.3.1 calc_tm. Independent check: yes	`61.39`	± 0.01	`61.386` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`61.386` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`61.386` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`61.386` matchIn the final answer: yes
Difference of the two primer start positions in the E. coli numbering (the published amplicon size) (reference)Printed in the paper	`464`	exact	`465` Reference value. Not scored	`465` Reference value. Not scored	`465` Reference value. Not scored	`465` Reference value. Not scored

Open the runs

Research paper

Yun 2026

Yun H, Sigei F, Appiah NYA, et al.

Development and qualification of an enzyme-linked immunosorbent assay to detect human serum immunoglobulin G reactive to multiple lineages of Lassa virus nucleoprotein. PLOS ONE 21(7): e0340568 (2026). doi:10.1371/journal.pone.0340568

an ELISA for IgG against Lassa virus nucleoprotein

drc (R), through the drc adapter. The paper used BioTek Gen5 3.16.

Opus30 of 30 values match, in each of 3 runs

30 of 30

In the final answer: 28 of 28, in each of 3 runs

Sonnet30 of 30 values match, in each of 3 runs

30 of 30

In the final answer: 28 of 28, in each of 3 runs

Haiku30 of 30 values match, in each of 3 runs

30 of 30

In the final answer: 28 of 28, in each of 3 runs

qwen3:8b30 of 30 values match

30 of 30

In the final answer: 25 of 28

Show each value

Known values and run values for Yun 2026, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Number of runs with a standard curvePrinted in the paper	`9`	exact	`9` match in 3 of 3 runsNot asked in the question	`9` match in 3 of 3 runsNot asked in the question	`9` match in 3 of 3 runsNot asked in the question	`9` matchNot asked in the question
Mean interpolated concentration, dilution 1 (IU/mL)Printed in the paper	`7.5869`	± 0.001	`7.5869` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7.5869` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7.586938` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7.586938` matchIn the final answer: yes
Mean interpolated concentration, dilution 2 (IU/mL)Printed in the paper	`3.7072`	± 0.001	`3.7072` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.7072` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.707172` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.707172` matchIn the final answer: yes
Mean interpolated concentration, dilution 3 (IU/mL)Printed in the paper	`1.8719`	± 0.001	`1.8719` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.8719` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.87188` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.87188` matchIn the final answer: yes
Mean interpolated concentration, dilution 4 (IU/mL)Printed in the paper	`0.91706`	± 0.001	`0.917061` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.917061` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.917061` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.917061` matchIn the final answer: yes
Mean interpolated concentration, dilution 5 (IU/mL)Printed in the paper	`0.46371`	± 0.001	`0.4637134` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4637134` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.463713` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4637134` matchIn the final answer: yes
Mean interpolated concentration, dilution 6 (IU/mL)Printed in the paper	`0.2334`	± 0.001	`0.2334` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.2334` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.233402` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.2334023` matchIn the final answer: yes
Mean interpolated concentration, dilution 7 (IU/mL)Printed in the paper	`0.12046`	± 0.0006	`0.1204556` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1204556` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.120456` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1204556` matchIn the final answer: yes
Mean interpolated concentration, dilution 8 (IU/mL)Printed in the paper	`0.055794`	± 0.0006	`0.05579413` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.05579413` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.055794` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.05579413` matchIn the final answer: yes
Mean interpolated concentration, dilution 9 (IU/mL)Printed in the paper	`0.026613`	± 0.0006	`0.02661318` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.02661318` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.026613` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.02661318` matchIn the final answer: yes
Mean interpolated concentration, dilution 10 (IU/mL)Printed in the paper	`0.013885`	± 0.0006	`0.01388456` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.01388456` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.013885` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.01388456` matchIn the final answer: yes
CV of the interpolated concentration, dilution 1 (%)Printed in the paper	`11.07`	± 0.015	`11.07` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`11.07` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`11.07001` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`11.07001` matchIn the final answer: yes
CV of the interpolated concentration, dilution 4 (%)Printed in the paper	`10.927`	± 0.015	`10.9268` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10.9268` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10.92676` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10.92676` matchIn the final answer: yes
CV of the interpolated concentration, dilution 7 (%)Printed in the paper	`15.714`	± 0.015	`15.71361` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`15.71361` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`15.71361` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`15.71361` matchIn the final answer: yes
CV of the interpolated concentration, dilution 8 (%)Printed in the paper	`15.553`	± 0.015	`15.55339` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`15.55339` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`15.55339` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`15.55339` matchIn the final answer: yes
CV of the interpolated concentration, dilution 9 (%)Printed in the paper	`25.895`	± 0.015	`25.89455` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`25.89455` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`25.89455` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`25.89455` matchIn the final answer: yes
CV of the OD, dilution 7 (%)Printed in the paper	`21.919`	± 0.015	`21.9188` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`21.9188` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`21.9188` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`21.9188` matchIn the final answer: no
CV of the OD, dilution 8 (%)Printed in the paper	`26.621`	± 0.015	`26.6206` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`26.6206` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`26.62058` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`26.62058` matchIn the final answer: no
Mean OD, dilution 1Printed in the paper	`2.8867`	± 0.001	`2.8867` match in 3 of 3 runsNot asked in the question	`2.8867` match in 3 of 3 runsNot asked in the question	`2.886722` match in 3 of 3 runsNot asked in the question	`2.886722` matchNot asked in the question
Recovery, dilution 1 (%)Printed in the paper	`97.97`	± 0.06	`97.9718` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`97.9718` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`97.97182` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`97.97182` matchIn the final answer: yes
Recovery, dilution 4 (%)Printed in the paper	`94.74`	± 0.06	`94.73771` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`94.73771` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`94.73771` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`94.73771` matchIn the final answer: yes
Recovery, dilution 7 (%)Printed in the paper	`99.55`	± 0.06	`99.55007` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`99.55007` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`99.55007` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`99.55007` matchIn the final answer: yes
Recovery, dilution 9 (%)Printed in the paper	`88.71`	± 0.06	`88.7106` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`88.7106` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`88.71061` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`88.71061` matchIn the final answer: yes
Recovery, dilution 8, as printed in Table 1 (does not reproduce; computed 92.99) (reference)Printed in the paper	`92`	± 0.06	`92.56376` Reference value. Not scored	`92.56376` Reference value. Not scored	`92.56376` Reference value. Not scored	`92.56376` Reference value. Not scored
HPC mean interpolated concentration (IU/mL)Printed in the paper	`1.9932`	± 0.001	`1.9932` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.9932` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.993188` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.993188` matchIn the final answer: yes
HPC CV of the interpolated concentration (%)Printed in the paper	`22.447`	± 0.015	`22.44729` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`22.44729` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`22.44729` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`22.44729` matchIn the final answer: yes
HPC recovery (%)Printed in the paper	`107.28`	± 0.06	`107.276` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`107.276` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`107.276` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`107.276` matchIn the final answer: yes
LPC mean interpolated concentration (IU/mL)Printed in the paper	`0.45688`	± 0.001	`0.4568795` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4568795` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4568795` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4568795` matchIn the final answer: yes
LPC recovery (%)Printed in the paper	`98.25`	± 0.06	`98.2536` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`98.2536` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`98.25365` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`98.25365` matchIn the final answer: yes
LPC CV of the interpolated concentration, as printed (does not reproduce; computed 20.03) (reference)Printed in the paper	`20.09`	± 0.015	`20.028` Reference value. Not scored	`20.028` Reference value. Not scored	`20.02798` Reference value. Not scored	`20.02798` Reference value. Not scored
LLOQ (IU/mL)Printed in the paper	`0.121`	± 0.0005	`0.121` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.121` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.121` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.121` matchIn the final answer: yes
ULOQ (IU/mL)Printed in the paper	`7.744`	± 0.0005	`7.744` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7.744` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7.744` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7.744` matchIn the final answer: no

Open the runs

Statistics

Research paper

Bates 2015

Bates D, Maechler M, Bolker B, et al.

Fitting linear mixed-effects models using lme4. Journal of Statistical Software 67(1):1-48 (2015). doi:10.18637/jss.v067.i01

Also: Belenky G, et al. Patterns of performance degradation and restoration during sleep restriction and subsequent recovery: a sleep dose-response study. Journal of Sleep Research 12(1):1-12 (2003). doi:10.1046/j.1365-2869.2003.00337.x

lme4, the sleepstudy data

lme4 with lmerTest (R), through the lme4 adapter

Opus9 of 9 values match, in each of 3 runs

9 of 9

In the final answer: 7 of 7, in each of 3 runs

Sonnet9 of 9 values match, in each of 3 runs

9 of 9

In the final answer: 7 of 7, in each of 3 runs

Haiku9 of 9 values match, in each of 3 runs

9 of 9

In the final answer: 7 of 7, in each of 3 runs

qwen3:8b9 of 9 values match

9 of 9

In the final answer: 7 of 7

Show each value

Known values and run values for Bates 2015, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Fixed effect InterceptPrinted in the paper	`251.405`	± 0.01	`251.4051` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`251.4051` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`251.4051` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`251.4051` matchIn the final answer: yes
Fixed effect DaysPrinted in the paper	`10.467`	± 0.01	`10.46729` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10.46729` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10.46729` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10.46729` matchIn the final answer: yes
REML criterionPrinted in the paper	`1743.6`	± 0.1	`1743.628` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1743.628` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1743.628` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1743.628` matchIn the final answer: yes
SD of Subject interceptPrinted in the paper	`24.74`	± 0.05	`24.74066` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`24.74066` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`24.74066` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`24.74066` matchIn the final answer: yes
SD of Subject DaysPrinted in the paper	`5.92`	± 0.05	`5.922138` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`5.922138` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`5.922138` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`5.922138` matchIn the final answer: yes
Residual SDPrinted in the paper	`25.59`	± 0.05	`25.5918` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`25.5918` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`25.5918` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`25.5918` matchIn the final answer: yes
Correlation of random intercept and slopePrinted in the paper	`0.066`	± 0.01	`0.06555124` match in 3 of 3 runsNot asked in the question	`0.06555124` match in 3 of 3 runsNot asked in the question	`0.06555124` match in 3 of 3 runsNot asked in the question	`0.06555124` matchNot asked in the question
Likelihood ratio chi-square for Days, MLWe calculated it with lme4 2.0.6 and lmerTest 3.2.1 on R 4.6.1. Independent check: yes	`23.54`	± 0.1	`23.53654` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`23.53654` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`23.53654` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`23.53654` matchIn the final answer: yes
Number of subjectsPrinted in the paper	`18`	exact	`18` match in 3 of 3 runsNot asked in the question	`18` match in 3 of 3 runsNot asked in the question	`18` match in 3 of 3 runsNot asked in the question	`18` matchNot asked in the question

Open the runs

Tutorial or software test data

Belenky 2003

Belenky G, Wesensten NJ, Thorne DR, et al.

Patterns of performance degradation and restoration during sleep restriction and subsequent recovery: a sleep dose-response study. J Sleep Res 12(1):1-12 (2003). doi:10.1046/j.1365-2869.2003.00337.x

Also: Bates D, Maechler M, Bolker B, Walker S. Fitting linear mixed-effects models using lme4. Journal of Statistical Software 67(1):1-48 (2015). doi:10.18637/jss.v067.i01

reaction time over days of sleep restriction

statsmodels mixedlm (Python), through the biostats adapter

Opus6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 2 of 2, in each of 3 runs

Sonnet6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 2 of 2, in each of 3 runs

Haiku6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 2 of 2, in each of 3 runs

qwen3:8b6 of 6 values match

6 of 6

In the final answer: 2 of 2

Show each value

Known values and run values for Belenky 2003, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Days slope in ms per dayPrinted in the paper	`10.467`	± 0.01	`10.46729` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10.467` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10.46729` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`10.46729` matchIn the final answer: yes
Intercept in msPrinted in the paper	`251.405`	± 0.01	`251.4051` match in 3 of 3 runsNot asked in the question	`251.405` match in 3 of 3 runsNot asked in the question	`251.4051` match in 3 of 3 runsNot asked in the question	`251.4051` matchNot asked in the question
SE of Days, random slope modelPrinted in the paper	`1.546`	± 0.01	`1.545788` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.545788` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.545788` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.545788` matchIn the final answer: yes
Subject intercept variance, slope modelPrinted in the paper	`612.1`	± 0.5	`612.0965` match in 3 of 3 runsNot asked in the question	`612.0965` match in 3 of 3 runsNot asked in the question	`612.0965` match in 3 of 3 runsNot asked in the question	`612.0965` matchNot asked in the question
Slope variancePrinted in the paper	`35.07`	± 0.1	`35.07162` match in 3 of 3 runsNot asked in the question	`35.07162` match in 3 of 3 runsNot asked in the question	`35.07162` match in 3 of 3 runsNot asked in the question	`35.07162` matchNot asked in the question
Residual variance, slope modelPrinted in the paper	`654.94`	± 0.5	`654.9405` match in 3 of 3 runsNot asked in the question	`654.9405` match in 3 of 3 runsNot asked in the question	`654.9405` match in 3 of 3 runsNot asked in the question	`654.9405` matchNot asked in the question

Open the runs

Research paper

Oreshkova 2024

Oreshkova A, Scofield S, Amdam GV.

The effects of queen mandibular pheromone on nurse-aged honey bee (Apis mellifera) hypopharyngeal gland size and lipid metabolism. PLOS ONE 19(9):e0292500 (2024). doi:10.1371/journal.pone.0292500

Data: doi:10.6084/m9.figshare.24164316

Data: doi:10.6084/m9.figshare.24164265

queen mandibular pheromone, honey bee glands and lipid metabolism

statsmodels and SciPy (Python), through the biostats adapter

Opus19 of 19 values match, in each of 3 runs

19 of 19

In the final answer: 16 of 16, in each of 3 runs

Sonnet19 of 19 values match, in each of 3 runs

19 of 19

In the final answer: 16 of 16, in each of 3 runs

Haiku19 of 19 values match, in each of 3 runs

19 of 19

In the final answer: 11 to 16 of 16, over 3 runs

qwen3:8b18 of 19 values match

18 of 19

In the final answer: 15 of 16

Show each value

Known values and run values for Oreshkova 2024, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Abdominal protein, F for agePrinted in the paper	`35.693`	± 0.001	`35.6928` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`35.6928` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`35.6928` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`35.6928` matchIn the final answer: yes
Abdominal protein, F for QMP treatmentPrinted in the paper	`0.032`	± 0.0005	`0.03219331` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.03156445` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.03156445` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.03156445` matchIn the final answer: yes
Abdominal protein, F for replicatePrinted in the paper	`36`	± 0.001	`35.99952` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`35.99952` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`35.99952` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`35.99952` matchIn the final answer: yes
Abdominal protein, p for ageWe calculated it with statsmodels 0.15.0 anova_lm. Independent check: yes	`1.356e-07`	± 1e-07	`1.356049e-07` match in 3 of 3 runsNot asked in the question	`1.356049e-07` match in 3 of 3 runsNot asked in the question	`1.356049e-07` match in 3 of 3 runsNot asked in the question	`1.356049e-07` matchNot asked in the question
Abdominal protein, p for QMP treatmentPrinted in the paper	`0.8596`	± 0.0001	`0.8595854` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.8595854` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.8595854` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.8595854` matchIn the final answer: yes
Abdominal protein, F for age by treatmentPrinted in the paper	`1.057`	± 0.001	`1.056653` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.056653` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.056653` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.056653` matchIn the final answer: yes
Abdominal protein, F for age by replicatePrinted in the paper	`2.809`	± 0.001	`2.808638` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.809` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.808506` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.808506` matchIn the final answer: yes
Residual degrees of freedom of the three-way ANOVAPrinted in the paper	`60`	exact	`60` match in 3 of 3 runsNot asked in the question	`60` match in 3 of 3 runsNot asked in the question	`60` match in 3 of 3 runsNot asked in the question	`60` matchNot asked in the question
Acini area, F for agePrinted in the paper	`27.78`	± 0.001	`27.77991` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`27.77991` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`27.77991` match in 3 of 3 runsIn the final answer: yes in 2 of 3 runs	`27.77991` matchIn the final answer: yes
Acini area, F for QMP treatmentPrinted in the paper	`29.156`	± 0.001	`29.15553` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`29.15553` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`29.15553` match in 3 of 3 runsIn the final answer: yes in 2 of 3 runs	`29.15553` matchIn the final answer: yes
Acini area, F for replicatePrinted in the paper	`6.853`	± 0.001	`6.853006` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`6.853006` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`6.853006` match in 3 of 3 runsIn the final answer: yes in 2 of 3 runs	`6.853006` matchIn the final answer: yes
Acini area, p for replicatePrinted in the paper	`0.00209`	± 1e-05	`0.002086653` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.002086653` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.002086653` match in 3 of 3 runsIn the final answer: yes in 2 of 3 runs	`0.002086653` matchIn the final answer: yes
Acini area, F for age by treatment by replicatePrinted in the paper	`4.457`	± 0.001	`4.457195` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`4.457195` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`4.457195` match in 3 of 3 runsIn the final answer: yes in 2 of 3 runs	`4.457195` matchIn the final answer: yes
Acini area, p for age by treatment by replicatePrinted in the paper	`0.01568`	± 1e-05	`0.01567617` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.01567617` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.01567617` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.01567617` matchIn the final answer: yes
Kruskal-Wallis chi-squared for the four treatment groupsPrinted in the paper	`9.3498`	± 0.0001	`9.349822` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`9.349822` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`9.349822` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`9.349822` matchIn the final answer: yes
Kruskal-Wallis degrees of freedomPrinted in the paper	`3`	exact	`3` match in 3 of 3 runsNot asked in the question	`3` match in 3 of 3 runsNot asked in the question	`3` match in 3 of 3 runsNot asked in the question	`3` matchNot asked in the question
Kruskal-Wallis p for the four treatment groupsPrinted in the paper	`0.02498`	± 1e-05	`0.02498385` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.02498385` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.02498385` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.02498385` matchIn the final answer: yes
Dunn adjusted p, the only significant pairPrinted in the paper	`0.0397`	± 0.0001	`0.03969327` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.03969327` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.03969327` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.03969327` matchIn the final answer: yes
Kruskal-Wallis chi-squared for the three replicatesPrinted in the paper	`23.87`	± 0.005	`23.87005` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`23.87005` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`23.87005` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`25.44444` no matchIn the final answer: no

Open the runs

Tutorial or software test data

Rossi 1980

Rossi PH, Berk RA, Lenihan KJ.

Money, Work, and Crime: Experimental Evidence. Academic Press (1980). https://vincentarelbundock.github.io/Rdatasets/doc/carData/Rossi.html

time to re-arrest after prison release

lifelines (Python), through the biostats adapter

Opus11 of 11 values match, in each of 3 runs

11 of 11

In the final answer: 4 of 4, in each of 3 runs

Sonnet10 to 11 of 11 values match, over 3 runs

10 to 11 of 11

In the final answer: 4 of 4, in each of 3 runs

Haiku11 of 11 values match, in each of 3 runs

11 of 11

In the final answer: 4 of 4, in each of 3 runs

qwen3:8b6 of 11 values match

6 of 11

In the final answer: 0 of 4

Show each value

Known values and run values for Rossi 1980, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Number of subjectsPrinted in the official tutorial	`432`	exact	`432` match in 3 of 3 runsNot asked in the question	`432` match in 3 of 3 runsNot asked in the question	`432` match in 3 of 3 runsNot asked in the question	`432` matchNot asked in the question
Number of events (arrests)Printed in the official tutorial	`114`	exact	`114` match in 3 of 3 runsNot asked in the question	`114` match in 3 of 3 runsNot asked in the question	`114` match in 3 of 3 runsNot asked in the question	`114` matchNot asked in the question
Cox coefficient for finPrinted in the paper	`-0.3794`	± 0.0005	`-0.3794222` match in 3 of 3 runsNot asked in the question	`-0.3794222` match in 3 of 3 runsNot asked in the question	`-0.3794222` match in 3 of 3 runsNot asked in the question	`-0.3787611` no matchNot asked in the question
Cox coefficient for agePrinted in the paper	`-0.0574`	± 0.0005	`-0.05743774` match in 3 of 3 runsNot asked in the question	`-0.05743774` match in 3 of 3 runsNot asked in the question	`-0.05743774` match in 3 of 3 runsNot asked in the question	`-0.09559877` no matchNot asked in the question
Cox coefficient for prioPrinted in the paper	`0.0915`	± 0.0005	`0.09149708` match in 3 of 3 runsNot asked in the question	`0.09149708` match in 3 of 3 runsNot asked in the question	`0.09149708` match in 3 of 3 runsNot asked in the question	`0.09361026` no matchNot asked in the question
Hazard ratio for finPrinted in the paper	`0.684`	± 0.002	`0.683756` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6842567` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6842567` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6838638` matchIn the final answer: no
P for finPrinted in the paper	`0.0474`	± 0.001	`0.0474161` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.0474161` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.0474161` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.04688014` matchIn the final answer: no
ConcordancePrinted in the official tutorial	`0.64`	± 0.005	`0.6403292` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6403292` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6403292` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.640955` matchIn the final answer: no
Log-rank chi-square by finWe calculated it with lifelines 0.30.3 logrank_test. Independent check: yes	`3.838`	± 0.01	`3.83757` match in 3 of 3 runsNot asked in the question	`2.503162` no match in 1 of 3 runsNot asked in the question	`3.83757` match in 3 of 3 runsNot asked in the question	`3.820604` no matchNot asked in the question
Log-rank p by finWe calculated it with lifelines 0.30.3 logrank_test. Independent check: yes	`0.0501`	± 0.001	`0.05011612` match in 3 of 3 runsNot asked in the question	`0.05011612` match in 3 of 3 runsNot asked in the question	`0.05011612` match in 3 of 3 runsNot asked in the question	`0.05` matchNot asked in the question
Proportional hazards test p for agePrinted in the official tutorial	`0.0007`	± 0.0005	`0.000696` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.000713596` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.000713596` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7.076249e-10` no matchIn the final answer: no

Open the runs

Research paper

Student 1908

Student (William Sealy Gosset).

The probable error of a mean. Biometrika 6(1):1-25 (1908). doi:10.2307/2331554

Also: Cushny AR, Peebles AR. The action of optical isomers II: hyoscines. Journal of Physiology 32:501-510 (1905). doi:10.1113/jphysiol.1905.sp001097

paired t-test on the Cushny and Peebles sleep data

SciPy (Python), through the biostats adapter

Opus6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 4 of 4, in each of 3 runs

Sonnet6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 4 of 4, in each of 3 runs

Haiku6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 4 of 4, in each of 3 runs

qwen3:8b6 of 6 values match

6 of 6

In the final answer: 4 of 4

Show each value

Known values and run values for Student 1908, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Mean paired difference, group 2 minus group 1Printed in the paper	`1.58`	± 0.005	`1.58` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.58` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.58` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.58` matchIn the final answer: yes
Paired t statistic, absolute valueWe calculated it with SciPy 1.x ttest_rel. Independent check: yes	`4.0621`	± 0.001	`4.062128` match in 3 of 3 runsNot asked in the question	`4.062128` match in 3 of 3 runsNot asked in the question	`4.062128` match in 3 of 3 runsNot asked in the question	`4.062128` matchNot asked in the question
Degrees of freedomWe calculated it with SciPy 1.x ttest_rel. Independent check: yes	`9`	exact	`9` match in 3 of 3 runsNot asked in the question	`9` match in 3 of 3 runsNot asked in the question	`9` match in 3 of 3 runsNot asked in the question	`9` matchNot asked in the question
Paired two-sided pWe calculated it with SciPy 1.x ttest_rel. Independent check: yes	`0.002833`	± 0.0001	`0.00283289` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.00283289` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.00283289` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.00283289` matchIn the final answer: yes
95% CI lower boundWe calculated it with SciPy 1.x. Independent check: yes	`0.7`	± 0.005	`0.7001142` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7001142` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7001142` matchIn the final answer: yes
95% CI upper boundWe calculated it with SciPy 1.x. Independent check: yes	`2.46`	± 0.005	`2.459886` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.459886` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.459886` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.459886` matchIn the final answer: yes

Open the runs

Clinical and epidemiology

Research paper

Andreasson 2019

Andreasson A, Karshikoff B, Lidberg L, et al.

The effect of a transient immune activation on subjective health perception in two placebo controlled randomised experiments. PLOS ONE 14(3):e0212313 (2019). doi:10.1371/journal.pone.0212313

Data: doi:10.5061/dryad.sh00j9s

endotoxin and self-rated health, a placebo-controlled experiment

statsmodels and SciPy (Python), through the biostats adapter

Opus10 of 10 values match, in each of 3 runs

10 of 10

In the final answer: 7 of 7, in each of 3 runs

Sonnet10 of 10 values match, in each of 3 runs

10 of 10

In the final answer: 7 of 7, in each of 3 runs

Haiku10 of 10 values match, in each of 3 runs

10 of 10

In the final answer: 7 of 7, in each of 3 runs

qwen3:8b9 of 10 values match

9 of 10

In the final answer: 6 of 7

Show each value

Known values and run values for Andreasson 2019, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Current health ratings in the modelWe calculated it with statsmodels 0.15.0 MixedLM. Independent check: yes	`151`	exact	`151` match in 3 of 3 runsNot asked in the question	`151` match in 3 of 3 runsNot asked in the question	`151` match in 3 of 3 runsNot asked in the question	`151` matchNot asked in the question
Subjects in the modelPrinted in the paper	`52`	exact	`52` match in 3 of 3 runsNot asked in the question	`52` match in 3 of 3 runsNot asked in the question	`52` match in 3 of 3 runsNot asked in the question	`52` matchNot asked in the question
Intercept (placebo at baseline)Printed in the paper	`6.238`	± 0.005	`6.238095` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`6.238095` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`6.238095` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`6.238095` matchIn the final answer: yes
LPS at baselinePrinted in the paper	`0.02`	± 0.005	`0.01996928` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.01996928` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.01996928` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.01996928` matchIn the final answer: yes
Time 90 minutes, placeboPrinted in the paper	`-0.619`	± 0.005	`-0.6190476` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-0.6190476` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-0.6190476` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-0.6190476` matchIn the final answer: yes
Time 270 minutes, placeboPrinted in the paper	`-0.218`	± 0.005	`-0.2181137` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-0.2181137` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-0.2181137` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-0.2181137` matchIn the final answer: yes
LPS by 90 minutes interactionPrinted in the paper	`-1.786`	± 0.005	`-1.785698` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-1.785698` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-1.785698` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-1.785698` matchIn the final answer: yes
LPS by 270 minutes interactionPrinted in the paper	`-0.906`	± 0.005	`-0.9055435` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-0.9055435` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-0.9055435` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-0.9055435` matchIn the final answer: yes
SE of the LPS by 90 minutes interaction, random intercept, MLWe calculated it with statsmodels 0.15.0 MixedLM, ML. Independent check: yes	`0.3157`	± 0.001	`0.315681` match in 3 of 3 runsNot asked in the question	`0.315681` match in 3 of 3 runsNot asked in the question	`0.315681` match in 3 of 3 runsNot asked in the question	`0.315681` matchNot asked in the question
Mann-Whitney p, general health at 90 minutesPrinted in the paper	`0.00416`	± 0.0005	`0.004163466` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.004163466` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.004163466` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	no matchIn the final answer: no

Open the runs

Research paper

Hackshaw 1997

Hackshaw AK, Law MR, Wald NJ.

The accumulated evidence on lung cancer and environmental tobacco smoke. BMJ 315(7114):980-988 (1997). doi:10.1136/bmj.315.7114.980

Also: Hackshaw AK. Lung cancer and passive smoking. Statistical Methods in Medical Research 7(2):119-136 (1998). doi:10.1177/096228029800700203

lung cancer and environmental tobacco smoke, a meta-analysis of 37 studies

metafor (R), through the metafor adapter

Opus4 of 4 values match, in each of 3 runs

4 of 4

In the final answer: 3 of 3, in each of 3 runs

Sonnet4 of 4 values match, in each of 3 runs

4 of 4

In the final answer: 3 of 3, in each of 3 runs

Haiku4 of 4 values match, in each of 3 runs

4 of 4

In the final answer: 3 of 3, in each of 3 runs

qwen3:8b4 of 4 values match

4 of 4

In the final answer: 3 of 3

Show each value

Known values and run values for Hackshaw 1997, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Number of studiesPrinted in the paper	`37`	exact	`37` match in 3 of 3 runsNot asked in the question	`37` match in 3 of 3 runsNot asked in the question	`37` match in 3 of 3 runsNot asked in the question	`37` matchNot asked in the question
Pooled odds ratio, random effects (DerSimonian-Laird)Printed in the paper	`1.2385`	± 0.005	`1.238485` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.238485` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.238485` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.238485` matchIn the final answer: yes
Pooled odds ratio, 95% CI lower boundPrinted in the paper	`1.1292`	± 0.005	`1.129224` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.129224` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.129224` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.129224` matchIn the final answer: yes
Pooled odds ratio, 95% CI upper boundPrinted in the paper	`1.3583`	± 0.005	`1.358319` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.358319` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.358319` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.358319` matchIn the final answer: yes

Open the runs

Research paper

Hochman 1999

Hochman JS, Sleeper LA, Webb JG, et al.

Early revascularization in acute myocardial infarction complicated by cardiogenic shock. N Engl J Med 341(9):625-634 (1999). doi:10.1056/NEJM199908263410901

SHOCK trial, early revascularization in cardiogenic shock

survRM2, survival and base R, through the trial-endpoints adapter

No run yetno result file has this paper

No result file has this paper. It is not in the totals.

Show each value

Known values and run values for Hochman 1999, earlier run of 9 October 2026
Value	Known value	Tolerance	Haiku
Patients in the early revascularization armPrinted in the paper	`152`	exact	not in this run
Patients in the medical stabilization armPrinted in the paper	`150`	exact	not in this run
30-day mortality, early revascularization (fraction)Printed in the paper	`0.467`	± 0.001	not in this run
30-day mortality, medical stabilization (fraction)Printed in the paper	`0.56`	± 0.001	not in this run
30-day risk difference (fraction, early minus medical)Printed in the paper	`-0.093`	± 0.001	not in this run
Lower limit of the 95% interval of the 30-day risk differencePrinted in the paper	`-0.205`	± 0.001	not in this run
Upper limit of the 95% interval of the 30-day risk differencePrinted in the paper	`0.019`	± 0.001	not in this run
P value of the 30-day chi-square test (the paper prints 0.11)Printed in the paper	`0.11`	± 0.006	not in this run
Hazard ratio of death, early revascularization against medical stabilizationPrinted in the paper	`0.74`	± 0.005	not in this run
Lower limit of the 95% interval of the hazard ratioPrinted in the paper	`0.57`	± 0.005	not in this run
Upper limit of the 95% interval of the hazard ratioPrinted in the paper	`0.97`	± 0.005	not in this run
Log-rank p value (the paper prints P = .03)Printed in the paper	`0.03`	± 0.005	not in this run
Kaplan-Meier survival at 1 year, early revascularization (fraction)Printed in the paper	`0.467`	± 0.001	not in this run
Kaplan-Meier survival at 1 year, medical stabilization (fraction)Printed in the paper	`0.336`	± 0.001	not in this run
Difference in 1-year survival (fraction)Printed in the paper	`0.132`	± 0.001	not in this run
Lower limit of the 95% interval of the 1-year survival differencePrinted in the paper	`0.022`	± 0.001	not in this run
Upper limit of the 95% interval of the 1-year survival differencePrinted in the paper	`0.241`	± 0.001	not in this run
Difference of the restricted mean survival time up to 1825 days (days; no published value, check.py) (reference)We calculated it with check.py (NumPy step integral of the Kaplan-Meier curve). Independent check: yes	`224.26`	± 0.5	not in this run

Open the runs

Research paper

Jang 2016

Jang ES, Jeong S-H, Kim J-W, et al.

Diagnostic performance of alpha-fetoprotein, protein induced by vitamin K absence, osteopontin, Dickkopf-1 and its combinations for hepatocellular carcinoma. PLOS ONE 11(3):e0151069 (2016). doi:10.1371/journal.pone.0151069

serum AFP, PIVKA-II, osteopontin and Dickkopf-1 to tell HCC from cirrhosis

pROC (R), through the proc adapter

Opus22 of 22 values match, in each of 3 runs

22 of 22

In the final answer: 20 of 20, in each of 3 runs

Sonnet22 of 22 values match, in each of 3 runs

22 of 22

In the final answer: 20 of 20, in each of 3 runs

Haiku22 of 22 values match, in each of 3 runs

22 of 22

In the final answer: 20 of 20, in each of 3 runs

qwen3:8b22 of 22 values match

22 of 22

In the final answer: 20 of 20

Show each value

Known values and run values for Jang 2016, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Patients with HCCPrinted in the paper	`208`	exact	`208` match in 3 of 3 runsNot asked in the question	`208` match in 3 of 3 runsNot asked in the question	`208` match in 3 of 3 runsNot asked in the question	`208` matchNot asked in the question
Patients with cirrhosisPrinted in the paper	`193`	exact	`193` match in 3 of 3 runsNot asked in the question	`193` match in 3 of 3 runsNot asked in the question	`193` match in 3 of 3 runsNot asked in the question	`193` matchNot asked in the question
AUC of AFPPrinted in the paper	`0.7856`	± 0.001	`0.7856342` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7856342` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7856342` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7856342` matchIn the final answer: yes
AUC of AFP, 95% CI lower boundPrinted in the paper	`0.7404`	± 0.001	`0.7403953` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7403953` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7403953` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7403953` matchIn the final answer: yes
AUC of AFP, 95% CI upper boundPrinted in the paper	`0.8309`	± 0.001	`0.8308731` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.8308731` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.8308731` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.8308731` matchIn the final answer: yes
AUC of PIVKA-IIPrinted in the paper	`0.7294`	± 0.001	`0.7293867` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7293867` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7293867` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7293867` matchIn the final answer: yes
AUC of PIVKA-II, 95% CI lower boundPrinted in the paper	`0.6799`	± 0.001	`0.6798585` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6798585` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6798585` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6798585` matchIn the final answer: yes
AUC of PIVKA-II, 95% CI upper boundPrinted in the paper	`0.7789`	± 0.001	`0.778915` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.778915` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.778915` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.778915` matchIn the final answer: yes
AUC of OPNPrinted in the paper	`0.6596`	± 0.001	`0.6596004` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6596004` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6596004` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6596004` matchIn the final answer: yes
AUC of OPN, 95% CI lower boundPrinted in the paper	`0.6061`	± 0.001	`0.6061004` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6061004` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6061004` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6061004` matchIn the final answer: yes
AUC of OPN, 95% CI upper boundPrinted in the paper	`0.7131`	± 0.001	`0.7131004` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7131004` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7131004` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7131004` matchIn the final answer: yes
AUC of DKK-1Printed in the paper	`0.665`	± 0.001	`0.6649811` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6649811` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6649811` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6649811` matchIn the final answer: yes
AUC of DKK-1, 95% CI lower boundPrinted in the paper	`0.612`	± 0.001	`0.6120051` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6120051` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6120051` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6120051` matchIn the final answer: yes
AUC of DKK-1, 95% CI upper boundPrinted in the paper	`0.718`	± 0.001	`0.7179571` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7179571` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7179571` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.7179571` matchIn the final answer: yes
Sensitivity of AFP at 20 ng/mLPrinted in the paper	`0.6202`	± 0.001	`0.6201923` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6201923` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6201923` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6201923` matchIn the final answer: yes
Specificity of AFP at 20 ng/mLPrinted in the paper	`0.9016`	± 0.001	`0.9015544` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9015544` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9015544` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9015544` matchIn the final answer: yes
Sensitivity of PIVKA-II at 10 ng/mLPrinted in the paper	`0.5096`	± 0.001	`0.5096154` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5096154` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5096154` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5096154` matchIn the final answer: yes
Specificity of PIVKA-II at 10 ng/mLPrinted in the paper	`0.9119`	± 0.001	`0.9119171` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9119171` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9119171` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9119171` matchIn the final answer: yes
Sensitivity of OPN at 100 ng/mLPrinted in the paper	`0.4615`	± 0.001	`0.4615385` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4615385` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4615385` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4615385` matchIn the final answer: yes
Specificity of OPN at 100 ng/mLPrinted in the paper	`0.8031`	± 0.001	`0.8031088` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.8031088` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.8031088` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.8031088` matchIn the final answer: yes
Sensitivity of DKK-1 at 500 pg/mLPrinted in the paper	`0.5`	± 0.001	`0.5` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.5` matchIn the final answer: yes
Specificity of DKK-1 at 500 pg/mLPrinted in the paper	`0.8083`	± 0.001	`0.8082902` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.8082902` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.8082902` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.8082902` matchIn the final answer: yes

Open the runs

Research paper

Kasurinen 2018

Kasurinen A, Tervahartiala T, Laitinen A, et al.

High serum MMP-14 predicts worse survival in gastric cancer. PLOS ONE 13(12):e0208800 (2018). doi:10.1371/journal.pone.0208800

Data: doi:10.5061/dryad.hb62394

serum MMP-14 and disease-specific survival in gastric cancer

lifelines and statsmodels (Python), through the biostats adapter

Opus25 of 25 values match, in each of 3 runs

25 of 25

In the final answer: 17 of 17, in each of 3 runs

Sonnet25 of 25 values match, in each of 3 runs

25 of 25

In the final answer: 17 of 17, in each of 3 runs

Haiku25 of 25 values match, in each of 3 runs

25 of 25

In the final answer: 17 of 17, in each of 3 runs

qwen3:8b19 of 25 values match

19 of 25

In the final answer: 0 of 17

Show each value

Known values and run values for Kasurinen 2018, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Number of patientsPrinted in the paper	`240`	exact	`240` match in 3 of 3 runsNot asked in the question	`240` match in 3 of 3 runsNot asked in the question	`240` match in 3 of 3 runsNot asked in the question	`240` matchNot asked in the question
Deaths from gastric cancerWe calculated it with lifelines 0.30.3 KaplanMeierFitter. Independent check: yes	`138`	exact	`138` match in 3 of 3 runsNot asked in the question	`138` match in 3 of 3 runsNot asked in the question	`138` match in 3 of 3 runsNot asked in the question	`138` matchNot asked in the question
5-year disease-specific survival, low MMP-14Printed in the paper	`0.4923`	± 0.001	`0.4922753` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4922753` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4922753` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4922753` matchIn the final answer: no
5-year disease-specific survival, high MMP-14Printed in the paper	`0.2211`	± 0.001	`0.221053` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.2210526` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.2210526` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.2210526` matchIn the final answer: no
Log-rank chi-square, high against low MMP-14We calculated it with lifelines 0.30.3 logrank_test. Independent check: yes	`10.234`	± 0.01	`10.23372` match in 3 of 3 runsNot asked in the question	`10.23372` match in 3 of 3 runsNot asked in the question	`10.23372` match in 3 of 3 runsNot asked in the question	`10.23372` matchNot asked in the question
Log-rank p, high against low MMP-14Printed in the paper	`0.00138`	± 0.0002	`0.00137896` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.00137896` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.00137896` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.00137896` matchIn the final answer: no
Hazard ratio, high MMP-14, unadjustedPrinted in the paper	`1.918`	± 0.005	`1.918004` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.918004` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.918004` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.918004` matchIn the final answer: no
95% CI lower bound, unadjustedPrinted in the paper	`1.278`	± 0.005	`1.277805` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.277805` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.277805` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.277805` matchIn the final answer: no
95% CI upper bound, unadjustedPrinted in the paper	`2.879`	± 0.005	`2.878953` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.878953` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.878953` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.878953` matchIn the final answer: no
P for high MMP-14, unadjustedPrinted in the paper	`0.00167`	± 0.0002	`0.001672369` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.001672369` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.001672369` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.001672369` matchIn the final answer: no
Hazard ratio, high MMP-14, adjusted for age, stage and Lauren typePrinted in the paper	`1.5487`	± 0.005	`1.548716` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.548716` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.548716` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.548716` matchIn the final answer: no
95% CI lower bound, adjustedPrinted in the paper	`1.0205`	± 0.005	`1.020453` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.020453` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.020453` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.020453` matchIn the final answer: no
95% CI upper bound, adjustedPrinted in the paper	`2.3504`	± 0.005	`2.35045` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.35045` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.35045` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.35045` matchIn the final answer: no
P for high MMP-14, adjustedPrinted in the paper	`0.0399`	± 0.0005	`0.03987094` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.03987094` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.03987094` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.03987094` matchIn the final answer: no
Adjusted hazard ratio, age 67 or morePrinted in the paper	`2.516`	± 0.01	`2.515643` match in 3 of 3 runsNot asked in the question	`2.515643` match in 3 of 3 runsNot asked in the question	`2.515643` match in 3 of 3 runsNot asked in the question	`2.515643` matchNot asked in the question
Adjusted hazard ratio, stage IIPrinted in the paper	`6.876`	± 0.02	`6.875289` match in 3 of 3 runsNot asked in the question	`6.875289` match in 3 of 3 runsNot asked in the question	`6.875289` match in 3 of 3 runsNot asked in the question	`6.875289` matchNot asked in the question
Adjusted hazard ratio, stage IIIPrinted in the paper	`22.01`	± 0.05	`22.01381` match in 3 of 3 runsNot asked in the question	`22.01381` match in 3 of 3 runsNot asked in the question	`22.01381` match in 3 of 3 runsNot asked in the question	`22.01381` matchNot asked in the question
Adjusted hazard ratio, stage IVPrinted in the paper	`74.45`	± 0.2	`74.45261` match in 3 of 3 runsNot asked in the question	`74.45261` match in 3 of 3 runsNot asked in the question	`74.45261` match in 3 of 3 runsNot asked in the question	`74.45261` matchNot asked in the question
Adjusted hazard ratio, diffuse typePrinted in the paper	`2.232`	± 0.005	`2.232204` match in 3 of 3 runsNot asked in the question	`2.232204` match in 3 of 3 runsNot asked in the question	`2.232204` match in 3 of 3 runsNot asked in the question	`2.232204` matchNot asked in the question
Hazard ratio, high MMP-14, intestinal typePrinted in the paper	`3.542`	± 0.005	`3.542426` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.542426` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.542426` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.598528` no matchIn the final answer: no
95% CI lower bound, intestinal typePrinted in the paper	`1.507`	± 0.005	`1.50659` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.50659` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.50659` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.501023` no matchIn the final answer: no
95% CI upper bound, intestinal typePrinted in the paper	`8.329`	± 0.02	`8.329265` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`8.329265` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`8.329265` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7.976084` no matchIn the final answer: no
P for high MMP-14, intestinal typePrinted in the paper	`0.00374`	± 0.0005	`0.003737309` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.003737309` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.003737309` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.001672369` no matchIn the final answer: no
Hazard ratio, high MMP-14, diffuse typePrinted in the paper	`1.5355`	± 0.005	`1.535523` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.535523` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.535523` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.548716` no matchIn the final answer: no
P for high MMP-14, diffuse typePrinted in the paper	`0.0711`	± 0.001	`0.07105189` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.07105189` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.07105189` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.05` no matchIn the final answer: no

Open the runs

Research paper

Kochanek 2023

Kochanek KD, Murphy SL, Xu JQ, et al.

Deaths: Final data for 2020. National Vital Statistics Reports 72(10). Hyattsville, MD, National Center for Health Statistics (2023). doi:10.15620/cdc:131355

Deaths, final data for 2020 (United States)

epitools (R), through the epi-rates adapter

No run yetno result file has this paper

No result file has this paper. It is not in the totals.

Show each value

Known values and run values for Kochanek 2023, earlier run of 9 October 2026
Value	Known value	Tolerance	Haiku
Age-adjusted death rate, all causes, 2020, per 100,000Printed in the paper. Independent check: yes	`835.4`	± 0.1	not in this run
Age-adjusted death rate, all causes, 2019, per 100,000Printed in the paper. Independent check: yes	`715.2`	± 0.1	not in this run
Age-adjusted death rate, heart disease, 2020, per 100,000Printed in the paper. Independent check: yes	`168.2`	± 0.1	not in this run
Age-adjusted death rate, males, 2020, per 100,000Printed in the paper. Independent check: yes	`998.3`	± 0.1	not in this run
Age-adjusted death rate, females, 2020, per 100,000Printed in the paper. Independent check: yes	`695.1`	± 0.1	not in this run
Ratio of the age-adjusted rate of 2020 to the rate of 2019We calculated it with ratio of two printed age-adjusted rates (835.4 and 715.2). Independent check: yes	`1.168`	± 0.003	not in this run
Ratio of the age-adjusted male rate to the female rateWe calculated it with ratio of two printed age-adjusted rates (998.3 and 695.1). Independent check: yes	`1.436`	± 0.003	not in this run

Open the runs

Research paper

Lackey 2015

Lackey B, Seas C, Van der Stuyft P, et al.

Patient characteristics associated with tuberculosis treatment default: a cohort study in a high-incidence area of Lima, Peru. PLOS ONE 10(6):e0128541 (2015). doi:10.1371/journal.pone.0128541

risk factors for tuberculosis treatment default in Lima, Peru

statsmodels and SciPy (Python), through the biostats adapter

Opus30 of 30 values match, in each of 3 runs

30 of 30

In the final answer: 28 of 28, in each of 3 runs

Sonnet30 of 30 values match, in each of 3 runs

30 of 30

In the final answer: 28 of 28, in each of 3 runs

Haiku30 of 30 values match, in each of 3 runs

30 of 30

In the final answer: 27 to 28 of 28, over 3 runs

qwen3:8b28 of 30 values match

28 of 30

In the final answer: 23 of 28

Show each value

Known values and run values for Lackey 2015, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Patients in the analysisPrinted in the paper	`1233`	exact	`1233` match in 3 of 3 runsNot asked in the question	`1233` match in 3 of 3 runsNot asked in the question	`1233` match in 3 of 3 runsNot asked in the question	`1233` matchNot asked in the question
Patients who defaultedPrinted in the paper	`127`	exact	`127` match in 3 of 3 runsNot asked in the question	`127` match in 3 of 3 runsNot asked in the question	`127` match in 3 of 3 runsNot asked in the question	`127` matchNot asked in the question
Chi-square p, age group by defaultPrinted in the paper	`0.00604`	± 0.0002	`0.006038988` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.006038988` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.006038988` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.007700489` no matchIn the final answer: no
Chi-square p, sex by defaultPrinted in the paper	`3.95e-05`	± 5e-06	`3.945092e-05` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.945092e-05` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.945092e-05` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.97635e-05` matchIn the final answer: no
Chi-square p, marital status by defaultPrinted in the paper	`0.6815`	± 0.001	`0.6814545` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6814545` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6814545` match in 3 of 3 runsIn the final answer: yes in 1 of 3 runs	`0.707159` no matchIn the final answer: no
Chi-square p, poverty by defaultPrinted in the paper	`0.03436`	± 0.0005	`0.03436349` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.03436349` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.03436349` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.03445384` matchIn the final answer: no
Adjusted OR, drug usePrinted in the paper	`4.7816`	± 0.005	`4.781551` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`4.781551` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`4.781551` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`4.781551` matchIn the final answer: yes
Adjusted OR, drug use, CI lowerPrinted in the paper	`3.0522`	± 0.005	`3.052178` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.052178` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.052178` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.052178` matchIn the final answer: yes
Adjusted OR, drug use, CI upperPrinted in the paper	`7.4907`	± 0.005	`7.490792` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7.490792` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7.490792` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`7.490792` matchIn the final answer: yes
Adjusted OR, underweightPrinted in the paper	`2.0795`	± 0.005	`2.079503` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.079503` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.079503` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.079503` matchIn the final answer: yes
Adjusted OR, underweight, CI lowerPrinted in the paper	`1.2137`	± 0.005	`1.213699` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.213699` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.213699` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.213699` matchIn the final answer: yes
Adjusted OR, underweight, CI upperPrinted in the paper	`3.5629`	± 0.005	`3.562935` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.562935` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.562935` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.562935` matchIn the final answer: yes
Adjusted OR, overweight or obesePrinted in the paper	`0.8777`	± 0.005	`0.8777123` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.8777123` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.8777123` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.8777123` matchIn the final answer: yes
Adjusted OR, overweight or obese, CI lowerPrinted in the paper	`0.4445`	± 0.005	`0.4444368` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4444368` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4444368` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4444368` matchIn the final answer: yes
Adjusted OR, overweight or obese, CI upperPrinted in the paper	`1.7333`	± 0.005	`1.733382` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.733382` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.733382` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.733382` matchIn the final answer: yes
Adjusted OR, MDR treatmentPrinted in the paper	`3.0385`	± 0.005	`3.038484` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.038484` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.038484` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.038484` matchIn the final answer: yes
Adjusted OR, MDR treatment, CI lowerPrinted in the paper	`1.578`	± 0.005	`1.577974` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.577974` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.577974` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.577974` matchIn the final answer: yes
Adjusted OR, MDR treatment, CI upperPrinted in the paper	`5.8507`	± 0.005	`5.850784` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`5.850784` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`5.850784` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`5.850784` matchIn the final answer: yes
Adjusted OR, weekly alcoholPrinted in the paper	`2.222`	± 0.005	`2.221989` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.221989` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.221989` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.221989` matchIn the final answer: yes
Adjusted OR, weekly alcohol, CI lowerPrinted in the paper	`1.4008`	± 0.005	`1.400785` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.400785` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.400785` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.400785` matchIn the final answer: yes
Adjusted OR, weekly alcohol, CI upperPrinted in the paper	`3.5246`	± 0.005	`3.52462` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.52462` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.52462` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.52462` matchIn the final answer: no
Adjusted OR, HIV test not donePrinted in the paper	`2.3031`	± 0.005	`2.303112` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.303112` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.303112` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.303112` matchIn the final answer: yes
Adjusted OR, HIV test not done, CI lowerPrinted in the paper	`1.5`	± 0.005	`1.499949` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.499949` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.499949` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.499949` matchIn the final answer: yes
Adjusted OR, HIV test not done, CI upperPrinted in the paper	`3.5363`	± 0.005	`3.536336` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.536336` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.536336` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`3.536336` matchIn the final answer: yes
Adjusted OR, HIV positivePrinted in the paper	`1.3925`	± 0.005	`1.392486` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.392486` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.392486` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.392486` matchIn the final answer: yes
Adjusted OR, HIV positive, CI lowerPrinted in the paper	`0.4167`	± 0.005	`0.4166728` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4166728` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4166728` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.4166728` matchIn the final answer: yes
Adjusted OR, HIV positive, CI upperPrinted in the paper	`4.6534`	± 0.005	`4.653571` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`4.653571` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`4.653571` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`4.653571` matchIn the final answer: yes
Adjusted OR, secondary school not completedPrinted in the paper	`1.5499`	± 0.005	`1.549942` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.549942` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.549942` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.549942` matchIn the final answer: yes
Adjusted OR, secondary school not completed, CI lowerPrinted in the paper	`1.0317`	± 0.005	`1.031655` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.031655` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.031655` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.031655` matchIn the final answer: yes
Adjusted OR, secondary school not completed, CI upperPrinted in the paper	`2.3286`	± 0.005	`2.32861` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.32861` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.32861` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.32861` matchIn the final answer: yes

Open the runs

Research paper

Sriphoosanaphan 2021

Sriphoosanaphan S, Thanapirom K, Kerr SJ, et al.

Effect of vitamin D supplementation in patients with chronic hepatitis C after direct-acting antiviral treatment: a randomized, double-blind, placebo-controlled trial. PeerJ 9:e10709 (2021). doi:10.7717/peerj.10709

Data: doi:10.5061/dryad.573n5tb4h

vitamin D and liver fibrosis markers after hepatitis C cure, a randomized trial

SciPy (Python), through the biostats adapter

Opus21 of 21 values match, in each of 3 runs

21 of 21

In the final answer: 19 of 19, in each of 3 runs

Sonnet21 of 21 values match, in each of 3 runs

21 of 21

In the final answer: 19 of 19, in each of 3 runs

Haiku21 of 21 values match, in each of 3 runs

21 of 21

In the final answer: 19 of 19, in each of 3 runs

qwen3:8b5 of 21 values match

5 of 21

In the final answer: 3 of 19

Show each value

Known values and run values for Sriphoosanaphan 2021, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Patients in the vitamin D armPrinted in the paper	`37`	exact	`37` match in 3 of 3 runsNot asked in the question	`37` match in 3 of 3 runsNot asked in the question	`37` match in 3 of 3 runsNot asked in the question	`37` matchNot asked in the question
Patients in the placebo armPrinted in the paper	`38`	exact	`38` match in 3 of 3 runsNot asked in the question	`38` match in 3 of 3 runsNot asked in the question	`38` match in 3 of 3 runsNot asked in the question	`38` matchNot asked in the question
25(OH)D change, vitamin D minus placebo, ng/mLPrinted in the paper	`23.086`	± 0.01	`23.086` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`23.08642` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`23.0864` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`23.08642` matchIn the final answer: yes
25(OH)D, 95% CI lower boundPrinted in the paper	`19.734`	± 0.01	`19.73379` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`19.73379` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`19.73379` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`19.73379` matchIn the final answer: yes
25(OH)D, 95% CI upper boundPrinted in the paper	`26.439`	± 0.01	`26.43904` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`26.43904` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`26.43904` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`26.43904` matchIn the final answer: yes
TGF-beta1 change, vitamin D minus placebo, ng/mLPrinted in the paper	`-0.554`	± 0.005	`-0.554` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-0.5542817` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-0.5542817` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0` no matchIn the final answer: no
TGF-beta1, 95% CI lower boundPrinted in the paper	`-2.839`	± 0.005	`-2.839268` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-2.839268` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-2.839268` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0` no matchIn the final answer: no
TGF-beta1, 95% CI upper boundPrinted in the paper	`1.731`	± 0.005	`1.730704` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.730704` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.730704` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.497368` no matchIn the final answer: no
TGF-beta1, pPrinted in the paper	`0.6302`	± 0.001	`0.6302216` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6302216` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6302216` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.64` no matchIn the final answer: no
TIMP-1 change, vitamin D minus placebo, ng/mLPrinted in the paper	`-5.51`	± 0.01	`-5.51` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-5.510156` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-5.510156` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0` no matchIn the final answer: no
TIMP-1, 95% CI lower boundPrinted in the paper	`-26.361`	± 0.01	`-26.36133` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-26.36133` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-26.36133` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0` no matchIn the final answer: no
TIMP-1, 95% CI upper boundPrinted in the paper	`15.341`	± 0.01	`15.34101` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`15.34101` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`15.34101` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`16.38` no matchIn the final answer: no
TIMP-1, pPrinted in the paper	`0.6`	± 0.001	`0.6` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6000182` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.6` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.64` no matchIn the final answer: no
MMP-9 change, vitamin D minus placebo, ng/mLPrinted in the paper	`122.92`	± 0.05	`122.9206` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`122.9206` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`122.9206` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`103` no matchIn the final answer: no
MMP-9, 95% CI lower boundPrinted in the paper	`-68.96`	± 0.05	`-68.963` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-68.96466` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-68.9633` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0` no matchIn the final answer: no
MMP-9, 95% CI upper boundPrinted in the paper	`314.81`	± 0.05	`314.8059` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`314.8059` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`314.8059` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`357` no matchIn the final answer: no
MMP-9, pPrinted in the paper	`0.2058`	± 0.001	`0.2057536` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.2057536` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.2057536` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.14` no matchIn the final answer: no
P3NP change, vitamin D minus placebo, ng/mLPrinted in the paper	`-0.11`	± 0.005	`-0.11` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-0.1103912` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-0.1103912` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0` no matchIn the final answer: no
P3NP, 95% CI lower boundPrinted in the paper	`-2.372`	± 0.005	`-2.371921` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-2.371921` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`-2.371921` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0` no matchIn the final answer: no
P3NP, 95% CI upper boundPrinted in the paper	`2.151`	± 0.005	`2.151139` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.151139` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.151139` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2` no matchIn the final answer: no
P3NP, pPrinted in the paper	`0.9228`	± 0.001	`0.9227679` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9227679` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.9227679` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1` no matchIn the final answer: no

Open the runs

Research paper

Sriphoosanaphan 2021 sample size

Sriphoosanaphan S, Thanapirom K, Kerr SJ, et al.

the sample size calculation of a vitamin D trial in chronic hepatitis C

pwr (R), through the pwr adapter

Opus3 of 3 values match, in each of 3 runs

3 of 3

In the final answer: 2 of 2, in each of 3 runs

Sonnet3 of 3 values match, in each of 3 runs

3 of 3

In the final answer: 2 of 2, in each of 3 runs

Haiku3 of 3 values match, in each of 3 runs

3 of 3

In the final answer: 2 of 2, in each of 3 runs

qwen3:8b3 of 3 values match

3 of 3

In the final answer: 2 of 2

Show each value

Known values and run values for Sriphoosanaphan 2021 sample size, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Sample size per arm before roundingWe calculated it with pwr.t.test (R pwr 1.3.0). Independent check: yes	`29.94`	± 0.01	`29.94159` match in 3 of 3 runsNot asked in the question	`29.94159` match in 3 of 3 runsNot asked in the question	`29.94159` match in 3 of 3 runsNot asked in the question	`29.94159` matchNot asked in the question
Patients per arm, rounded upPrinted in the paper	`30`	exact	`30` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`30` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`30` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`30` matchIn the final answer: yes
Patients in totalPrinted in the paper	`60`	exact	`60` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`60` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`60` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`60` matchIn the final answer: yes

Open the runs

Ecology

Tutorial or software test data

Oksanen 2026

Oksanen J, Simpson GL, Blanchet FG, et al.

vegan: Community Ecology Package, version 2.7-6. R package on CRAN (2026). doi:10.32614/CRAN.package.vegan

vegan, the dune meadow data

vegan (R), through the vegan adapter

Opus6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 5 of 5, in each of 3 runs

Sonnet6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 5 of 5, in each of 3 runs

Haiku6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 5 of 5, in each of 3 runs

qwen3:8b6 of 6 values match

6 of 6

In the final answer: 5 of 5

Show each value

Known values and run values for Oksanen 2026, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Shannon index of site 1, natural logWe calculated it with vegan 2.7.6. Independent check: yes	`1.4405`	± 0.001	`1.440482` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.440482` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.440482` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`1.440482` matchIn the final answer: yes
NMDS stress, Bray-Curtis, k=2Printed in the official tutorial	`0.1183`	± 0.001	`0.1183186` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1183186` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1183186` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.1183186` matchIn the final answer: yes
PERMANOVA R2 of ManagementWe calculated it with vegan 2.7.6 adonis2. Independent check: yes	`0.3416`	± 0.001	`0.3416107` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.3416107` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.3416107` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.3416107` matchIn the final answer: yes
PERMANOVA F of ManagementWe calculated it with vegan 2.7.6 adonis2. Independent check: yes	`2.767`	± 0.01	`2.767243` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.767243` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.767243` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.767243` matchIn the final answer: yes
PERMANOVA p of Management, 999 permutationsWe calculated it with vegan 2.7.6 adonis2, 999 permutations. Independent check: yes	`0.003`	± 0.003	`0.002` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.002` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.003` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`0.005` matchIn the final answer: yes
Dispersion test F of ManagementWe calculated it with vegan 2.7.6 betadisper and permutest. Independent check: yes	`1.951`	± 0.01	`1.95064` match in 3 of 3 runsNot asked in the question	`1.95064` match in 3 of 3 runsNot asked in the question	`1.95064` match in 3 of 3 runsNot asked in the question	`1.95064` matchNot asked in the question

Open the runs

Astronomy

Tutorial or software test data

Bradley 2026

Bradley L, Sipőcz BM, Robitaille TP, et al.

Photutils 3.0.0. Zenodo, software release (2026). doi:10.5281/zenodo.19636730

photutils, and the Astropy Collaboration papers (2013, 2018, 2022)

astropy and photutils (Python), through the astropy adapter

Opus6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 6 of 6, in each of 3 runs

Sonnet6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 6 of 6, in each of 3 runs

Haiku6 of 6 values match, in each of 3 runs

6 of 6

In the final answer: 3 to 6 of 6, over 3 runs

qwen3:8b6 of 6 values match

6 of 6

In the final answer: 6 of 6

Show each value

Known values and run values for Bradley 2026, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Sky level (clipped median).Printed in the official tutorial	`5.152`	± 0.01	`5.1524` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`5.1524` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`5.1524` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`5.152442` matchIn the final answer: yes
Sky noise (clipped standard deviation).Printed in the official tutorial	`2.094`	± 0.01	`2.093725` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.093725` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.093725` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`2.094237` matchIn the final answer: yes
Sources detected.We calculated it with photutils 3.0.0 with astropy 8.0.1, checked with a second method in SciPy only. Independent check: yes	`82`	exact	`82` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`82` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`82` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`82` matchIn the final answer: yes
Net flux at (48.70, 200.33).We calculated it with photutils 3.0.0, table of the injected sources of make_100gaussians_image. Independent check: yes	`895.4`	± 106	`925.2357` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`925.2357` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`925.2357` match in 3 of 3 runsIn the final answer: yes in 2 of 3 runs	`925.2357` matchIn the final answer: yes
Net flux at (145.05, 168.52).We calculated it with photutils 3.0.0, table of the injected sources of make_100gaussians_image. Independent check: yes	`734.1`	± 106	`760.1982` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`760.1982` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`760.1982` match in 3 of 3 runsIn the final answer: yes in 2 of 3 runs	`760.1982` matchIn the final answer: yes
Net flux at (355.56, 252.20).We calculated it with photutils 3.0.0, table of the injected sources of make_100gaussians_image. Independent check: yes	`873.9`	± 106	`925.2357` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`925.2357` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`925.2357` match in 3 of 3 runsIn the final answer: yes in 2 of 3 runs	`925.2357` matchIn the final answer: yes

Open the runs

Geospatial

Tutorial or software test data

Rouault 2026

Rouault E, Warmerdam F, Schwehr K, et al.

GDAL. Zenodo software record, version 3.13.3 (2026). doi:10.5281/zenodo.5884351

GDAL and OGR

GDAL (command-line programs), through the gdal adapter

Opus14 of 14 values match, in each of 3 runs

14 of 14

In the final answer: 14 of 14, in each of 3 runs

Sonnet14 of 14 values match, in each of 3 runs

14 of 14

In the final answer: 14 of 14, in each of 3 runs

Haiku14 of 14 values match, in each of 3 runs

14 of 14

In the final answer: 14 of 14, in each of 3 runs

qwen3:8b8 of 14 values match

8 of 14

In the final answer: 8 of 14

Show each value

Known values and run values for Rouault 2026, final run of 9 October 2026
Value	Known value	Tolerance	Opus	Sonnet	Haiku	qwen3:8b
Raster width in pixelsPrinted in the official tutorial	`20`	exact	`20` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`20` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`20` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`20` matchIn the final answer: yes
Raster height in pixelsPrinted in the official tutorial	`20`	exact	`20` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`20` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`20` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`20` matchIn the final answer: yes
EPSG code of the rasterPrinted in the official tutorial	`26711`	exact	`26711` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`26711` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`26711` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`26711` matchIn the final answer: yes
Pixel size in metresPrinted in the official tutorial	`60`	exact	`60` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`60` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`60` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`60` matchIn the final answer: yes
Band minimumPrinted in the official tutorial	`74`	exact	`74` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`74` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`74` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`74` matchIn the final answer: yes
Band maximumPrinted in the official tutorial	`255`	exact	`255` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`255` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`255` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`255` matchIn the final answer: yes
Band meanPrinted in the official tutorial	`126.765`	± 0.01	`126.765` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`126.765` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`126.765` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`126.765` matchIn the final answer: yes
Band standard deviationPrinted in the official tutorial	`22.93`	± 0.05	`22.928` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`22.928` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`22.928` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`22.928` matchIn the final answer: yes
Width after warp to EPSG:4326We calculated it with GDAL 3.13.3 gdalwarp, checked with pyproj. Independent check: yes	`22`	exact	`22` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`22` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`22` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`22.928` no matchIn the final answer: no
Height after warp to EPSG:4326We calculated it with GDAL 3.13.3 gdalwarp, checked with pyproj. Independent check: yes	`18`	exact	`18` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`18` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`18` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`20` no matchIn the final answer: no
Zone A pixel countWe calculated it with numpy and shapely. Independent check: yes	`100`	exact	`100` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`100` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`100` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`74` no matchIn the final answer: no
Zone A meanWe calculated it with numpy and shapely. Independent check: yes	`129.5`	± 0.01	`129.5` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`129.5` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`129.5` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`126.765` no matchIn the final answer: no
Zone B pixel count, pixel centersWe calculated it with numpy and shapely. Independent check: yes	`42`	exact	`42` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`42` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`42` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`60` no matchIn the final answer: no
Zone B mean, pixel centersWe calculated it with numpy and shapely. Independent check: yes	`135`	± 0.01	`135` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`135` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`135` match in 3 of 3 runsIn the final answer: yes in 3 of 3 runs	`126.765` no matchIn the final answer: no

Open the runs

Sources: the result files final-2026-10-09-*.json of the final run and papers-2026-10-09-*.json of the earlier Haiku runs, and the case files in bench/papers/<name>/ of the Cuvette repository. The known values come from bench.yaml and case.yaml of each paper. See the source of each known value and the method.