Validation / Papers / Aulicino 2022

Highly efficient CRISPR-mediated large DNA docking and multiplexed prime editing using a single baculovirus

Flow cytometry · research paper · flowCore (R), through the flowcore adapter

How to read this page. In this validation, a script plays the scientist. It gives the answers that we wrote before the run, from the methods of the paper. A known value comes from the paper, from a tutorial or from a check that we ran. This page has no combined run of the paper yet.

No run is scored for this paper yet.

The paper

Aulicino F et al. Highly efficient CRISPR-mediated large DNA docking and multiplexed prime editing using a single baculovirus. Nucleic Acids Research (2022). doi:10.1093/nar/gkac587

Related sources:

Data: raw flow cytometry files of the paper on figshare, CC BY 4.0. fetch.sh downloads the files of Fig 2b for day 10. doi:10.6084/m9.figshare.20110364

What it measured

The study delivers a large DNA insert with an mCherry reporter into HEK293T cells with a baculovirus. It compares the homology-directed repair design (HDR) with the HITI-2c design. Flow cytometry on day 10 counts the mCherry-positive cells. The Results text gives the absolute editing as about 5% for HDR and about 20% for HITI-2c.

Data

figshare record 10.6084/m9.figshare.20110364, the files Fig2b_BV_HEK_HDR_10d_rep001 to 003, Fig2b_BV_HEK_HITI-2c_10d_rep001 to 003 and Fig2b_plasmid_control_HEK_untransfected_d10_rep001 to 003. fetch.sh writes them with short names. Size: 9 FCS files, 19 MB, 12000 to 20000 events each.

License: CC BY 4.0, from the figshare record. The paper is CC BY 4.0.

Data source

The instruction

A script sends this message as the scientist.

ScientistSet the mCherry-positive cutoff from the untransfected control and give the mean percent positive of each design with its SD.

Basis: Results (Fig 2B text) and the figure legend. The paper does not state the exact cutoff, so the percentile of the control is a decision of the scientist.

The decisions

The model asks questions during a run. A script gives these answers to the questions of the model. We wrote the answers before the run.

**Table 1 |** Answers that a script gives to the questions of the model.
Decision	Value	Source
Percentile of the control for the cutoff	`99.9`	Not in the paper. The 99.9th percentile of the pooled untransfected control gives 4.4% and 20.9%, which agree with the printed about 5% and about 20%. The 99th percentile gives 6.6% and 25.9%.
Parent gate on scatter	`none`	Not in the paper. The deposited files carry no gate.
Compensation of the samples	`none`	The reporter is read in one channel. The deposited files carry the identity matrix.
Transform of the fluorescence channels	`none`	The cutoff is a percentile of the control, so the percent positive does not depend on a monotonic transform.
Cofactor of the asinh transform	`150`	Not used.
Decades of a log-amplified scale	`0`	The LSRFortessa writes linear values.
Cutoff for positive events in a control	`99`	Not used. The benchmark makes no spillover matrix.
Cutoff between negative and positive	`none`	The cutoff comes from the control file.
Statistical test for the groups	`welch`	Not in the paper. Welch is the default of R t.test.
Correction for many comparisons	`none`	One planned comparison.
Other questions of the agent	`Use the values in the decision record.`	Not in the paper. The benchmark answers each free question with this text, so that the record of decisions stays the only source of the settings.

Known values

The tolerance is the largest difference from the known value that we accept. We set it before the run. Exact: the number must be the same.

**Table 2 |** Known values for Aulicino 2022.
Value	Known value	Tolerance	Source
`hdr_pct_printed`Mean percent of mCherry-positive cells after the HDR design, day 10 (the paper prints about 5%) Source of the known value Printed in the paperWhere: Results, Fig 2B text: absolute editing of about 5% for HDR in HEK293T cells. The value is approximate in the paper.Check: check.py gives 5 or close to it from the data (check.out).Note in the list of known values: paper, Results	5	± 1	Printed in the paper
`hiti_pct_printed`Mean percent of mCherry-positive cells after the HITI-2c design, day 10 (the paper prints about 20%) Source of the known value Printed in the paperWhere: Results, Fig 2B text: absolute editing of about 20% for HITI-2c in HEK293T cells. The value is approximate in the paper.Check: check.py gives 20 or close to it from the data (check.out).Note in the list of known values: paper, Results	20	± 1.5	Printed in the paper
`control_cutoff`mCherry cutoff: 99.9th percentile of the pooled untransfected control events, PE-CF594-A Source of the known value Independent check: we calculated itTool: population_stats of the flowcore adapterWhere: Not printed. The paper gives no cutoff.Check: check.py (SciPy, fcsparser) gives the same value (check.out).Note in the list of known values: computed by check.py	3791.17	± 0.5	Independent check: we calculated it
`hdr_pct_mean`Mean percent mCherry-positive, HDR, three replicates Source of the known value Independent check: we calculated itTool: compare_groups of the flowcore adapterWhere: Not printed as an exact value.Check: check.py (SciPy, fcsparser) gives the same value (check.out).Note in the list of known values: computed by check.py	4.419	± 0.01	Independent check: we calculated it
`hdr_pct_sd`SD of the percent mCherry-positive, HDR Source of the known value Independent check: we calculated itTool: compare_groups of the flowcore adapterWhere: Not printed.Check: check.py (SciPy, fcsparser) gives the same value (check.out).Note in the list of known values: computed by check.py	0.146	± 0.01	Independent check: we calculated it
`hiti_pct_mean`Mean percent mCherry-positive, HITI-2c, three replicates Source of the known value Independent check: we calculated itTool: compare_groups of the flowcore adapterWhere: Not printed as an exact value.Check: check.py (SciPy, fcsparser) gives the same value (check.out).Note in the list of known values: computed by check.py	20.903	± 0.01	Independent check: we calculated it
`hiti_pct_sd`SD of the percent mCherry-positive, HITI-2c Source of the known value Independent check: we calculated itTool: compare_groups of the flowcore adapterWhere: Not printed.Check: check.py (SciPy, fcsparser) gives the same value (check.out).Note in the list of known values: computed by check.py	2.256	± 0.02	Independent check: we calculated it
`welch_p`Welch t test p value, HITI-2c against HDR Source of the known value Independent check: we calculated itTool: compare_groups of the flowcore adapterWhere: Not printed. The paper gives no test for this comparison.Check: check.py (SciPy, fcsparser) gives the same value (check.out).Note in the list of known values: computed by check.py	0.006018	± 0.0001	Independent check: we calculated it
`hdr_pct_mean_p99`Trap: mean percent mCherry-positive, HDR, with the 99th percentile as the cutoff Source of the known value Independent check: we calculated itTool: population_stats of the flowcore adapterWhere: Not in the paper. The result of a cutoff at the 99th percentile of the control.Check: check.py gives the value (check.out).Note in the list of known values: computed by check.py	6.623	± 0.01	Independent check: we calculated it
`hiti_pct_mean_p99`Trap: mean percent mCherry-positive, HITI-2c, with the 99th percentile as the cutoff Source of the known value Independent check: we calculated itTool: population_stats of the flowcore adapterWhere: Not in the paper. The result of a cutoff at the 99th percentile of the control.Check: check.py gives the value (check.out).Note in the list of known values: computed by check.py	25.945	± 0.01	Independent check: we calculated it

Latest scored run

No run is scored for this paper yet.

Notes

Triage notes by the maintainers

The text below is from the triage notes. We show it as the maintainers wrote it.

Classes: a = tool or adapter fault, b = harness fault, c = benchmark spec fault, d = model fault.

claude-haiku-5-5 run 1 (blind, before the fixes): 20261009-040024-8040, computed 5/8, reported 5/8.
claude-haiku-5-5 run 2 (blind, after the first fix): 20261009-040250-ac15, computed 7/8, reported 8/8.
claude-haiku-5-5 run 3 (blind, final): computed 8/8, reported 8/8, 102 s, 0 paths outside the allow list, 0 blocked reads, 0 unsourced claims.

Run	Item	Expected	Got	Class	Cause	Fix
1	control_cutoff, hdr_pct_mean, hiti_pct_mean	3791.17, 4.419, 20.903	3762.9, 4.431, 20.945	a	The description of `control` in `population_stats` said "a control FCS file". The model used control replicate 1 alone, then tried replicates 2 and 3 one by one, and asked the scientist which one to use.	`control` takes a list of files and pools their events (flowcore 0.1.0). The description says so.
2	welch_p	0.006018	5.4e-06	a	The model passed the marker PE-CF594-A and the population PE-CF594-A+. The tool wrote the same population twice, so each sample counted twice in `compare_groups`.	`population_stats` skips a population that has the name of a marker plus "+". `compare_groups` stops if a sample appears twice for one population. `list_files` drops a repeated path. Test `population-named-like-marker`.

Other findings:

The paper prints about 5% and about 20%. The tolerances of the two published items are 1.0 and 1.5 percentage points. The cutoff at the 99.9th percentile of the pooled control gives 4.42% and 20.90%. The 99th percentile gives 6.62% and 25.95%, which the paper does not support.
The model wrote the design file with run_script. The tool compare_groups needs a file with the columns sample and group.