diff --git a/CHANGELOG.md b/CHANGELOG.md
index 024c92a..31bc963 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -95,6 +95,7 @@ directly comparable with 1.0.1.
- **A drawing on every page that explains something**: the duplication family and communities of CDC1551 and M. bovis, from a real run of both duplication steps; the sample sheet with its tabs and the messages the validator prints; the ranges a genome is validated against; the container and what it holds; a run on a cluster; what happens to a task that fails; the report section by section; the five outcomes and the three inputs of a decision; the command builder on the quickstart. Most are built from the code or the config, so they change when it does
- **The benchmark on the front page**: the seven genomes whose answer is known, and how many of their genes AMAP finds, checked against `tests/benchmark/README.md`. A shared link to any page shows a card drawn from the front page's genes
- **More that the docs got wrong**: the duplication page listed ten network metrics, the table has four, and its columns ending in 1 and 2 are the two communities of a pair, not two levels; Nextflow needs Java 17, not 11; `--skip_functional_annotation` leaves no GFF3 or GenBank, which come after eggNOG; tasks are capped at 16 GB on one machine and at 128 GB under a scheduler profile, not 50 GB; and lineage typing is not skipped on its own for other species, only Mycobrowser is
+- **The tutorial's numbers are today's**: its case counts and QC line, and their copies on the front page, the outputs page and the custom-reference page, came from runs in July. They now come from the tutorial's own commands run again on 18 October 2026 with `--skip_functional_annotation`, the QC line from GET_OUTPUTS' own script run on that run's corrected genes: CDC1551 has 4014 genes and 123 pseudogenes as AMAP calls them, where the old line counted Mycobrowser's 13, and Sakai keeps 1387 genes K-12 lacks, not 1356. The tutorial also says where `--validate_only` writes its verdict and where the number of reference proteins extracted is, as neither reaches the console
- **The README in the site's style**: the metro map and the five outcomes in light and dark, the benchmark's scores, and the links a newcomer needs first
- **The README opens with the logo, and its drawing plays**: the picture at the top was a still of the front page's drawing, dark whatever GitHub's theme. It now plays the drawing's four steps, in light or dark as the theme is, made by `tools/pictures.mjs` from the front page's own figure. The README's metro map gains the interactive report's station
- **One DOI**: the README, CITATION.cff and every page of the website cited the DOI of one version on Zenodo, and the manifest, `main.nf` and the interactive report the concept DOI, which always resolves to the latest version. All of them cite the concept DOI now
diff --git a/docs/about/changelog.html b/docs/about/changelog.html
index 385eb27..9aad5bc 100644
--- a/docs/about/changelog.html
+++ b/docs/about/changelog.html
@@ -102,7 +102,7 @@
reconcile_coordinates is gone: it read reference coordinates from protein identifiers that have not carried them since March 2026, so it changed nothing. The paralog check does its job with the sample's own offsets
A website in plain HTML, drawn with real genes: MkDocs is gone. The front page starts a newcomer in three steps and builds the command for their machine, the pipeline is a metro map whose stations are main.nf's processes, and the four cases, the missing genes and the harder cases (paralogs, small proteins, contig ends, frameshifts, copies) are drawn on genes of CDC1551, M. bovis and E. coli from the benchmark. The tutorial and user guide stay markdown, readable on GitHub, and tools/build_docs.py turns them into pages of the same site, dark by default
The site is checked against the code: tools/check_site.py runs every drawn window through bin/, compares the thresholds on the pages with the constants in the code, the metro map with main.nf and every option with the pipeline's parameters, so a change that makes the site wrong fails the docs workflow. tools/figure_data.py redraws the windows from a benchmark run
What the docs said that the pipeline does not do: with --skip_functional_annotation a run still writes AMAP/, with the corrected table and sequences, where the outputs page said it did not; no step writes PATHOTYPR/renamed_fastas.tsv; and the 19-residue toxin the algorithm page names is IbsC (b4665), not an ldr toxin
A drawing on every page that explains something: the duplication family and communities of CDC1551 and M. bovis, from a real run of both duplication steps; the sample sheet with its tabs and the messages the validator prints; the ranges a genome is validated against; the container and what it holds; a run on a cluster; what happens to a task that fails; the report section by section; the five outcomes and the three inputs of a decision; the command builder on the quickstart. Most are built from the code or the config, so they change when it does
The benchmark on the front page: the seven genomes whose answer is known, and how many of their genes AMAP finds, checked against tests/benchmark/README.md. A shared link to any page shows a card drawn from the front page's genes
More that the docs got wrong: the duplication page listed ten network metrics, the table has four, and its columns ending in 1 and 2 are the two communities of a pair, not two levels; Nextflow needs Java 17, not 11; --skip_functional_annotation leaves no GFF3 or GenBank, which come after eggNOG; tasks are capped at 16 GB on one machine and at 128 GB under a scheduler profile, not 50 GB; and lineage typing is not skipped on its own for other species, only Mycobrowser is
The README in the site's style: the metro map and the five outcomes in light and dark, the benchmark's scores, and the links a newcomer needs first
The README opens with the logo, and its drawing plays: the picture at the top was a still of the front page's drawing, dark whatever GitHub's theme. It now plays the drawing's four steps, in light or dark as the theme is, made by tools/pictures.mjs from the front page's own figure. The README's metro map gains the interactive report's station
One DOI: the README, CITATION.cff and every page of the website cited the DOI of one version on Zenodo, and the manifest, main.nf and the interactive report the concept DOI, which always resolves to the latest version. All of them cite the concept DOI now
Bakta's options on a SLURM cluster: the HPC page says how to give Bakta a site's options, since for Bakta the QoS the slurm profile sets takes the place of -process.clusterOptions
+
A website in plain HTML, drawn with real genes: MkDocs is gone. The front page starts a newcomer in three steps and builds the command for their machine, the pipeline is a metro map whose stations are main.nf's processes, and the four cases, the missing genes and the harder cases (paralogs, small proteins, contig ends, frameshifts, copies) are drawn on genes of CDC1551, M. bovis and E. coli from the benchmark. The tutorial and user guide stay markdown, readable on GitHub, and tools/build_docs.py turns them into pages of the same site, dark by default
The site is checked against the code: tools/check_site.py runs every drawn window through bin/, compares the thresholds on the pages with the constants in the code, the metro map with main.nf and every option with the pipeline's parameters, so a change that makes the site wrong fails the docs workflow. tools/figure_data.py redraws the windows from a benchmark run
What the docs said that the pipeline does not do: with --skip_functional_annotation a run still writes AMAP/, with the corrected table and sequences, where the outputs page said it did not; no step writes PATHOTYPR/renamed_fastas.tsv; and the 19-residue toxin the algorithm page names is IbsC (b4665), not an ldr toxin
A drawing on every page that explains something: the duplication family and communities of CDC1551 and M. bovis, from a real run of both duplication steps; the sample sheet with its tabs and the messages the validator prints; the ranges a genome is validated against; the container and what it holds; a run on a cluster; what happens to a task that fails; the report section by section; the five outcomes and the three inputs of a decision; the command builder on the quickstart. Most are built from the code or the config, so they change when it does
The benchmark on the front page: the seven genomes whose answer is known, and how many of their genes AMAP finds, checked against tests/benchmark/README.md. A shared link to any page shows a card drawn from the front page's genes
More that the docs got wrong: the duplication page listed ten network metrics, the table has four, and its columns ending in 1 and 2 are the two communities of a pair, not two levels; Nextflow needs Java 17, not 11; --skip_functional_annotation leaves no GFF3 or GenBank, which come after eggNOG; tasks are capped at 16 GB on one machine and at 128 GB under a scheduler profile, not 50 GB; and lineage typing is not skipped on its own for other species, only Mycobrowser is
The tutorial's numbers are today's: its case counts and QC line, and their copies on the front page, the outputs page and the custom-reference page, came from runs in July. They now come from the tutorial's own commands run again on 18 October 2026 with --skip_functional_annotation, the QC line from GET_OUTPUTS' own script run on that run's corrected genes: CDC1551 has 4014 genes and 123 pseudogenes as AMAP calls them, where the old line counted Mycobrowser's 13, and Sakai keeps 1387 genes K-12 lacks, not 1356. The tutorial also says where --validate_only writes its verdict and where the number of reference proteins extracted is, as neither reaches the console
The README in the site's style: the metro map and the five outcomes in light and dark, the benchmark's scores, and the links a newcomer needs first
The README opens with the logo, and its drawing plays: the picture at the top was a still of the front page's drawing, dark whatever GitHub's theme. It now plays the drawing's four steps, in light or dark as the theme is, made by tools/pictures.mjs from the front page's own figure. The README's metro map gains the interactive report's station
One DOI: the README, CITATION.cff and every page of the website cited the DOI of one version on Zenodo, and the manifest, main.nf and the interactive report the concept DOI, which always resolves to the latest version. All of them cite the concept DOI now
Bakta's options on a SLURM cluster: the HPC page says how to give Bakta a site's options, since for Bakta the QoS the slurm profile sets takes the place of -process.clusterOptions
tests/selftest/h37rv_selftest.sh annotates H37Rv with itself inside the container and fails unless every GenBank CDS comes back exactly
tests/test_cache_keys.py fails when a process runs AMAP code that is not in its cache key, or when its script names a bin/ entry as a bare word without declaring it
tests/benchmark/benchmark.sh scores the correction algorithm on seven genomes whose right answer is known (H37Rv twice, CDC1551 complete and in 40 contigs, M. bovis, E. coli K-12 and Sakai), downloading them and running Bakta and miniprot in the container the first time, and lists every gene that changed since an earlier run. A gene whose sequence changed counts as found when a gene with its name covers half of its own bases there, not half of the stretch it lands on, which an insertion can make far longer (63 kb with a prophage inside for b1004 in Sakai); a gene mostly deleted is not counted. Changed genes found: 50 of 50 in CP110619, 860 of 862 in CDC1551, 1515 of 1515 in M. bovis and 3363 of 3367 in Sakai, and the README says why each of the other six is not
tests/test_name_by_lineage.py runs PATHOTYPR's own script, rendered as Nextflow renders it, with a stand-in for pathotypr that writes the summaries pathotypr wrote for CDC1551 complete and in 40 contigs, and fails unless Bakta would read a renamed genome as circular exactly when it reads the original so
tests/test_sample_ids.py runs TYPE_CASES, GET_OUTPUTS, GET_COUNT, GET_DUPLICATIONS, REPORT_INPUTS and GET_ALLDUP as Nextflow renders them, chained as the pipeline chains them, for a typed and an untyped run whose sample IDs hold underscores, and then the report's own R code where R is installed, and FINAL_REPORT's own script on a run's tables and on one that leaves every table it can empty
tests/test_validate_inputs.py runs VALIDATE_INPUTS's own script, rendered as Nextflow renders it, in a task directory laid out as Nextflow lays one out: genomes in a folder the task cannot see, relative paths, two genomes with the same file name and a sheet of 500 samples; and checks both databases, eggNOG's in each state EGGNOG_DB can leave it in
tests/test_eggnog_db.py runs EGGNOG_DB's own script, rendered as Nextflow renders it, with its folder staged as Nextflow stages an input and stand-ins for eggNOG-mapper's download and build: the database is built in that folder whatever it is called, one already there is used as it is, and a short bacteria.dmnd is built again
tests/test_eggnog_annotation.py runs ANNOT_EGGNOGMAP_BATCH's own script the same way, with a stand-in for emapper.py that looks for the database as eggNOG-mapper does: the database is found whatever its folder is called, a folder called split, where the script writes each sample's annotations, is left as it was, and the tag and log count a single sample as one
tests/test_reference_files.py runs the scripts of BAKTA_ANNOTATOR, PATHOTYPR, GET_OUTPUTS and BUNDLED_REFERENCE, rendered as Nextflow renders them, their own Groovy included, with stand-ins for Bakta and pathotypr and each reference file on the host, under a relative path or inside the container; and fails when a process reads a reference file's param other than as what it falls back on when nothing is staged; and checks the Mycobrowser table whenever a step reads it
tests/test_pangene_genomes.py runs PANGENE_GENOME and PANGENE's scripts as Nextflow renders them, with stand-ins for miniprot and pangene, and fails unless each sample is aligned onto every other sample's genome and never onto its own, typed or not
tests/test_report_tables.py merges two Rtabs as PANGENE writes them and checks that the report reads every table a # can reach without a comment character; where R is installed, as in the container, it also runs the report's pangenome code on the merged matrix
tests/test_cog_counts.py counts COG categories as count_annot.py does, for genes in none, in two and in lowercase, then runs the steps after the correction as tests/test_sample_ids.py chains them, with eggNOG giving some genes no category, then no gene one, then some samples no gene in a category another sample has genes in, and reads what the report's own overview, per-COG figures and statistics table show where R is installed: each category in a figure of its own, with every sample, at 0 where it has no gene. The Mycobrowser figures, statistics and ratios to the median are read the same way for a sample with no gene in one of their categories
tests/test_duplication_report.py runs GET_DUPLICATIONS and GET_ALLDUP as Nextflow renders them, chained as the pipeline chains them, with eggNOG giving one copy of a duplicated gene a category, another none and the last no row at all, and checks the columns and COG counts of homologous_communities.tsv; where R is installed, it knits the report's duplication summary on that table and on the one GET_ALLDUP writes when nothing is duplicated. It also fails when a chunk of the report prints markdown without passing it on as markdown
tests/test_profile_caps.py fails when a profile sets a cap, or when the order of -profile changes one: it reads nextflow.config and applies its profiles in every order, as both of Nextflow's parsers do, and with Nextflow on PATH also asks nextflow config
tests/test_qc_pseudogenes.py runs the steps after the correction as tests/test_sample_ids.py chains them, with FIX_CDS calling some genes pseudogenes and Mycobrowser marking another, with Mycobrowser on and off, and checks that *-AMAP.qc.tsv, REPORT_INPUTS and the report's own QC table count the pseudogenes AMAP called, as many as the GFF3 and the GenBank mark
tests/test_duplication_reproducible.py runs get_duplications.py and network_genomes.py under three hash seeds, on the Diamond tables of three genomes with a family whose average clustering changes with the order of its proteins, and fails unless every file they write is the same under all three; it also fails when either script opens a web browser, with hits or without, as a stand-in browser that records what it is asked to open tells, and when a network page asks a CDN for its drawing library
tests/test_custom_reference.py extracts a reference whose plasmid has a gene across its origin from a GenBank record and from GFF3 written either way, on either strand, compares Bakta's calls with it sequence by sequence, and runs PREPARE_REFERENCE's own script, rendered as Nextflow renders it, with a stand-in for Bakta that names sequences as Bakta does
tests/test_amap_report.py writes the report from four genomes of a real run cut to two stretches of the reference (tests/data/report/, rebuilt by its make_fixture.py), where M. bovis lacks RD4 and a contig end cuts dnaE2, plus a gene across the origin of a circular plasmid, and checks that every gene, and every Bakta call AMAP did not keep, comes back out of the page as it went in, with Python and, where node is installed, with the page's own JavaScript, along with the states, the flags, the correction's steps and what the process hands the script
tests/test_dump_params.py runs DUMP_PARAMS with Nextflow where it is installed, under both parsers, with values holding quotes, $, backticks, a backslash and a line break, and again with -resume and one value changed; everywhere, it fails if the step can come from the cache
tests/test_cluster_options.py reads which profile gives Bakta the QoS, and with Nextflow resolves Bakta's and every task's cluster options for each scheduler, in both orders with a container and under both parsers
tests/test_environment_pins.py fails when environment.yml pins a tool at a version docker/Dockerfile does not install
tests/test_citation.py fails when anything cites AMAP with a DOI other than the concept DOI of the manifest
tests/test_clean_proteome.py runs the steps after the correction as tests/test_sample_ids.py chains them, with FIX_CDS calling some genes pseudogenes, one with a stop codon in its protein and the others without, and Mycobrowser marking a gene AMAP finds intact, with Mycobrowser on and off, and checks that *-AMAP.clean.faa holds every protein FIX_CDS wrote but the pseudogenes', as it wrote them: the CDS the GFF3 does not mark pseudo=true, with the GenBank's translations
tests/test_report_pseudogenes.py runs the steps after the correction as tests/test_sample_ids.py chains them, with FIX_CDS calling a different number of pseudogenes in each sample, none in one, and Mycobrowser marking a gene that every sample has intact, with Mycobrowser on and off, and checks that REPORT_INPUTS hands over each master table row with its sample; where R is installed, it runs the report's own pseudogene section on what REPORT_INPUTS made and checks that it draws each sample at the number AMAP called, the QC table's, a sample with none at 0, under a title that does not name Mycobrowser
diff --git a/docs/assets/search-index.js b/docs/assets/search-index.js
index 532f563..e5e3ef9 100644
--- a/docs/assets/search-index.js
+++ b/docs/assets/search-index.js
@@ -4,7 +4,7 @@ window.AMAP_DOCS_INDEX = [
"page": "Tutorial",
"section": "",
"url": "getting-started/tutorial.html",
-"text": "Every command spelled out, nothing assumed. It uses a real genome you can download in one line, so you can follow along and compare your numbers against the ones printed here. Those numbers come from an actual run, not from memory. The genome is Mycobacterium tuberculosis CDC1551, corrected against the H37Rv reference that ships inside the AMAP container. At the end there is a second pass showing the same thing for a non-MTBC organism. The correction itself takes minutes on one genome. What dominates the first run is setup: pulling a 12 GB image and building the eggNOG database, both of which "
+"text": "Every command spelled out, nothing assumed. It uses a real genome you can download in one line, so you can follow along and compare your numbers against the ones printed here. Those numbers come from actual runs of these commands, not from memory, made on 18 October 2026 with AMAP at commit 947a99d. The runs skipped eggNOG, with --skip_functional_annotation. None of the numbers depends on it, and the one table shown here that is written after it was made by hand from the same run, as the sample check explains. The genome is Mycobacterium tuberculosis CDC1551, corrected against the H37Rv refere"
},
{
"page": "Tutorial",
@@ -40,7 +40,7 @@ window.AMAP_DOCS_INDEX = [
"page": "Tutorial",
"section": "5. Validate before committing to a long run",
"url": "getting-started/tutorial.html#5-validate-before-committing-to-a-long-run",
-"text": "nextflow run PathoGenOmics-Lab/AMAP -profile docker \\ --tsv samples.tsv --validate_only --- FASTA Validation (1 samples) --- ============================================================ Results: 1 OK, 0 warnings, 0 failed Total: 0 error(s), 0 warning(s) ============================================================ โ PASSED โ all 1 samples look good This costs seconds and catches the mistakes that would otherwise surface twenty minutes in. Preparing inputs lists every check and its threshold."
+"text": "nextflow run PathoGenOmics-Lab/AMAP -profile docker \\ --tsv samples.tsv --validate_only The checks write their verdict to results/pipeline_info/validation.log, which ends: tail -n 8 results/pipeline_info/validation.log --- FASTA Validation (1 samples) --- ============================================================ Results: 1 OK, 0 warnings, 0 failed Total: 0 error(s), 0 warning(s) ============================================================ โ PASSED โ all 1 samples look good This costs seconds and catches the mistakes that would otherwise surface minutes in. Preparing inputs lists eve"
},
{
"page": "Tutorial",
@@ -52,19 +52,19 @@ window.AMAP_DOCS_INDEX = [
"page": "Tutorial",
"section": "7. Look at what came out",
"url": "getting-started/tutorial.html#7-look-at-what-came-out",
-"text": "ls results/ AMAP/ BAKTA/ CDS_CASES/ DUPLICATIONS/ PANGENE/ PATHOTYPR/ pipeline_info/ final_report.html Open results/final_report.html first. It is self-contained, so it travels by email."
+"text": "ls -F results/ AMAP/ BAKTA/ CDS_CASES/ DUPLICATIONS/ PANGENE/ PATHOTYPR/ amap_report.html final_report.html pipeline_info/ A run without eggNOG, like the ones behind this page, has no DUPLICATIONS/, PANGENE/ or final_report.html. Open results/final_report.html first. It is self-contained, so it travels by email. results/amap_report.html goes through the same run question by question, and a run without eggNOG still gets it."
},
{
"page": "Tutorial",
"section": "Did the correction do anything?",
"url": "getting-started/tutorial.html#did-the-correction-do-anything",
-"text": "column -t -s$'\\t' results/CDS_CASES/summary/*-summary.tsv Type Number case2 2816 coordinates matched the reference exactly case3 24 coordinates matched, alignment length differed case4 1097 overlapped the reference but disagreed, resolved case1_initial 141 Bakta called it, the reference has nothing t Read that as: of 4077 Bakta calls, 2816 already agreed with the reference, 1097 had to be adjudicated, and 97 reference genes Bakta had missed were rescued. Only 11 of the 141 apparently novel genes survived scrutiny, which is what you would"
+"text": "column -t -s$'\\t' results/CDS_CASES/summary/*-summary.tsv Type Number case2 2813 coordinates matched the reference exactly case3 24 coordinates matched, alignment length differed case4 1100 overlapped the reference but disagreed, resolved case1_initial 141 Bakta called it, the reference has nothing t Read that as: of 4077 Bakta calls, 2813 already agreed with the reference, 1100 had to be adjudicated, and 97 reference genes Bakta had missed were rescued. Only 11 of the 141 apparently novel genes survived scrutiny, which is what you would"
},
{
"page": "Tutorial",
"section": "Sanity check the sample",
"url": "getting-started/tutorial.html#sanity-check-the-sample",
-"text": "column -t -s$'\\t' results/AMAP//-AMAP.qc.tsv genome_size n_contigs gc_pct total_cds pseudogene_count coding_density_pct 4403837 1 65.61 4020 13 91.5 65.6% GC and 91.5% coding density are what a healthy MTBC genome looks like. A number well outside that is worth investigating before trusting the rest."
+"text": "column -t -s$'\\t' results/AMAP//-AMAP.qc.tsv Six of its eleven columns: genome_size n_contigs gc_pct total_cds pseudogene_count coding_density_pct 4403837 1 65.61 4014 123 91.3 65.6% GC and 91.3% coding density are what a healthy MTBC genome looks like. A number well outside that is worth investigating before trusting the rest. This table is written after eggNOG, so a run without it has none. The line above was made by running that step's own script, GET_OUTPUTS, on the corrected genes of the run behind this page, with an eggNOG table that has no rows. "
},
{
"page": "Tutorial",
@@ -400,7 +400,7 @@ window.AMAP_DOCS_INDEX = [
"page": "Outputs",
"section": "What a healthy summary looks like",
"url": "user-guide/outputs.html#what-a-healthy-summary-looks-like",
-"text": "Two real runs, to calibrate expectations. On the left, M. tuberculosis CDC1551 against the bundled H37Rv reference. On the right, E. coli O157:H7 Sakai against K-12 MG1655, using a custom reference. case2, exact match 2816 3325 case3, length differs 24 34 case4, resolved conflict 1097 537 case1_kept, genuinely novel 11 1356 rv_missing, rescued 97 102 total_cds before correction 4077 5367 The one number that differs by two orders of magnitude is case 1, and that is biology: CDC1551 is the same species as the reference so almost nothing is genuinely new, while O157:H7 carries prophages and the L"
+"text": "Two real runs, those of the tutorial, to calibrate expectations. On the left, M. tuberculosis CDC1551 against the bundled H37Rv reference. On the right, E. coli O157:H7 Sakai against K-12 MG1655, using a custom reference. case2, exact match 2813 3324 case3, length differs 24 31 case4, resolved conflict 1100 507 case1_kept, genuinely novel 11 1387 rv_missing, rescued 97 133 total_cds before correction 4077 5367 The one number that differs by two orders of magnitude is case 1, and that is biology: CDC1551 is the same species as the reference so almost nothing is genuinely new, while O157:H7 carr"
},
{
"page": "Outputs",
@@ -574,7 +574,7 @@ window.AMAP_DOCS_INDEX = [
"page": "Custom reference",
"section": "What to expect from a real run",
"url": "user-guide/custom-reference.html#what-to-expect-from-a-real-run",
-"text": "E. coli O157:H7 Sakai corrected against K-12 MG1655, both straight from RefSeq: reference proteins extracted from the K-12 GenBank 4318 CDS Bakta called on that same reference genome 4296 Bakta false positives derived from the difference 142 case2 3325 case3 34 case4 The 142 are the automatically derived equivalent of the hand-curated MTBC false-positive list, and 1356 case 1 genes is what you would expect between two E. coli strains that genuinely differ. If your case 2 comes out a"
+"text": "E. coli O157:H7 Sakai corrected against K-12 MG1655, both straight from RefSeq, in the tutorial's run: reference proteins extracted from the K-12 GenBank 4318 CDS Bakta called on that same reference genome 4296 Bakta false positives derived from the difference 142 case2 3324 case3 31 case4 The 142 are the automatically derived equivalent of the hand-curated MTBC false-positive list, and 1387 case 1 genes is what you would expect between two E. coli strains that genuinely differ. If "
},
{
"page": "Custom reference",
diff --git a/docs/getting-started/tutorial.html b/docs/getting-started/tutorial.html
index ec23107..e607583 100644
--- a/docs/getting-started/tutorial.html
+++ b/docs/getting-started/tutorial.html
@@ -90,7 +90,7 @@
Tutorial: from an empty folder to an annotated genome
-
Every command spelled out, nothing assumed. It uses a real genome you can download in one line, so you can follow along and compare your numbers against the ones printed here. Those numbers come from an actual run, not from memory.
+
Every command spelled out, nothing assumed. It uses a real genome you can download in one line, so you can follow along and compare your numbers against the ones printed here. Those numbers come from actual runs of these commands, not from memory, made on 18 October 2026 with AMAP at commit 947a99d. The runs skipped eggNOG, with --skip_functional_annotation. None of the numbers depends on it, and the one table shown here that is written after it was made by hand from the same run, as the sample check explains.
The genome is Mycobacterium tuberculosis CDC1551, corrected against the H37Rv reference that ships inside the AMAP container. At the end there is a second pass showing the same thing for a non-MTBC organism.
The correction itself takes minutes on one genome. What dominates the first run is setup: pulling a 12 GB image and building the eggNOG database, both of which happen once and are reused afterwards. Expect the first run to take considerably longer than every run after it, and do not read a long silence during the database build as a failure.
nextflow run PathoGenOmics-Lab/AMAP -profile docker --tsv samples.tsv
What happens, in order: lineage typing, Bakta annotation, alignment of the H37Rv proteome onto your genome, case classification, correction, eggNOG functional annotation, duplication analysis, pangenome, report.
A run without eggNOG, like the ones behind this page, has no DUPLICATIONS/, PANGENE/ or final_report.html.
+
Open results/final_report.html first. It is self-contained, so it travels by email. results/amap_report.html goes through the same run question by question, and a run without eggNOG still gets it.
Type Number
-case2 2816 coordinates matched the reference exactly
+case2 2813 coordinates matched the reference exactly
case3 24 coordinates matched, alignment length differed
-case4 1097 overlapped the reference but disagreed, resolved
+case4 1100 overlapped the reference but disagreed, resolved
case1_initial 141 Bakta called it, the reference has nothing there
rv_missing 97 the reference has a gene there, Bakta missed it
total_cds 4077 what Bakta called before correction
case1_removed 130 "novel" genes that were not really novel
case1_kept 11 kept as genuinely absent from H37Rv
-
CDC1551 against H37Rv4,077 Bakta calls
2,8161,097
case 2 2,816case 3 24case 4 1,097case 1, dropped 130case 1, kept 11missing, put back 97
The same numbers, drawn: most of Bakta's calls agree with H37Rv, about a quarter overlap it with other coordinates and are settled, and very few are new.
-
Read that as: of 4077 Bakta calls, 2816 already agreed with the reference, 1097 had to be adjudicated, and 97 reference genes Bakta had missed were rescued. Only 11 of the 141 apparently novel genes survived scrutiny, which is what you would expect for a strain of the same species as the reference.
+
CDC1551 against H37Rv4,077 Bakta calls
2,8131,100
case 2 2,813case 3 24case 4 1,100case 1, dropped 130case 1, kept 11missing, put back 97
The same numbers, drawn: most of Bakta's calls agree with H37Rv, about a quarter overlap it with other coordinates and are settled, and very few are new.
+
Read that as: of 4077 Bakta calls, 2813 already agreed with the reference, 1100 had to be adjudicated, and 97 reference genes Bakta had missed were rescued. Only 11 of the 141 apparently novel genes survived scrutiny, which is what you would expect for a strain of the same species as the reference.
65.6% GC and 91.5% coding density are what a healthy MTBC genome looks like. A number well outside that is worth investigating before trusting the rest.
+4403837 1 65.61 4014 123 91.3
+
65.6% GC and 91.3% coding density are what a healthy MTBC genome looks like. A number well outside that is worth investigating before trusting the rest.
+
This table is written after eggNOG, so a run without it has none. The line above was made by running that step's own script, GET_OUTPUTS, on the corrected genes of the run behind this page, with an eggNOG table that has no rows. None of the six columns depends on eggNOG; the one that does, eggnog_annotation_pct, is left out.
--skip_typing because lineage typing is specific to tuberculosis, and --skip_mycobrowser because those functional categories are an MTBC resource.
Before annotating anything, AMAP extracts the proteome from your reference, annotates that same reference genome with Bakta, and takes the difference to work out which genes Bakta invents. For MTBC that list was curated by hand; here it is derived automatically. On this pair it found 142.
The case counts come out differently from the MTBC run, and the difference is biology rather than a problem:
-
case2 3325
-case3 34
-case4 537
-case1_kept 1356 genes O157:H7 has and K-12 does not
-rv_missing 102
-
Those 1356 are the prophages and the LEE island that O157:H7 carries and K-12 lacks. Against a reference of your own species you should expect a large case 1 when the strains genuinely differ, and a small one when they do not.
-
The annotation and the genome must be the same assembly
Coordinates from the annotation are used directly against the FASTA. An annotation from a different assembly produces nonsense: proteins that do not translate, and a nearly empty reference proteome. AMAP prints how many proteins it extracted, so check the number looks like a genome's worth.
+
case2 3324
+case3 31
+case4 507
+case1_kept 1387 genes O157:H7 has and K-12 does not
+rv_missing 133
+
Those 1387 are the prophages and the LEE island that O157:H7 carries and K-12 lacks. Against a reference of your own species you should expect a large case 1 when the strains genuinely differ, and a small one when they do not.
+
The annotation and the genome must be the same assembly
Coordinates from the annotation are used directly against the FASTA. An annotation from a different assembly produces nonsense: proteins that do not translate, and a nearly empty reference proteome. AMAP writes how many proteins it extracted into the work directory, where grep -h 'Curated proteins' work/*/*/.command.out finds it (4318 on this pair), so check the number looks like a genome's worth.
Custom reference goes into what makes a good reference and what changes when you use one.
Nextflow prints the work directory of the failed task. Everything you need is in it:
diff --git a/docs/getting-started/tutorial.md b/docs/getting-started/tutorial.md
index ab7ba7b..4ba2837 100644
--- a/docs/getting-started/tutorial.md
+++ b/docs/getting-started/tutorial.md
@@ -2,7 +2,12 @@
Every command spelled out, nothing assumed. It uses a real genome you can
download in one line, so you can follow along and compare your numbers against
-the ones printed here. Those numbers come from an actual run, not from memory.
+the ones printed here. Those numbers come from actual runs of these commands,
+not from memory, made on 18 October 2026 with AMAP at commit `947a99d`. The
+runs skipped eggNOG, with `--skip_functional_annotation`. None of the numbers
+depends on it, and the one table shown here that is written after it was made
+by hand from the same run, as [the sample check](#sanity-check-the-sample)
+explains.
The genome is *Mycobacterium tuberculosis* CDC1551, corrected against the H37Rv
reference that ships inside the AMAP container. At the end there is a second
@@ -104,9 +109,10 @@ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
```
You still get the corrected genes, which is what AMAP exists to produce, as a
-table with their proteins and sequences. What you lose is eggNOG categories, the
-GFF3 and GenBank files written after it, the pangenome, the duplication analysis
-and the report. The heaviest step left asks for 6 GB.
+table with their proteins and sequences, and `amap_report.html`. What you lose
+is eggNOG categories, the GFF3 and GenBank files written after it, the
+pangenome, the duplication analysis and `final_report.html`. The heaviest step
+left asks for 6 GB.
> [!CAUTION]
> **Docker Desktop has its own memory limit, and it is the one that counts**
@@ -198,6 +204,13 @@ nextflow run PathoGenOmics-Lab/AMAP -profile docker \
--tsv samples.tsv --validate_only
```
+The checks write their verdict to `results/pipeline_info/validation.log`, which
+ends:
+
+```bash
+tail -n 8 results/pipeline_info/validation.log
+```
+
```text
--- FASTA Validation (1 samples) ---
@@ -209,7 +222,7 @@ Total: 0 error(s), 0 warning(s)
โ PASSED โ all 1 samples look good
```
-This costs seconds and catches the mistakes that would otherwise surface twenty
+This costs seconds and catches the mistakes that would otherwise surface
minutes in. [Preparing inputs](inputs.md) lists every check and its threshold.
## 6. Run it
@@ -236,16 +249,20 @@ If the run dies partway, `-resume` picks up from the last completed step.
## 7. Look at what came out
```bash
-ls results/
+ls -F results/
```
```text
AMAP/ BAKTA/ CDS_CASES/ DUPLICATIONS/ PANGENE/ PATHOTYPR/
-pipeline_info/ final_report.html
+amap_report.html final_report.html pipeline_info/
```
+A run without eggNOG, like the ones behind this page, has no `DUPLICATIONS/`,
+`PANGENE/` or `final_report.html`.
+
Open `results/final_report.html` first. It is self-contained, so it travels by
-email.
+email. `results/amap_report.html` goes through the same run question by
+question, and a run without eggNOG still gets it.
### Did the correction do anything?
@@ -255,9 +272,9 @@ column -t -s$'\t' results/CDS_CASES/summary/*-summary.tsv
```text
Type Number
-case2 2816 coordinates matched the reference exactly
+case2 2813 coordinates matched the reference exactly
case3 24 coordinates matched, alignment length differed
-case4 1097 overlapped the reference but disagreed, resolved
+case4 1100 overlapped the reference but disagreed, resolved
case1_initial 141 Bakta called it, the reference has nothing there
rv_missing 97 the reference has a gene there, Bakta missed it
total_cds 4077 what Bakta called before correction
@@ -267,8 +284,8 @@ case1_kept 11 kept as genuinely absent from H37Rv
-Read that as: of 4077 Bakta calls, 2816 already agreed with the reference,
-1097 had to be adjudicated, and 97 reference genes Bakta had missed were
+Read that as: of 4077 Bakta calls, 2813 already agreed with the reference,
+1100 had to be adjudicated, and 97 reference genes Bakta had missed were
rescued. Only 11 of the 141 apparently novel genes survived scrutiny, which is
what you would expect for a strain of the same species as the reference.
@@ -278,14 +295,22 @@ what you would expect for a strain of the same species as the reference.
column -t -s$'\t' results/AMAP/*/*-AMAP.qc.tsv
```
+Six of its eleven columns:
+
```text
genome_size n_contigs gc_pct total_cds pseudogene_count coding_density_pct
-4403837 1 65.61 4020 13 91.5
+4403837 1 65.61 4014 123 91.3
```
-65.6% GC and 91.5% coding density are what a healthy MTBC genome looks like. A
+65.6% GC and 91.3% coding density are what a healthy MTBC genome looks like. A
number well outside that is worth investigating before trusting the rest.
+This table is written after eggNOG, so a run without it has none. The line
+above was made by running that step's own script, GET_OUTPUTS, on the
+corrected genes of the run behind this page, with an eggNOG table that has no
+rows. None of the six columns depends on eggNOG; the one that does,
+`eggnog_annotation_pct`, is left out.
+
### The files you will actually use
Lineage typing puts the lineage in front of the sample ID, so CDC1551 comes
@@ -375,14 +400,14 @@ The case counts come out differently from the MTBC run, and the difference is
biology rather than a problem:
```text
-case2 3325
-case3 34
-case4 537
-case1_kept 1356 genes O157:H7 has and K-12 does not
-rv_missing 102
+case2 3324
+case3 31
+case4 507
+case1_kept 1387 genes O157:H7 has and K-12 does not
+rv_missing 133
```
-Those 1356 are the prophages and the LEE island that O157:H7 carries and K-12
+Those 1387 are the prophages and the LEE island that O157:H7 carries and K-12
lacks. Against a reference of your own species you should expect a large case 1
when the strains genuinely differ, and a small one when they do not.
@@ -391,8 +416,10 @@ when the strains genuinely differ, and a small one when they do not.
>
> Coordinates from the annotation are used directly against the FASTA. An
> annotation from a different assembly produces nonsense: proteins that do
-> not translate, and a nearly empty reference proteome. AMAP prints how many
-> proteins it extracted, so check the number looks like a genome's worth.
+> not translate, and a nearly empty reference proteome. AMAP writes how many
+> proteins it extracted into the work directory, where
+> `grep -h 'Curated proteins' work/*/*/.command.out` finds it (4318 on this
+> pair), so check the number looks like a genome's worth.
[Custom reference](../user-guide/custom-reference.md) goes into what makes a
good reference and what changes when you use one.
diff --git a/docs/index.html b/docs/index.html
index 97abc24..89654f4 100644
--- a/docs/index.html
+++ b/docs/index.html
@@ -332,9 +332,9 @@
Files you can open, and the reasons behind them
$ column -t -s$'\t' results/CDS_CASES/summary/*-summary.tsv
-case2 2816 coordinates matched the reference exactly
+case2 2813 coordinates matched the reference exactly
case3 24 coordinates matched, alignment length differed
-case4 1097 overlapped the reference but disagreed, resolved
+case4 1100 overlapped the reference but disagreed, resolved
case1_initial 141 Bakta called it, the reference has nothing there
rv_missing 97 the reference has a gene there, Bakta missed it
total_cds 4077 what Bakta called before correction
diff --git a/docs/user-guide/custom-reference.html b/docs/user-guide/custom-reference.html
index 932b0d6..4062578 100644
--- a/docs/user-guide/custom-reference.html
+++ b/docs/user-guide/custom-reference.html
@@ -115,18 +115,18 @@
Everything else. Case classification, conflict resolution, missing-gene rescue, the stop codon check, eggNOG functional annotation, duplication analysis, the pangenome and the report all work identically. The report simply omits the lineage and Mycobrowser sections, and the pangenome calls each genome by its sample ID where a typed run would use its lineage name.
E. coli O157:H7 Sakai corrected against K-12 MG1655, both straight from RefSeq:
+
E. coli O157:H7 Sakai corrected against K-12 MG1655, both straight from RefSeq, in the tutorial's run:
reference proteins extracted from the K-12 GenBank 4318
CDS Bakta called on that same reference genome 4296
Bakta false positives derived from the difference 142
-case2 3325
-case3 34
-case4 537
-case1_kept 1356
-rv_missing 102
-final annotation 5244 genes
-
The 142 are the automatically derived equivalent of the hand-curated MTBC false-positive list, and 1356 case 1 genes is what you would expect between two E. coli strains that genuinely differ.
The 142 are the automatically derived equivalent of the hand-curated MTBC false-positive list, and 1387 case 1 genes is what you would expect between two E. coli strains that genuinely differ.
If your case 2 comes out at or near zero, stop and look at the reference rather than the sample. Every gene missing an exact match by a constant offset points at the reference, not at 4000 coincidences.
diff --git a/docs/user-guide/custom-reference.md b/docs/user-guide/custom-reference.md
index cf48bcd..0b45477 100644
--- a/docs/user-guide/custom-reference.md
+++ b/docs/user-guide/custom-reference.md
@@ -97,23 +97,23 @@ sample ID where a typed run would use its lineage name.
## What to expect from a real run
*E. coli* O157:H7 Sakai corrected against K-12 MG1655, both straight from
-RefSeq:
+RefSeq, in [the tutorial's run](../getting-started/tutorial.md#9-the-same-thing-for-another-organism):
```text
reference proteins extracted from the K-12 GenBank 4318
CDS Bakta called on that same reference genome 4296
Bakta false positives derived from the difference 142
-case2 3325
-case3 34
-case4 537
-case1_kept 1356
-rv_missing 102
-final annotation 5244 genes
+case2 3324
+case3 31
+case4 507
+case1_kept 1387
+rv_missing 133
+final annotation 5335 genes
```
The 142 are the automatically derived equivalent of the hand-curated MTBC
-false-positive list, and 1356 case 1 genes is what you would expect between two
+false-positive list, and 1387 case 1 genes is what you would expect between two
*E. coli* strains that genuinely differ.
If your case 2 comes out at or near zero, stop and look at the reference rather
diff --git a/docs/user-guide/outputs.html b/docs/user-guide/outputs.html
index bd4f360..8475c87 100644
--- a/docs/user-guide/outputs.html
+++ b/docs/user-guide/outputs.html
@@ -182,9 +182,9 @@
The audit trail for the correction. Each case table lists the genes that fell into it, so you can see exactly what was changed and why. The correction algorithm explains the cases; *-summary.tsv aggregates the counts across samples and feeds the report. Its last three columns say which sample the counts are for: Sequence is the name its files carry (L4_G26689), Lineage what typing called it (Untyped under --skip_typing) and Sample its ID in the sample sheet (G26689).
Two real runs, to calibrate expectations. On the left, M. tuberculosis CDC1551 against the bundled H37Rv reference. On the right, E. coli O157:H7 Sakai against K-12 MG1655, using a custom reference.
-
Case
CDC1551 vs H37Rv
Sakai vs K-12
case2, exact match
2816
3325
case3, length differs
24
34
case4, resolved conflict
1097
537
case1_kept, genuinely novel
11
1356
rv_missing, rescued
97
102
total_cds before correction
4077
5367
-
CDC1551 vs H37Rv4,077 Bakta calls
2,8161,097
case 2 2,816case 3 24case 4 1,097case 1, kept 11missing, put back 97
Sakai vs K-125,367 Bakta calls
3,3255371,356
case 2 3,325case 3 34case 4 537case 1, kept 1,356missing, put back 102
The same table, drawn. The number that differs by two orders of magnitude is case 1: the prophages and the LEE island that O157:H7 carries and K-12 does not.
+
Two real runs, those of the tutorial, to calibrate expectations. On the left, M. tuberculosis CDC1551 against the bundled H37Rv reference. On the right, E. coli O157:H7 Sakai against K-12 MG1655, using a custom reference.
+
Case
CDC1551 vs H37Rv
Sakai vs K-12
case2, exact match
2813
3324
case3, length differs
24
31
case4, resolved conflict
1100
507
case1_kept, genuinely novel
11
1387
rv_missing, rescued
97
133
total_cds before correction
4077
5367
+
CDC1551 vs H37Rv4,077 Bakta calls
2,8131,100
case 2 2,813case 3 24case 4 1,100case 1, kept 11missing, put back 97
Sakai vs K-125,367 Bakta calls
3,3245071,387
case 2 3,324case 3 31case 4 507case 1, kept 1,387missing, put back 133
The same table, drawn. The number that differs by two orders of magnitude is case 1: the prophages and the LEE island that O157:H7 carries and K-12 does not.
The one number that differs by two orders of magnitude is case 1, and that is biology: CDC1551 is the same species as the reference so almost nothing is genuinely new, while O157:H7 carries prophages and the LEE island that K-12 does not have.
What should worry you is the opposite: case 2 at or near zero. Every gene disagreeing with the reference by a few bases means something systematic is wrong with the reference, not with 4000 individual genes. The usual cause is a reference proteome whose coordinates do not describe the sequence its proteins were translated from.
diff --git a/docs/user-guide/outputs.md b/docs/user-guide/outputs.md
index 10ee8d6..c38a154 100644
--- a/docs/user-guide/outputs.md
+++ b/docs/user-guide/outputs.md
@@ -161,17 +161,18 @@ is the name its files carry (`L4_G26689`), `Lineage` what typing called it
#### What a healthy summary looks like
-Two real runs, to calibrate expectations. On the left, *M. tuberculosis*
-CDC1551 against the bundled H37Rv reference. On the right, *E. coli* O157:H7
-Sakai against K-12 MG1655, using a custom reference.
+Two real runs, those of the [tutorial](../getting-started/tutorial.md), to
+calibrate expectations. On the left, *M. tuberculosis* CDC1551 against the
+bundled H37Rv reference. On the right, *E. coli* O157:H7 Sakai against K-12
+MG1655, using a custom reference.
| Case | CDC1551 vs H37Rv | Sakai vs K-12 |
| :--- | ---: | ---: |
-| case2, exact match | 2816 | 3325 |
-| case3, length differs | 24 | 34 |
-| case4, resolved conflict | 1097 | 537 |
-| case1_kept, genuinely novel | 11 | 1356 |
-| rv_missing, rescued | 97 | 102 |
+| case2, exact match | 2813 | 3324 |
+| case3, length differs | 24 | 31 |
+| case4, resolved conflict | 1100 | 507 |
+| case1_kept, genuinely novel | 11 | 1387 |
+| rv_missing, rescued | 97 | 133 |
| total_cds before correction | 4077 | 5367 |